Marco has come over to the Netherlands to pay me a visit, and to hack a little bit together, in person. So with the weather clearly suggesting to stay inside, that’s what we did over the weekend, and how better to entertain yourself than to work on mobile software?
Marco has been working for a while on components that follow Plasma’s human interface guidelines and make it easy to implement applications with a common navigation pattern and look and feel. Obviously, these components use a lot of Plasma under the hood, so they get excellent integration at a visual and at a technical level. This high integration, however, comes at the price of having a non-trivial chain of dependencies. That’s not a problem in Plasma Mobile, or other Plasma workspaces, since all that is already there, anyway.
We thought that an interesting exercise would be to find out what really defines a “Plasma application”, and how we can make the concepts we engrained in their design available to application developers more easily. How hard could it be to use Plasma components in an Android app, for example? The answer is, not entirely trivial, but it just became a whole lot easier. So what did we do?
For those reading this article via a feed aggregator, hop over to youtube to watch the demo video.
We took Subsurface, which is a piece of Free software used for logging and analysing scuba dives. Subsurface has a mobile version, which is still in its infancy, so it’s an excellent candidate to experiment with. We also took Marco’s set of Plasma components that provide a reduced set of functionality, in fact, just enough to create what most applications will need. These components extend QtQuick components where we found them lacking. They’re very light weight, carry no dependencies other than QtQuick, and they’re entirely written in QML, so basically, you add a bunch of QML files to your app and concentrate on what makes your app great, not on overall navigation components or re-implementing for the n-th time a set of widgets.
So after solving some deployment issues, on Saturday night, we had the Plasma mobile components loading in an Android app. A first success. Running the app did show a number of problems, however, so we spent most of the Sunday to look into each problem one by one and trying to solve them. By early Monday morning, we had all of the glaring issues we found during our testing solved, and we got Subsurface mobile to a pretty solid appearance (pretty solid given its early state of development, not bug free by any means).
So, what to take a away from this? In a reduced form, Plasma can be a huge help to create also Android applications. The mobile components which we’re developing with Plasma Mobile as target in mind have had their first real world exposure and a lot of fixes, we got very useful feedback from the Subsurface community which we’re directly feeding back into our components.
A big thanks goes out to the Subsurface team and especially Dirk Hohndel for giving us excellent and timely feedback, for being open to our ideas and for willing to play guinea pig for the Plasma HIG and our components. The state you can see in the above video has already been reviewed and merged into Subsurface’s master tree, so divers around the world will be able to enjoy it when the app becomes available for a wider audience.
That moment when the application “just works” after all your unit tests pass…
A really nice experience after working on these low-levelbits was firing up the kscreen systemsettings module configured to use my wayland test server. I hadn’t done so in a while, so I didn’t expect much at all. The whole thing just worked right out of the box, however. Every single change I’ve tried had exactly the expected effect.
This screenshot shows Plasma’s screen configuration settings (“kscreen”). The settings module uses the new kwayland backend to communicate with a wayland server (which you can see “running” on the left hand side). That means that another big chunk of getting Plasma Wayland-ready for multi-display use-cases is falling nicely into place.
I’m working on this part of the stack using test-driven development methods, so I write unit tests for every bit of functionality, and then implement and polish the library parts. Something is done when all units tests pass reliably, when others have reviewed the code, when everything works in on the application side, and when I am happy with it.
The unit tests stay in place and are from then on compiled and run through our continuous integration system automatically on every code change. This system yells at us as soon as any of the unit tests breaks or shows problems, so we can fix it right away.
Interestingly, we run the unit tests live against a real wayland server. This test server is implemented using the KWayland library. The server runs headless, so it doesn’t do any rendering of windows, and it just implements the bits interesting for screen management. It’s sort of a mini kwin_wayland, the real kwin will use this exact same library on the server side, so our tests are not entirely synthetical. This wasn’t really possible for X11-based systems, because you can’t just fire up an X server that supports XRandR in automated tests — the machine running the test may not allow you to use its graphics card, if it even has one. It’s very easy to do, however, when using wayland.
Our autotests fire up a wayland server from one of many example configurations. We have a whole set of example configurations that we run tests against, and it’s easy to add more that we want to make sure work correctly. (I’m also thinking about user support, where we can ask to send us a problematic configuration written out to a json file, that we can then add to our unit tests, fix, and ensure that it never breaks again.
The wayland test server is only about 500 lines of relatively simple code, but it provides full functionality for setting up screens using the wayland protocol.
The real kwin_wayland will use the exact same library, on the server as we do in our tests, but instead of using “virtual screens”, it does actually interact with the hardware, for example through libdrm on more sensible system or through libhybris on ones less so.
Kwin takes a more central role in our wayland story, as we move initial mode-setting there, it just makes to have it do run-time mode setting as well.
The next steps are to hook the server side of the protocol up in kwin_wayland’s hardware backends.
In the back of my head are a few new features, which so far had a lower priority — first the core feature set needed to be made stable. There are three things which I’d like to see us doing:
per-display scaling — This is an interesting one. I’d love to be able to specify a floating point scaling factor. Wayland’s wl_output interface, which represents the application clients, only provides int-precision. I think that sucks since there is a lot of hardware around where a scaling factor of 1 is to small, and 2 is too high. That’s pretty much everything between 140 and 190 DPI according to my eyesight, your mileage may vary here. I’m wondering if I should go ahead and add the necessary API’s at least on our end of the stack to allow better than integer precision.
Also, of course we want the scaling be controlled per display (and not globally for all displays, as it is on X11), but that’s in fact already solved by just using wayland semantics — it needs to be fixed on the rendering side now.
pre-apply checks — at least the drm backend will allow us to ask it if it will be able to apply a new configuration to the hardware. I’d love to hook that up to the UI, so we can do things like enable or disable the apply button, and warn the user of something that the hardware is not going to like doing.
The low-level bits have arrived with the new drm infrastructure in the kernel, so we can hook it up in the libraries and the user interface.
configuration profiles — it would make sense to allow the user to save configurations for different situations and pick between them. It would be quite easy to allow the user to switch between setups not just through the systemsettings ui, but also for example when connecting or disabling a screen. I an imagine that this could be presented very nicely, and in tune with graphical effects that get their timing juuuuust right when switching between graphics setups. Let’s see how glitch-free we can make it.
So, first of all, this is all very much work-in-progress and highly experimental. It’s related to the work on screen management which I’ve outlined in an earlier article.
I ran a few benchmarks across our wayland stack, especially measuring interprocess communication performance when switching from X11 (or, in fact XCB and XRandR) to wayland. I haven’t done a highly scientific setup, just ran the same code with different backends to see how long it takes to receive information about screens connected, their modes, etc..
I also ran the numbers when loading the libkscreen backend in-process, more on that later.
The spreadsheet shows three data columns, in vertical blocks per backend the results for 4-5 individual runs and their mean values. One column for the default out-of-process mode, one with loading the backend in process and one showing the speedup between in- and out-of-process of the same backend.
The lower part contains some cross referencing of the mean values to compare different setups.
All values are nano seconds.
My results show a speedup of between 2 and 2.5 times when querying screen information on X11 and on wayland, wayland being much faster here.
The qscreen and xrandr backends perform pretty similar, they’re both going through XCB. That checks out. The difference between wayland and xrandr/qscreen can then be attributed to either the wayland protocol or its implementation in KWayland being much faster than the corresponding XCB implementations.
But, here’s the kicker…
in- vs. out-of-process
The main overhead, as it turns out, is libkscreen loading the backend plugins out-of-process. That means that it starts a dbus-autolaunched backend process and then passes data over DBus between the libkscreen front-end API and the backend plugin. It’s done that way to shield the client API (for example the plasma shell process or systemsettings) from unsafe calls into X11, as it encapsulates some crash-prone code in the XRandR backend. When using the wayland backend, this is not necessary, as we’re using KWayland, which is much safer.
I went ahead and instrumented libkscreen in a way that these backends are being loaded in process, which avoids most of the overhead. This change has an even more dramatic influence on performance: on X11, the speedup is 1.6x – 2x, on wayland loading the backend in-process makes it run 10 times faster. Of course, these speedups are complementary, so combined, querying screen information on wayland can be done about 20 times faster.
While this change from out-of-process to in-process backends introduces a bit more complexity in the library, it has a couple of other advantages additional to the performance gains. In-process means that debugging is much easier. If there are crashes, we do not hide them anymore, but identify and fix them. It also makes development more worthwhile, since it’s much easier to debug and test the backends and frontend API together. It also means that we can load backend plugins at the same time.
I’ve uploaded the benchmark data here. Before merging this, I’ll have to iron out some more wrinkles and have the code reviewed, so it’s not quite ready for prime-time yet.
One of the bigger things that is in the works in Plasma’s Wayland support is screen management. In most cases, that is reasonably easy, there’s one screen and it has a certain resolution and refresh rate set. For mobile devices, this is almost always good enough. Only once we starting thinking about convergence and using the same codebase on different devices, we need to be able to configure the screens used for rendering. Especially on desktops and laptops, where we often find multi-monitor setups or connected projectors is where the user should be able to decide a bunch of things, relative position of the screens, resolution (“mode”) for each, etc.. Another thing that we haven’t touched yet is scaling of the rendering per display, which becomes increasingly important with a wider range of displays connected, just imagine a 4K laptop running north of 300 pixels per inch (PPI) connected to a projector which throws 1024*768 pixels on a wall sized 4x3m.
The Wayland protocol currently does not provide a mechanism for setting up the screen, or tell us about displays that are not used for rendering, either because they’re disabled, or have just been connected, but not enabled “automatically” (yet). For most applications, that doesn’t matter, they’re just fine with knowing about the rendering screens and some details about those, which is provided by the wl_output interface. For screen management, this interface is insufficient, though, since it lacks a few things, EDID information, enabled/disabled flags and also ways to set the mode, scaling, rotation and position. This makes clearly recognizing display and setting them up harder than necessary, and thus error-prone. Let’s look at the background, first, however.
Setting up X11
On the hardware side, this has been a complete mess in the past. One problem is X11’s asynchronous nature. The XRandR extension that is used for this basically works by throwing a bunch of calls to the X server (“use this mode”, “position display there”) and then seeing what sticks to the wall. The problem is that we never really know what happened, there’s no well-defined “OK, this works” result, and we also don’t know when the whole procedure is done. The result is a flicker-fest and the desktop trying to catch up with what X11 made of the xrandr calls. It can also be an unpleasant experience, when a display gets connected, used for rendering, then the shell finds out about it, expanding the desktop to it, and then everything is resized again because there’s a predefined configuration for this. These kind of race conditions are very hard to fix due to the number of components involved in the process, and the lack of proper notification semantics around it.
X11 has the nasty habit of interacting with hardware directly, rather than through well-defined and modern kernel interfaces. On the kernel side, this has been fixed. We now have atomic mode setting, which allows us to check whether changes can be applied (through the DRM_MODE_ATOMIC_TEST_ONLY flag), and apply them all at once, or in portions that are known to not screw up, lock the user out, or are simply invalid in context with each other.
For the user, getting this right across the whole stack means quicker reconfiguration of the hardware and only minimal flickering when switching screen setups. We won’t be able to completely prevent the flickering on most displays, as that is simply how the hardware works, but we will be able to make it a lot less jarring. The compositor now being the one that calls the DRM subsystem on the user side, we can coordinate these things well with visual effects, so we’ll be able to make the user experience while re-configuring displays a bit smoother as well.
Atomic mode setting, DRM and kernel
From the kernel side, this needed quite radical changes, which have now landed throughout the DRM subsystem. The result is a kernel interface and helper library that allows interacting with the kernel using semantics that allow tighter control of the processes, better error prevention and handling and more modern power management semantics. Switching off the screen can now be done from the compositor, for example — this allows us to fix those cases where the display is black, but still has its backlight on, or where the display is off, but used for rendering (in which case you get huge blind spots in your user interface).
Daniel Vetter’s (part 1, part 2) provides an excellent overview over history, present and future of atomic mode setting on the kernel side. Pertaining is that a reasonably recent Linux kernel with working DRM drivers now provides all that we need to fix this problem on the user side. X11 is still in the way of a completely smooth solution, though.
Screen setup in Plasma
In Plasma, the screens can be set up using the Display configuration module in system settings. This module is internally called “KScreen”. KScreen provides a visual interface to position displays, set resolution, etc.. It’s backed by a daemon that can apply a configuration on login – useful stuff, but ultimately bound by the limits across the underlying software stack (X11, kernel, drivers, etc.).
KScreen is backed by libkscreen, a library that we ship with Plasma. libkscreen offers an API that allows to list displays, their properties, including disabled displays. libkscreen is driven by out-of-process running backends, commonly used is the “xrandr” backend, which talks to the X Server over the XRandR extension. libkscreen has other backends, notably a read-only QScreen backend a “fake” backend used for unit tests. A native Wayland backend is work in progress (you can find it in the libkscreen[sebas/wayland] branch.)
libkscreen been developed for the purpose of screen configuration, but we have also started using it for the Plasma shell. QScreen, the natural starting point of this was not up to the task yet, missing some functionality. In Qt 5.6, Aleix Pol has now landed the necessary missing functionality, so we can move the Plasma shell back onto QScreen entirely. QScreen is backed by the XCB Qt platform plugin (QPA). One problem in Plasma has been that we got acknowledged of changes through different code paths, which made it hard to set up the desktop, position panels, etc. In a Wayland session, this has to happen in a much more coordinated way, with clearly defined semantics when the screen setup changes, and as little of those changes as necessary.
KScreen should concentrate on doing what it’s good at: screen configuration. For X11 kscreen uses its xrandr backend, no changes there. In Plasma shell’s startup, we will be able to remove libkscreen and rely purely on QScreen directly as soon as we can depend on Qt 5.6, so that probably puts us into the time-frame of Q2 next year. For read-only access on wayland, we can use the libkscreen QScreen backend for now, it comes with some limitations around multi-screen, but these will be ironed by spring next year. The QScreen backend is actually used to start Plasma Mobile’s on kwin_wayland. For configuration, QScreen is not an option, however — it’s simply not its purpose and shouldn’t be.
In the Wayland protocol itself, there are no such semantics yet. Screen configuration has, so far, been outside of the scope of the agreed-upon wayland protocols. If we don’t run on top of an X server, who’s doing the actual hardware setup? Our answer is: KWin, the compositor.
KWin plays a more central role in a Wayland world. For rendering and compositing of windows, it interacts with the hardware. Since it already initializes hardware when it starts a Wayland server, it makes a lot of sense to put screen configuration also exactly there. This means that we will configure KWin at runtime through an interface that is designed around semantics of atomic mode setting, and KWin picks a suitable configuration for connected displays. KWin saves the configuration, applies it on startup or when a display gets switched off, connected or disconnected, and only then tells the workspace shell and the apps to use it. This design makes a lot of sense, since it is KWin that ultimately knows of all the constraints related to dynamic display configuration, and it can make concert how the hardware is used and how its changes are presented to the applications and workspace.
KWayland and unit testing
Much of Kwin/Wayland’s functionality is implemented in a library called KWayland. KWayland wraps the wayland protocol with a Qt-style API for wayland clients and servers, offers threaded connection and type-safety on top of the basic C implementation of libwayland.
KWayland provides a library that allows to run wayland servers, or just specific parts of it with very little code. The KWayland server classes allow us to test a great deal of the functionality in unittests, since we can run the unit tests on a “live” wayland server. Naturally, this is used a lot in kwayland’s own autotests. In the libkscreen wayland backend’s tests, we’re loading different configuration scenarios from json definitions, so we can not only test whether the library works in principle, but really test against live servers, so we cover a much larger part of the stack in our tests. This helps us a lot to make sure that the code works in the first place, but also helps us catch problems easily as soon as they arise. The good unit test coverage also allows much swifter development as a bonus.
Output management wayland interface design
The output management wayland protocol that we have implemented provides two things:
It lists connected output hardware and all of their properties, EDID, modes, physical size, and runtime information such as currently used mode, whether this output device is currently enabled for rendering, etc.
It provides an interface to change settings such as mode, rotation, scaling, position, for hardware and to apply them atomically
This works as follows:
The server announces that the global interfaces for OutputManagement and a list of OutputDevices is available
The configuration client (e.g. the Display Settings) requests the list of output devices and uses them to show the screen setup visually
The user changes some settings and hits “apply”, the client requests an OutputConfiguration object from the OutputManagement global
The configuration object is created on the server specifically for the client, it’s not exposed in the server API at this point.
The client receives the config object and calls setters with new settings for position, mode, rotation, etc.
The server buffers these changes in the per-client configuration object
The client is done changing settings and asks the server to apply them
The compositor now receives a sealed configuration object, tests and applies the new settings, for example through the DRM kernel interface
The compositor updates the global list of OutputDevices and changes its setup, then it signals the client failure or success back through the configuration object
The output management protocol, client- and server-side library, unit tests and documentation are quite a hefty beast, combined they come in at ca. 4700 lines of code. The API impact, however, has been kept quite low and easy to understand. The atomic semantics are reflected in the API, and it encourages to do the right thing, both for the client configuring the screens, and the compositor, which is responsible for applying the setup.
I am currently working on a libkscreen module for screen configuration under wayland, that implements atomic mode setting semantics in libkscreen. It uses a new wayland protocol which Martin Gräßlin and I have been working on in the past months. This protocol lands with the upcoming Plasma 5.5, the libkscreen module may or may not make the cut, this also depends on if we get the necessary bits finished in KWin and its DRM backend. That said, we’re getting really close to closing the last gaps in the stack.
On the compositor side, we can now connect the OutputManagement changes, for example in the DRM backend and implement the OutputDevices interface on top of real hardware.
At Blue Systems, we have been working on making Plasma shine for a while now. We’ve contributed much to the KDE Frameworks 5 and Plasma 5 projects, and helping with the transition to Qt5. Much of this work has been involving porting, stabilizing and improving existing code. With the new architecture in place, we’ve also worked on new topics, such as Plasma on non-desktop (and non-laptop) devices.
Plasma Mobile on an LG Nexus 5
This work is coming to fruition now, and we feel that it has reached a point where we want to present it to a more general public. Today we unveil the Plasma Mobile project. Its aim is to offer a Free (as in Freedom), user-friendly, privacy-enabling and customizable platform for mobile devices. Plasma Mobile runs on top of Linux, uses Wayland for rendering graphics and offers a device-specific user interface using the KDE Frameworks and Plasma library and tooling. Plasma Mobile is under development, and not usable by end users now. Missing functionality and stability problems are normal in this phase of development and will be ironed out. Plasma Mobile provides basic functionality and an opportunity for developers to jump in now and shape the mobile platform, and how we use our mobile devices.
As is necessary with development on mobile devices, we’ve not stopped at providing source code that “can be made to work”, rather we’re doing a reference implementation of Plasma Mobile that can be used by those who would like to build a product based on Plasma Mobile on their platform. The reference implementation is based on Kubuntu, which we chose because there is a lot of expertise in our team with Kubuntu, and at Blue Systems we already have continuous builds and package creation in place. Much of the last year was spent getting the hardware to work, and getting our code to boot on a phone. With pride, we’re now announcing the general availability of this project for public contribution. In order to make clear that this is not an in-house project, we have moved the project assets to KDE infrastructure and put under Free software licenses (GPL and LGPL according to KDE’s licensing policies). Plasma Mobile’s reference implementation runs on an LG Nexus 5 smartphone, using an Android kernel, Ubuntu user space and provides an integrated Plasma user interface on top of all that. We also have an x86 version, running on an ExoPC, which can be useful for testing.
Plasma Mobile uses the Wayland display protocol to render the user interface. KWin, Plasma’s window manager and compositor plays a central role. For apps that do not support Wayland, we provide X11 support through the XWayland compatibility layer.
Plasma Mobile is a truly converged user interface. More than 90% of its code is shared with the traditional desktop user interface. The mobile workspace is implemented in the form of a shell or workspace suitable for mobile phones. The shell provides an app launcher, a quick settings panel and a task switcher. Other functionality, such as a dialer, settings, etc. is implemented using specialized components that can be mixed and matched to create a specific user experience or to provide additional functionality — some of them already known from Plasma Desktop.
Architecture diagram of Plasma Mobile
Plasma Mobile is developed in a public and open development process. Contributions are welcome and encouraged throughout the process. We do not want to create another walled garden, but an inclusive platform for creation of mobile device user experiences. We do not want to create releases behind closed doors and throw them over the wall once in a while, but create a leveled playing field for contributors to work together and share their work. Plasma Mobile’s code is available on git.kde.org, and its development is discussed on the plasma-devel mailinglist. In the course of Akademy, we have a number of sessions planned to flesh out more and more detailed plans for further development.
With the basic workspace and OS integration work done, we have laid a good base for further development, and for others to get their code to run on Plasma Mobile. More work which is already in our pipeline includes support for running Android applications, which potentially brings a great number of mature apps to Plasma Mobile, better integration with other Plasma Devices, such as your desktop or laptop through KDE Connect, an improved SDK making it very easy to get a full-fledged development environment set up in minutes, and of course more applications.
One of the things I’ve been sorely missing when doing UI design and development was a good way to preview icons. The icon picker which is shipped with KDE Frameworks is quite nice, but for development purposes it lacks a couple of handy features that allow previewing and picking icons based on how they’re rendered.
Over the christmas downtime, I found some spare cycles to sit down and hammer out a basic tool which allows me to streamline that workflow. In the course of writing this little tool, I realised that it’s not only useful for a developer (like me), but also for artists and designers who often work on or with icons. I decided to target these two groups (UI developers and designers) and try to streamline this tool as good as possible for their usecases.
Cuttlefish is the result of that work. It’s a small tool to list, pick and preview icons. It tries to follow the way we render icons in Plasma UIs as close as possible, in order to make the previews as realistic as possible. I have just shown this little tool to a bunch of fellow Plasma hackers here at the sprint, and it was very well received. I’ve collected a few suggestions what to improve, and of course, cuttlefish being brand-new, it still has a few rough edges.
You can get the source code using the following command: git clone kde:scratch/sebas/cuttlefish
git clone kde:plasmate
and build it with the cmake.
[Edit] We moved cuttlefish to the Plasmate repository, it’s now part of Plasma’s SDK.
One of the important design cornerstones of Plasma is that we want to reduce the amount of “hidden features” as much as possible. We do not want to have to rely on the user knowing where to right-click in case she wants to find a certain, desktop-related option, say adding widgets, opening the desktop settings dialog, the activity switcher, etc.. For this, Plasma 4.0 introduced the toolbox, a small icon that when clicked opens a small dialog with actions related to the desktop. To many users, this is an important lifeline when they’re looking for a specific option.
In Plasma 4.x, there was a Plasmoid, provided by a third party, that used a pretty gross hack to remove the toolbox (which was depicted as the old Plasma logo, resembling a cashew a bit). We did not support this officially, but if people are deliberately risking to break their desktop, who are we to complain. They get to keep both pieces.
During the migration to QML (which begun during Plasma 4.x times), one of the parts I had been porting to QtQuick was this toolbox. Like so many other components in Plasma, this is actually a small plugin. That means it’s easy to replace the toolbox with something else. This feature has not really been documented as its more or less an internal thing, and we didn’t want to rob users of this important lifeline.
Some users want to reduce clutter on their desktop as much as possible, however. Since the options offered in the toolbox are also accessible elsewhere (if you know to find them). Replacing the toolbox is actually pretty easy. You can put a unicorn dancing on a rainbow around your desktop there, but you can also replace it with just an empty object, which means that you’re effectively hiding the toolbox.
For users who would rather like their toolbox to be gone, I’ve prepared a small package that overrides the installed toolbox with an empty one. Hiding the toolbox is as easy as installing this minimal package, which means the toolbox doesn’t get shown, or even get loaded.
I would not recommend doing this, especially not as default, but at the same time, I don’t want to limit what people do with their Plasma do what we as developers exactly envision, so there you go.
Now restart the Plasma Shell (either by stopping the plasmashell process, or by logging out and in again), and your toolbox should be gone.
If you want it back, run
plasmapkg2 -t package -r org.kde.desktoptoolbox
Then restart Plasma and it’s back again.
Even more than just removing the toolbox, I’d like to invite and encourage everybody to come up with nice, crazy and beautiful ideas how to display and interact with the toolbox. The toolbox being a QtQuick Plasmoid package, it’s easy to change and to share with others.
TL;DR: The coming year is full of challenges, old and new, for the Plasma team. In this post, I’m highlighting end-user readiness, support for Wayland as display server and support for high-dpi displays.
Before you continue reading, have a gratuitous fish! (Photo taken by my fine scuba diving buddy Richard Huisman.)
Next year will be interesting for Plasma. Two things that are lined up are particularly interesting. In 2015, distributions will start shipping Plasma 5 as their default user interface. This is the point where more “oblivious” users will make their first contact with Plasma 5. As we’re navigating through the just-after-a-big-release phase, which I think we’re mastering quite nicely, we approach a state where a desktop that has so many things changed under its hood is becoming a really polished and complete working environment, that feels modern, supports traditional workflows well, and is built on top of a top-notch modern modularized set of libraries, KDE’s Frameworks.
The other day, I’ve read on a forum a not particularly well-informed, yet interesting opinion: “Plasma 5 is not for end users, its Wayland support is still not ready”. The Plasma 5 is not for end users, I do actually agree with, in a way. While I know that there is a very sizable group of people that have been having a blast running Plasma since 5.0, when talking about end-users, one needs to look at the cases where it isn’t suitable. For one, these give concrete suggestions what to improve, so they’re important for prioritization. This user feedback channel has been working very well so far, we’ve been receiving hundreds of bug reports, which we could address in one way or another, we have been refining our release and QA processes, and we’ve filled in many smaller and bigger gaps. There’s still much more work to do, but the tendency is exactly right. By ironing out many real-world problems, each of those fixes increases the group of users Plasma is ready for, and improve the base to build a more complete user experience upon.
What’s also true about the statement of the above “commenter on the Internet” is that our Wayland support isn’t ready. It is entirely orthogonal to the “is it ready for end users?” question. Support for Wayland is a feature we’re gradually introducing, very much in a release-early-release-often fashion. I expect our support for this new display server system to reach a point where one can run a full session on top of Wayland in the course of next year. I expect that long-term, most of our users will run the user interface on top of Wayland, effectively deprecating X11. Yet, X11 will stay around for a long time, there’s so much code written on top of X11 APIs that we simply can’t expect it to just vanish from one day to the other. Some Linux distros may switch relatively early, while for Enterprise distros, that switch might only happen in the far future, that doesn’t even count existing installations. That is not a problem, though, since Wayland and X11 support are well encapsulated, and supposed to not get in the way of each other — we do the same trick already on other operating systems, and it’s a proven and working solution.
Then, there’s the mission to finish high-dpi support. High DPI support means rendering a usable UI on displays with more than 200 DPI. That means that UI elements have to scale or be rendered with more detail and fidelity. One approach is to simply scale up everything in every direction by a fixed factor, but while it would get the sizing right, it would also negate any benefit of the increased amount of pixels. Plasma 5 already solves many issues around high-dpi, but not without fiddling, and going over different settings to get them right. Our goal is to support high-dpi displays out of the box, no fiddling, just sensible defaults in case a high dpi display gets connected. As there are 101 corner cases to this, it’s not easy to get right, and will take time and feedback cycles. Qt 5.4, which is around the corner, brings some tools to support these displays better, and we’ll be adjusting our solutions to make use of that.
It seems we are not quite yet running out of interesting topics that make Plasma development a lot of fun. :)
With the Plasma 5.0 release out the door, we can lift our heads a bit and look forward, instead of just looking at what’s directly ahead of us, and make that work by fixing bug after bug. One of the important topics which we have (kind of) excluded from Plasma’s recent 5.0 release is support for Wayland. The reason is that much of the work that has gone into renovating our graphics stack was also needed in preparation for Wayland support in Plasma. In order to support Wayland systems properly, we needed to lift the software stack to Qt5, make X11 dependencies in our underlying libraries, Frameworks 5 optional. This part is pretty much done. We now need to ready support for non-X11 systems in our workspace components, the window manager and compositor, and the workspace shell.
Let’s dig a bit deeper and look at at aspects underlying to and resulting from this transition.
The short answer to this question, from a Plasma perspective, is:
Xorg lacks modern interfaces and protocols, instead it carries a lot of ballast from the past. This makes it complex and hard to work with.
Wayland offers much better graphics support than Xorg, especially in terms of rendering correctness. X11’s asynchronous rendering makes it impossible to be sure about correctness and timeliness of graphics that ends up on screen. Instead, Wayland provides the guarantee that every frame is perfect
Security considerations. It is almost impossible to shield applications properly from each other. X11 allows applications to wiretap each other’s input and output. This makes it a security nightmare.
I could go deeply into the history of Xorg, and add lots of technicalities to that story, but instead of giving you a huge swath of text, hop over to Youtube and watch Daniel Stone’s presentation “The Real Story Behind Wayland and X” from last year’s LinuxConf.au, which gives you all the information you need, in a much more entertaining way than I could present it. H-Online also has an interesting background story “Wayland — Beyond X”.
While Xorg is a huge beast that does everything, like input, printing, graphics (in many different flavours), Wayland is limited by design to the use-cases we currently need X for, without the ballast.
With all that in mind, we need to respect our elders and acknowledge Xorg for its important role in the history of graphical Linux, but we also need to look beyond it.
What is Wayland support?
KDE Frameworks 5 apps under Weston
Without communicating our goal, we might think of entirely different things when talking about Wayland support. Will Wayland retire X? I don’t think it will in the near future, the point where we can stop caring for X11-based setups is likely still a number of years away, and I would not be surprised if X11 was still a pretty common thing to find in enterprise setups ten years down the road from now. Can we stop caring about X11? Surely not, but what does this mean for Wayland? The answer to this question is that support for Wayland will be added, and that X11 will not be required anymore to run a Plasma desktop, but that it is possible to run Plasma (and apps) under both, X11 and Wayland systems. This, I believe, is the migration process that serves our users best, as the question “When can I run Plasma on Wayland?” can then be answered on an individual basis, and nobody is going to be thrown into the deep (at least not by us, your distro might still decide to not offer support for X11 anymore — that is not in our hands). To me, while a quick migration to Wayland (once ready) is something desirable, realistically, people will be running Plasma on X11 for years to come. Wayland can be offered as an alternative at first, and then promote to primary platform once the whole stack matures further.
Where at we now?
With the release of KDE Frameworks 5, most of the issues in our underlying libraries have been ironed out, that means X11-dependent codepaths have become optional. Today, it’s possible to run most applications built on top of Frameworks 5 under a Wayland compositor, independent from X11. This means that applications can run under both, X11 and Wayland with the same binary. This is already really cool, as without applications, having a workspace (which in a way is the glue between applications would be a pointless endeavour). This chicken-egg situation plays both ways, though: Without a workspace environment, just having apps run under Wayland is not all that useful. This video shows some of our apps under the Weston compositor. (This is not a pure Wayland session “on bare metal”, but one running in an X11 window in my Plasma 5 session for the purpose of the screen-recoding.)
For a full-blown workspace, the porting situation is a bit different, as the workspace interacts much more intimately with the underlying display server than applications do at this point. These interactions are well-hidden behind the Qt platform abstraction. The workspace provides the host for rendering graphics onto the screen (the compositor) and the machinery to start and switch between applications.
We are currently missing a number of important pieces of the full puzzle: Interfaces between the workspace shell, the compositor (KWin) and the display server are not yet well-defined or implemented, some pioneering work is ahead of us. There is also a number of workspace components that need bigger adjustments, global shortcut handling being a good example. Most importantly, KWin needs to take over the role of Wayland compositor. While some support for Wayland has already been added to KWin, the work is not yet complete. Besides KWin, we also need to add support for Wayland to various bits of our workspace. Information about attached screens and their layout has to be made accessible. Global keyboard shortcuts only support X11 right now. The screen locking mechanism needs to be implemented. Information about Windows for the task-manager has to be shared. Dialog positioning and rendering needs to be ported. There are also a few assumptions in startkde and klauncher that currently prevent them from being able to start a session under Wayland, and more bits and pieces which need additional work to offer a full workspace experience under Wayland.
The idea is to be able to run the same binaries under both, X11 and Wayland. This means that we (need to decide at runtime how to interact with the windowing system. The following strategy is useful (in descending order of preference):
Use abstract Qt and Frameworks (KF5) APIs
Use XCB when there are no suitable Qt and KF5 APIs
Decide at runtime whether to call X11-specific functions
In case we have to resort to functions specific to a display server, X11 should be optional both at build-time and at run-time:
The build of X11-dependent code optional. This can be done through plugins, which are optionally included by the build-system or (less desirably) by #ifdef’ing blocks of code.
Even with X11 support built into the binary, calls into X11-specific libraries should be guarded at runtime (QX11Info::isPlatformX11() can be used to check at runtime).
Get your Hands Dirty!
Computer graphics are an exciting thing, and many of us are longing for the day they can remove X11 from their systems. This day will eventually come, but it won’t come by itself. It’s a very exciting time to get involved, and make the migration happen. As you can see, we have a multitude of tasks that need work. An excellent first step is to build the thing on your system and try running, fix issues, and send us patches. Get in touch with us on Freenode’s #plasma IRC channel, or via our mailing list plasma-devel(at)kde.org.
In many cases, high-quality code counts more than bells and whistles. Fast, reliable and well-maintained libraries provide a solid base for excellent applications built on top of it. Investing time into improving existing code improves the value of that code, and of the software built on top of that. For shared components, such as libraries, this value is often multiplied by the number of users. With this in mind, let’s have a closer look of how the Frameworks 5 transition affects the quality of the code, so many developers and users rely on.
KDE Frameworks 5 will be released in 2 weeks from now. This fifth revision of what is currently known as the “KDE Development Platform” (or, technically “kdelibs”) is the result of 3 years of effort to modularize the individual libraries (and “bits and pieces”) we shipped as kdelibs and kde-runtime modules as part of KDE SC 4.x. KDE Frameworks contains about 60 individual modules, libraries, plugins, toolchain, and scripting (QtQuick, for example) extensions.
One of the important aspects that has seen little exposure when talking about the Frameworks 5 project, but which is really at the heart of it, are the processes behind it. The Frameworks project, as it happens with such transitions has created a new surge of energy for our libraries. The immediate results, KF5’s first stable release is a set of software frameworks that induce minimal overhead, are source- and binary-stable for the foreseeable future, are well maintained, get regular updates and are proven, high-quality, modern and performant code. There is a well-defined contribution process and no mandatory copyright assignment. In other words, it’s a reliable base to build software on in many different aspects.
Extension and improvement of existing software are two ways of increasing their values. KF5 does not contain revolutionary new code, instead of extending it, in this major cycle, we’re concentrating on widening the usecases and improving their quality. The initial KDE4 release contained a lot of rewritten code, changed APIs and meant a major cleanup of hard-to-scale and sometimes outright horrible code. Even over the course of 4.x, we had a couple of quite fundamental changes to core functionality, for example the introduction of semantic desktop features, Akonadi, in Plasma the move to QML 1.x.
All these new things have now seen a few years of work on them (and in the case of Nepomuk replacing of the guts of it with the migration to the much more performant Baloo framework). These things are mature, stable and proven to work by now. The transition to Qt5 and KF5 doesn’t actually change a lot about that, we’ve worked out most of the kinks of this transition by now. For many application-level code using KDE Frameworks, the porting will be rather easy to do, though not zero effort. The APIs themselves haven’t changed a lot, changes to make something work usually involve updating the build-system. From that point on, the application is often already functional, and can be gradually moved away from deprecated APIs. Frameworks 5 provides the necessary compatibility libraries to ease porting as much as possible.
Surely, with the inevitable and purposeful explosion of the user-base following a first stable release, we will get a lot of feedback how to further improve the code in Frameworks 5. Processes, requirements and tooling for this is in place. Also, being an open system, we’re ready to receive your patches.
Frameworks 5, in many ways encodes more than 15 years of experience into a clearly structured, stable base to build applications for all kinds of purposes, on all kinds of platforms on.
With the modularization of the libraries, we’ve looked for suitable maintainers for them, and we’ve been quite successful in finding responsible caretakers for most of them. This is quite important as it reduces bottlenecks and single points of failure. It also scales up the throughput of our development process, as the work can be shared across more shoulders more easily. This achieves quicker feedback for development questions, code review requests, or help with bug fixes. We don’t actually require module maintainers to fix every single bug right away, they are acting much more as orchestrators and go-to-guys for a specific framework.
More peer-review of code is generally a good thing. It provides safety nets for code problems, catches potential bugs, makes sure code doesn’t do dumb thing, or smart things in the wrong way. It also allows transfer of knowledge by talking about each others code. We have already been using Review Board for some time, but the work on Frameworks 5 and Plasma 5 has really boosted our use of review board, and review processes in general. It has become a more natural part of our collaboration process, and it’s a very good thing, both socially and code-quality-wise.
More code reviews also keeps us developers in check. It makes it harder to slip in a bit of questionable code, a psychological thing. If I know my patches will be looked at line-by-line critically, it triggers more care when submitting patches. The reasons for this are different, and range from saving other developers some time to point out issues which I could have found myself had I gone over the code once more, but also make me look more cool when I submit a patch that is clean and nice, and can be submitted as-is.
Surely, code reviews can be tedious and slow down the development, but with the right dose, in the end it leads to better code, which can be trusted down the line. The effects might not be immediately obvious, but they are usually positive.
Splitting up the libraries and getting the build-system up to the task introduced major breakage at the build-level. In order to make sure our changes would work, and actually result in buildable and working frameworks, we needed better tooling. One huge improvement in our process was the arrival of a continuous integration system. Pushing code into one of the Frameworks nowadays means that a it is built in a clean environment and automated tests are run. It’s also used to build its dependencies, so problems in the code that might have slipped the developer’s attention are more often caught automatically. Usually, the results of the Continuous integration system’s automated builds are available within a few minutes, and if something breaks, developers get notifications via IRC or email. Having these short turnaround cycles makes it easier to fix things, as the memory of the change leading to the problem is still fresh. It also saves others time, it’s less likely that I find a broken build when I update to latest code.
The build also triggers running autotests, which have been extended already, but are still quite far away from complete coverage. Having automated tests available makes it easier to spot problems, and increases the confidence that a particular change doesn’t wreak havoc elsewhere.
Neither continuous builds, nor autotests can make 100% sure that nothing ever breaks, but it makes it less likely, and it saves development resources. If a script can find a problem, that’s probably vastly more efficient than manual testing. (Which is still necessary, of course.)
A social aspect here is that not a single person is responsible if something breaks in autobuilds or autotests, it rather should be considered a “stop-the-line” event, and needs immediate attention — by anyone.
This harnessing allows us to concentrate more on further improvments. Software in general are subject to a continous evolution, and Frameworks 5.0 is “just another” milestone in that ongoing process. Better scalability of the development processes (including QA) is not about getting to a stable release, it supports the further improvement. As much as we’ve updated code with more modern and better solutions, we’re also “upgrading” the way we work together, and the way we improve our software further. It’s the human build system behind software.
The circle goes all the way round, the continuous improvement process, its backing tools and processes evolve over time. They do not just pop out of thin air, they’re not dictated from the top down, they are rather the result of the same level of experience that went into the software itself. The software as a product and its creation process are interlinked. Much of the important DNA of a piece of software is encoded in its creation and maintenance process, and they evolve together.