August 24th, 2016

Multisceen in Plasma: Improved tools and debugging

cube-small
Plasma 5.8 will be our first long-term supported release in the Plasma 5 series. We want to make this a release as polished and stable as possible. One area we weren’t quite happy with was our multi-screen user experience. While it works quite well for most of our users, there were a number of problems which made our multi-screen support sub-par.
Let’s take a step back to define what we’re talking about.

Multi-screen support means that connecting more than one screen to your computer. The following use cases give good examples of the scope:

  • Static workstation A desktop computer with more than one display connected, the desktop typically spans both screens to give more screen real estate.
  • Docking station A laptop computer that is hooked up to a docking station with additional displays connected. This is a more interesting case, since different configurations may be picked depending on whether the laptop’s lid is closed or not, and how the user switches between displays.
  • Projector The computer is connected to a projector or TV.

The idea is that the user plugs in or starts up with that configuration, if the user has already configured this hardware combination, this setup is restored. Otherwise, a reasonable guess is done to put the user to a good starting point to fine-tune the setup.

kcm-videowall
This is the job of KScreen. At a technical level, kscreen consists of three parts:

  • system settings module This can be reached through system settings
  • kscreen daemon Run in a background process, this component saves, restores and creates initial screen configurations.
  • libkscreen This is the library providing the screen setup reading and writing API. It has backends for X11, Wayland, and others that allow to talk to the exact same programming interface, independent of the display server in use.

At an architectural level, this is a sound design: the roles are clearly separated, the low-level bits are suitably abstracted to allow re-use of code, the API presents what matters to the user, implementation details are hidden. Most importantly, aside from a few bugs, it works as expected, and in principle, there’s no reason why it shouldn’t.

So much for the theory. In reality, we’re dealing with a huge amount of complexity. There are hardware events such as suspending, waking up with different configurations, the laptop’s lid may be closed or opened (and when that’s done, we don’t even get an event that it closed, displays come and go, depending on their connection, the same piece of hardware might support completely different resolutions, hardware comes with broken EDID information, display connectors come and go, so do display controllers (crtcs); and on top of all that: the only way we get to know what actually works in reality for the user is the “throw stuff against the wall and observe what sticks” tactic.

This is the fabric of nightmares. Since I prefer to not sleep, but hack at night, I seemed to be the right person to send into this battle. (Coincidentally, I was also “crowned” kscreen maintainer a few months ago, but let’s stick to drama here.)

So, anyway, as I already mentioned in an earlier blog entry, we had some problems restoring configurations. In certain situations, displays weren’t enabled or positioned unreliably, or kscreen failed to restore configurations altogether, making it “forget” settings.
kscreen-doctor

Better tools

Debugging these issues is not entirely trivial. We need to figure out at which level they happen (for example in our xrandr implementation, in other parts of the library, or in the daemon. We also need to figure out what happens exactly, and when it does. A complex architecture like this brings a number of synchronization problems with it, and these are hard to debug when you have to figure out what exactly goes on across log files. In Plasma 5.8, kscreen will log its activity into one consolidated, categorized and time-stamped log. This rather simple change has already been a huge help in getting to know what’s really going on, and it has helped us identify a number of problems.

A tool which I’ve been working on is kscreen-doctor. On the one hand, I needed a debugging helper tool that can give system information useful for debugging. Perhaps more importantly I know I’d be missing a command-line tool to futz around with screen configurations from the command-line or from scripts as Wayland arrives. kscreen-doctor allows to change the screen configuration at runtime, like this:

Disable the hdmi output, enable the laptop panel and set it to a specific mode
$ kscreen-doctor output.HDMI-2.disable output.eDP-1.mode.1 output.eDP-1.enable

Position the hdmi monitor on the right of the laptop panel
$ kscreen-doctor output.HDMI-2.position.0,1280 output.eDP-1.position.0,0

Please note that kscreen-doctor is quite experimental. It’s a tool that allows to shoot yourself in the foot, so user discretion is advised. If you break things, you get to keep the pieces. I’d like to develop this into a more stable tool in kscreen, but for now: don’t complain if it doesn’t work or eat your hamster.

Another neat testing tool is Wayland. The video wall configuration you see in the screenshot is unfortunately not real hardware I have around here. What I’ve done instead is run a Wayland server with these “virtual displays” connected, which in turn allowed me to reproduce a configuration issue. I’ll spare you the details of what exactly went wrong, but this kind of tricks allows us to reproduce problems with much more hardware than I ever want or need in my office. It doesn’t stop there, I’ve added this hardware configuration to our unit-testing suite, so we can make sure that this case is covered and working in the future.

May 31st, 2016

Multiscreen in Plasma 5.7 and beyond

Here’s a quick status update about where we currently stand with respect to multiscreen support in Plasma Desktop.

While for many people, multiscreen support in Plasma works nicely, for some of our users, it doesn’t. There are problems with restoring previously set up configurations, and around the primary display mechanism. We’re really unhappy about that, and we’re working on fixing it for all of our users. These kinds of bugs are the stuff nightmares are made of, so there’s not a silver bullet to fix everything of it, once and for all right away. Multiscreen support requires many different components to play in tune with each other, and they’re usually divided into separate processes communicating via different channels with each other. There’s X11 involved, XCB, Qt, libkscreen and of course the Plasma shell. I can easily at least three different protocols in this game, Wayland being a fourth (but likely not used at the same time as X11). There’s quite some complexity involved, and the individual components involved are actually doing their jobs quite well and have their specific purposes. Let me give an overview.

Multiscreen components

Plasma Shell renders the desktop, places panels, etc., When a new screen is connected, it checks whether it has an existing configuration (wallpaper, widgets, panels etc.) and extends the desktop. Plasma shell gets its information from QScreen now (more on that later on!)

KWin is the compositor and window manager. KWin/X11 interacts with X11 and is responsible for window management, movement, etc.. Under Wayland, it will also take the job of the graphical and display server work that X11 currently does, though mostly through Wayland and *GL APIs.

KScreen kded is a little daemon (actually a plugin) that keeps track of connected monitors and applies existing configs when they change

KScreen is a module in systemsettings that allows to set up the display hardware, positioning, resolution, etc.

Libkscreen is the library that backs the KScreen configuration. It offers an API abstraction over XRandR and Wayland. libkscreen sits pretty much at the heart of proper multiscreen support when it comes to configuring manually and loading the configuration.

Primary Desktop

The primary display mechanism is a bit of API (rooted in X11) to mark a display as primary. This is used to place the Panel in Plasma, and for example to show the login manager window on the correct monitor.

Libkscreen and Qt’s native QScreen are two different mechanism to reflect screen information. QScreen is mainly used for querying info (and is of course used throughout QtGui to place windows, get information about resolution and DPI, etc.). Libkscreen has all this information as well, but also some more, such as write support. Libkscreen’s backends get this information directly from Xorg, not going through Qt’s QScreen API. For plasmashell, we ended up needing both, since it was not possible to find the primary display using Qt’s API. This causes quite some problems since X11 is async by its nature, so essentially we ended up having “unfixable” race conditions, also in plasmashell. These are likely the root cause of the bug you’re seeing here.

This API has been added in Qt 5.6 (among a few other fixes) by Aleix Pol, one of our devs in the screen management team. We have removed libkscreen from plasmashell today and replaced it with “pure QScreen” code, since all the API we need for plasmashell is now available in the Qt we depend on.

These changes should fix much of the panel placement grief that bug 356225 causes. It does need some good testing, now it’s merged. Therefore, we’d like to see as many people, especially those reporting problem with multiscreen, to test against latest Plasma git master (or the upcoming Plasma 5.7 Beta, which is slated for release on June, 16th).

Remember the config

Another rough area that is under observation right now is remembering and picking the right configuration from a previous setup, for example when you return to your docking station which has another display connected. Bug 358011 is an example for that. Here, we get “spurious” events about hardware changes from X11, and I’m unsure where they come from. The problem is that it’s not easy to reproduce, it only happens for certain setups. This bug was likely introduced with the move to Qt 5 and Frameworks, it’s a regression compared to Plasma 4.
I’ve re-reviewed the existing code, added more autotests and made the code more robust in some places that seemed relevant, but I can’t say that we’ve found a sure solution to these problems. The code is now also better instrumented for debugging the areas at play here. Now we need some more testing of the upcoming beta. This is certainly not unfixable, but needs feedback from testing so we can apply further fixes if needed.

Code quality musings

From a software engineering point of view, we’re facing some annoying problems. It took us a long time to get upstream QScreen code to be robust and featureful enough to draw the lines between the components involved, especially QScreen and libkscreen more clearly, which certainly helps to reduce hard-to-debug race conditions involving hardware events. The fact that it’s almost impossible to properly unit test large parts of the stack (X11 and hardware events are especially difficult in that regard) means that it’s hard to control the quality. On the other hand, we’re lacking testers, especially those that face said problems and are able to test the latest versions of Qt and Plasma.
QA processes is something we spent some serious work on, on the one hand, our review processes for new code and changes to current code are a lot stricter, so we catch more problems and potential side-effects before code gets merged. For new code, especially the Wayland support, our QA story also looks a lot better. We’re aiming for near-100% autotest coverage, and in many cases, the autotests are a lot more demanding than the real world use cases. Still, it’s a lot of new code that needs some real world exposure, which we hope to get more of when users test Plasma 5.7 using Wayland.

March 11th, 2016

Inside the Particle Accelerator

IMG_20160311_164446We visited the CMS today, the Compact Muon Solenoid, one of the “experiments” that are part of the LHC, the large hadron collider, which is a research project aiming to find out more about the basic structures that make up our universe, like how gravity works, which particles make up our universe, and so on.

In order to do that, researchers accelerate elementary particles to almost the speed of light, make them crash into each other and record this process. There’s a 27km long circular tunnel 100m below the ground near Geneva, Switzerland (the tunnel actually extends into France, so the CMS experiment is located in France.

IMG_20160311_175655

As we’re currently at a KDE Plasma sprint at CERN, the European Research center for nuclear energie, we were invited to a tour through the CMS. We got to learn a lot about physics and also descended down into the chamber holding the giant CMS machine. (It’s switched off currently, since energy costs during winter are too high to run it — it takes a lot of power to accelerate particles to almost the speed of light, in the range of 200 Megawatt). There’s some maintenance work going on right now before this monster is switched on again towards the end of the month.

CERN staff were friendly and patient and answered all our questions. The tour was interesting and a lot of fun.

Here are some more pictures I took during the visit. You may notice that some of the computers in the control room are running Plasma on their desktops.
IMG_20160311_171423
IMG_20160311_165658

More photos of my visit can be found here.

March 5th, 2016

Getting physical

plasma-cernbannerI’ve just finished my travel preparations for this year’s Plasma sprint, which will start tomorrow with the arrival of my fellow hackers in Geneva, CH. Together with a few other groups in KDE and organized by our WikiToLearn friends, we’re honored to be guests at CERN, the European Organization for Nuclear Research where physicists and engineers are probing the fundamental structure of the universe. Of course, these scientists and engineers couldn’t do their work without Plasma, so we’re obliged to hold our meeting there. :-)

Topics range from our Wayland support to Plasma Mobile, documentation, how we can improve our desktop experience, and general planning for the next months. I’m also looking forward to some face-time with my fellow hackers, and discussions with the artists and usability experts who are holding a meeting of the visual design group in KDE. Only good stuff can come out of this.

Many thanks go to all the people who support KDE. This support makes meetings like next week’s possible. Please consider supporting us.

January 15th, 2016

Is privacy Free software’s next milestone?

I am concerned. In the past years, it has become clear that real privacy has become harder to come by. Our society is quickly heading into a situation where an unknown number of entities and people can follow my every single step, and it’s not possible to keep to myself what I don’t want others to know. With every step into that direction, there’s less and less things about my life of which I don’t control who knows about it.

Privacy as product or weapon

Realistically, I won’t be able to do that, however, since in this modern age, tools that need to share data are rather the norm, than the exception. Most of the time, this sharing of data (even if only between my own devices) goes through the hand of a third party. On top of that, there’s a whole lot of spying going on, and of course malicious hackers which are keen to acquire large personal sets of identity data. My personal data can make me a product, and worse, it can be used as a weapon against myself. It is really in my best interest to share only the absolute minimal amount of data with as little others as possible.

Traditionally, this urgency for privacy has been closely connected to the goals of Free software. This is not a coincidence. Free software and was intended as a way to give control to the users, and copyleft is an effective tool to achieve “software democracy”, in the best interest of the user. In reverse, someone who is not in control of his data cannot truly be free. Privacy and freedom are in fact closely related concepts.

Software Freedom: economics and ideology

I prefer Free software over proprietary solutions. It puts me in control what my machine does, it allows me to fulfill my needs and influence the tools I use for communication, work and entertainment into a direction that is driven by value to the user, rather than return-on-investment measured in money.

When I started using computers, Free software was sub-par to proprietary solutions, that is largely not the case anymore. In many cases, Free software surpasses what proprietary alternatives offer. In a lot of areas, Free software has come to dominate the market.
This is not surprising, given the economic model behind Free software. In the long run, building on the shoulder of giants, sharing the work across more stakeholders, open code and processes are more economical, scale better and tend to be more sustainable.
The ideological point of view benefits from that, I can lead a fully functional digital life using almost exclusively Free software and I certain guarantees of continuity often unmet in the proprietary world.

Shifting purpose

To me, the purpose of Free software has shifted a bit, or rather expands to enabling privacy. A good measurement whether the Free software movement has achieved its goal is the degree of privacy it allows me to have, while enabling all the modern amendments that our digital age makes possible, or even just to have a private conversation with a friend.

Effective privacy

Effective privacy needs network effects, so it doesn’t work very well for niche products. Of what use is a secure and private communication tool if I can’t use it to talk with my friends? Luckily the initial successes of Free software still play in our advantage: being able to collaboratively develop and share the work across many shoulders, we should be able to not just build all the pieces, but put together a complete set of solutions that make better privacy achievable for more people. In terms of achieving network effects, we’re not starting at zero, but our adversaries are strong, and often ahead of our game, some tend to play unfair.

Purpose means responsibility

Is it not our responsibility as Free software community (or even just as citizens) to provide the tools that maximize privacy for the users? If the answer is yes, then I suppose the measurement for success is how much can we make possible while maximizing privacy? How attractive can we make the tools in terms of functionality, effectiveness and availability?

A happy user is one who finds that a useful and fun-to-use tool also protects him from threats that he often may not fully appreciate until it’s too late.

November 30th, 2015

Plasma Components on Android: Accelerating Subsurface Mobile Development

Marco has come over to the Netherlands to pay me a visit, and to hack a little bit together, in person. So with the weather clearly suggesting to stay inside, that’s what we did over the weekend, and how better to entertain yourself than to work on mobile software?

Marco has been working for a while on components that follow Plasma’s human interface guidelines and make it easy to implement applications with a common navigation pattern and look and feel. Obviously, these components use a lot of Plasma under the hood, so they get excellent integration at a visual and at a technical level. This high integration, however, comes at the price of having a non-trivial chain of dependencies. That’s not a problem in Plasma Mobile, or other Plasma workspaces, since all that is already there, anyway.
We thought that an interesting exercise would be to find out what really defines a “Plasma application”, and how we can make the concepts we engrained in their design available to application developers more easily. How hard could it be to use Plasma components in an Android app, for example? The answer is, not entirely trivial, but it just became a whole lot easier. So what did we do?

For those reading this article via a feed aggregator, hop over to youtube to watch the demo video.
We took Subsurface, which is a piece of Free software used for logging and analysing scuba dives. Subsurface has a mobile version, which is still in its infancy, so it’s an excellent candidate to experiment with. We also took Marco’s set of Plasma components that provide a reduced set of functionality, in fact, just enough to create what most applications will need. These components extend QtQuick components where we found them lacking. They’re very light weight, carry no dependencies other than QtQuick, and they’re entirely written in QML, so basically, you add a bunch of QML files to your app and concentrate on what makes your app great, not on overall navigation components or re-implementing for the n-th time a set of widgets.

So after solving some deployment issues, on Saturday night, we had the Plasma mobile components loading in an Android app. A first success. Running the app did show a number of problems, however, so we spent most of the Sunday to look into each problem one by one and trying to solve them. By early Monday morning, we had all of the glaring issues we found during our testing solved, and we got Subsurface mobile to a pretty solid appearance (pretty solid given its early state of development, not bug free by any means).

So, what to take a away from this? In a reduced form, Plasma can be a huge help to create also Android applications. The mobile components which we’re developing with Plasma Mobile as target in mind have had their first real world exposure and a lot of fixes, we got very useful feedback from the Subsurface community which we’re directly feeding back into our components.

A big thanks goes out to the Subsurface team and especially Dirk Hohndel for giving us excellent and timely feedback, for being open to our ideas and for willing to play guinea pig for the Plasma HIG and our components. The state you can see in the above video has already been reviewed and merged into Subsurface’s master tree, so divers around the world will be able to enjoy it when the app becomes available for a wider audience.

November 27th, 2015

screen config victory!

kscreen wayland backend in action

kscreen wayland backend in action

That moment when the application “just works” after all your unit tests pass…

A really nice experience after working on these low-level bits was firing up the kscreen systemsettings module configured to use my wayland test server. I hadn’t done so in a while, so I didn’t expect much at all. The whole thing just worked right out of the box, however. Every single change I’ve tried had exactly the expected effect.
This screenshot shows Plasma’s screen configuration settings (“kscreen”). The settings module uses the new kwayland backend to communicate with a wayland server (which you can see “running” on the left hand side). That means that another big chunk of getting Plasma Wayland-ready for multi-display use-cases is falling nicely into place.

testing

I’m working on this part of the stack using test-driven development methods, so I write unit tests for every bit of functionality, and then implement and polish the library parts. Something is done when all units tests pass reliably, when others have reviewed the code, when everything works in on the application side, and when I am happy with it.
The unit tests stay in place and are from then on compiled and run through our continuous integration system automatically on every code change. This system yells at us as soon as any of the unit tests breaks or shows problems, so we can fix it right away.

Interestingly, we run the unit tests live against a real wayland server. This test server is implemented using the KWayland library. The server runs headless, so it doesn’t do any rendering of windows, and it just implements the bits interesting for screen management. It’s sort of a mini kwin_wayland, the real kwin will use this exact same library on the server side, so our tests are not entirely synthetical. This wasn’t really possible for X11-based systems, because you can’t just fire up an X server that supports XRandR in automated tests — the machine running the test may not allow you to use its graphics card, if it even has one. It’s very easy to do, however, when using wayland.
Our autotests fire up a wayland server from one of many example configurations. We have a whole set of example configurations that we run tests against, and it’s easy to add more that we want to make sure work correctly. (I’m also thinking about user support, where we can ask to send us a problematic configuration written out to a json file, that we can then add to our unit tests, fix, and ensure that it never breaks again.
The wayland test server is only about 500 lines of relatively simple code, but it provides full functionality for setting up screens using the wayland protocol.

Next steps…

The real kwin_wayland will use the exact same library, on the server as we do in our tests, but instead of using “virtual screens”, it does actually interact with the hardware, for example through libdrm on more sensible system or through libhybris on ones less so.
Kwin takes a more central role in our wayland story, as we move initial mode-setting there, it just makes to have it do run-time mode setting as well.

The next steps are to hook the server side of the protocol up in kwin_wayland’s hardware backends.

In the back of my head are a few new features, which so far had a lower priority — first the core feature set needed to be made stable. There are three things which I’d like to see us doing:

  • per-display scaling — This is an interesting one. I’d love to be able to specify a floating point scaling factor. Wayland’s wl_output interface, which represents the application clients, only provides int-precision. I think that sucks since there is a lot of hardware around where a scaling factor of 1 is to small, and 2 is too high. That’s pretty much everything between 140 and 190 DPI according to my eyesight, your mileage may vary here. I’m wondering if I should go ahead and add the necessary API’s at least on our end of the stack to allow better than integer precision.
    Also, of course we want the scaling be controlled per display (and not globally for all displays, as it is on X11), but that’s in fact already solved by just using wayland semantics — it needs to be fixed on the rendering side now.
  • pre-apply checks — at least the drm backend will allow us to ask it if it will be able to apply a new configuration to the hardware. I’d love to hook that up to the UI, so we can do things like enable or disable the apply button, and warn the user of something that the hardware is not going to like doing.
    The low-level bits have arrived with the new drm infrastructure in the kernel, so we can hook it up in the libraries and the user interface.
  • configuration profiles — it would make sense to allow the user to save configurations for different situations and pick between them. It would be quite easy to allow the user to switch between setups not just through the systemsettings ui, but also for example when connecting or disabling a screen. I an imagine that this could be presented very nicely, and in tune with graphical effects that get their timing juuuuust right when switching between graphics setups. Let’s see how glitch-free we can make it.
November 12th, 2015

wayland and libkscreen benchmarks

So, first of all, this is all very much work-in-progress and highly experimental. It’s related to the work on screen management which I’ve outlined in an earlier article.

libkscreen wayland benchmark data

I ran a few benchmarks across our wayland stack, especially measuring interprocess communication performance when switching from X11 (or, in fact XCB and XRandR) to wayland. I haven’t done a highly scientific setup, just ran the same code with different backends to see how long it takes to receive information about screens connected, their modes, etc..
I also ran the numbers when loading the libkscreen backend in-process, more on that later.

data

The spreadsheet shows three data columns, in vertical blocks per backend the results for 4-5 individual runs and their mean values. One column for the default out-of-process mode, one with loading the backend in process and one showing the speedup between in- and out-of-process of the same backend.
The lower part contains some cross referencing of the mean values to compare different setups.
All values are nano seconds.

2x

My results show a speedup of between 2 and 2.5 times when querying screen information on X11 and on wayland, wayland being much faster here.
The qscreen and xrandr backends perform pretty similar, they’re both going through XCB. That checks out. The difference between wayland and xrandr/qscreen can then be attributed to either the wayland protocol or its implementation in KWayland being much faster than the corresponding XCB implementations.

But, here’s the kicker…

in- vs. out-of-process

The main overhead, as it turns out, is libkscreen loading the backend plugins out-of-process. That means that it starts a dbus-autolaunched backend process and then passes data over DBus between the libkscreen front-end API and the backend plugin. It’s done that way to shield the client API (for example the plasma shell process or systemsettings) from unsafe calls into X11, as it encapsulates some crash-prone code in the XRandR backend. When using the wayland backend, this is not necessary, as we’re using KWayland, which is much safer.
I went ahead and instrumented libkscreen in a way that these backends are being loaded in process, which avoids most of the overhead. This change has an even more dramatic influence on performance: on X11, the speedup is 1.6x – 2x, on wayland loading the backend in-process makes it run 10 times faster. Of course, these speedups are complementary, so combined, querying screen information on wayland can be done about 20 times faster.

While this change from out-of-process to in-process backends introduces a bit more complexity in the library, it has a couple of other advantages additional to the performance gains. In-process means that debugging is much easier. If there are crashes, we do not hide them anymore, but identify and fix them. It also makes development more worthwhile, since it’s much easier to debug and test the backends and frontend API together. It also means that we can load backend plugins at the same time.

I’ve uploaded the benchmark data here. Before merging this, I’ll have to iron out some more wrinkles and have the code reviewed, so it’s not quite ready for prime-time yet.

November 6th, 2015

Screen management in Wayland

One of the bigger things that is in the works in Plasma’s Wayland support is screen management. In most cases, that is reasonably easy, there’s one screen and it has a certain resolution and refresh rate set. For mobile devices, this is almost always good enough. Only once we starting thinking about convergence and using the same codebase on different devices, we need to be able to configure the screens used for rendering. Especially on desktops and laptops, where we often find multi-monitor setups or connected projectors is where the user should be able to decide a bunch of things, relative position of the screens, resolution (“mode”) for each, etc.. Another thing that we haven’t touched yet is scaling of the rendering per display, which becomes increasingly important with a wider range of displays connected, just imagine a 4K laptop running north of 300 pixels per inch (PPI) connected to a projector which throws 1024*768 pixels on a wall sized 4x3m.

The Wayland protocol currently does not provide a mechanism for setting up the screen, or tell us about displays that are not used for rendering, either because they’re disabled, or have just been connected, but not enabled “automatically” (yet). For most applications, that doesn’t matter, they’re just fine with knowing about the rendering screens and some details about those, which is provided by the wl_output interface. For screen management, this interface is insufficient, though, since it lacks a few things, EDID information, enabled/disabled flags and also ways to set the mode, scaling, rotation and position. This makes clearly recognizing display and setting them up harder than necessary, and thus error-prone. Let’s look at the background, first, however.

Setting up X11

On the hardware side, this has been a complete mess in the past. One problem is X11’s asynchronous nature. The XRandR extension that is used for this basically works by throwing a bunch of calls to the X server (“use this mode”, “position display there”) and then seeing what sticks to the wall. The problem is that we never really know what happened, there’s no well-defined “OK, this works” result, and we also don’t know when the whole procedure is done. The result is a flicker-fest and the desktop trying to catch up with what X11 made of the xrandr calls. It can also be an unpleasant experience, when a display gets connected, used for rendering, then the shell finds out about it, expanding the desktop to it, and then everything is resized again because there’s a predefined configuration for this. These kind of race conditions are very hard to fix due to the number of components involved in the process, and the lack of proper notification semantics around it.

X11 has the nasty habit of interacting with hardware directly, rather than through well-defined and modern kernel interfaces. On the kernel side, this has been fixed. We now have atomic mode setting, which allows us to check whether changes can be applied (through the DRM_MODE_ATOMIC_TEST_ONLY flag), and apply them all at once, or in portions that are known to not screw up, lock the user out, or are simply invalid in context with each other.
For the user, getting this right across the whole stack means quicker reconfiguration of the hardware and only minimal flickering when switching screen setups. We won’t be able to completely prevent the flickering on most displays, as that is simply how the hardware works, but we will be able to make it a lot less jarring. The compositor now being the one that calls the DRM subsystem on the user side, we can coordinate these things well with visual effects, so we’ll be able to make the user experience while re-configuring displays a bit smoother as well.

Atomic mode setting, DRM and kernel

From the kernel side, this needed quite radical changes, which have now landed throughout the DRM subsystem. The result is a kernel interface and helper library that allows interacting with the kernel using semantics that allow tighter control of the processes, better error prevention and handling and more modern power management semantics. Switching off the screen can now be done from the compositor, for example — this allows us to fix those cases where the display is black, but still has its backlight on, or where the display is off, but used for rendering (in which case you get huge blind spots in your user interface).
Daniel Vetter’s (part 1, part 2) provides an excellent overview over history, present and future of atomic mode setting on the kernel side. Pertaining is that a reasonably recent Linux kernel with working DRM drivers now provides all that we need to fix this problem on the user side. X11 is still in the way of a completely smooth solution, though.

Screen setup in Plasma

In Plasma, the screens can be set up using the Display configuration module in system settings. This module is internally called “KScreen”. KScreen provides a visual interface to position displays, set resolution, etc.. It’s backed by a daemon that can apply a configuration on login – useful stuff, but ultimately bound by the limits across the underlying software stack (X11, kernel, drivers, etc.).

KScreen is backed by libkscreen, a library that we ship with Plasma. libkscreen offers an API that allows to list displays, their properties, including disabled displays. libkscreen is driven by out-of-process running backends, commonly used is the “xrandr” backend, which talks to the X Server over the XRandR extension. libkscreen has other backends, notably a read-only QScreen backend a “fake” backend used for unit tests. A native Wayland backend is work in progress (you can find it in the libkscreen[sebas/wayland] branch.)

libkscreen been developed for the purpose of screen configuration, but we have also started using it for the Plasma shell. QScreen, the natural starting point of this was not up to the task yet, missing some functionality. In Qt 5.6, Aleix Pol has now landed the necessary missing functionality, so we can move the Plasma shell back onto QScreen entirely. QScreen is backed by the XCB Qt platform plugin (QPA). One problem in Plasma has been that we got acknowledged of changes through different code paths, which made it hard to set up the desktop, position panels, etc. In a Wayland session, this has to happen in a much more coordinated way, with clearly defined semantics when the screen setup changes, and as little of those changes as necessary.
KScreen should concentrate on doing what it’s good at: screen configuration. For X11 kscreen uses its xrandr backend, no changes there. In Plasma shell’s startup, we will be able to remove libkscreen and rely purely on QScreen directly as soon as we can depend on Qt 5.6, so that probably puts us into the time-frame of Q2 next year. For read-only access on wayland, we can use the libkscreen QScreen backend for now, it comes with some limitations around multi-screen, but these will be ironed by spring next year. The QScreen backend is actually used to start Plasma Mobile’s on kwin_wayland. For configuration, QScreen is not an option, however — it’s simply not its purpose and shouldn’t be.

In the Wayland protocol itself, there are no such semantics yet. Screen configuration has, so far, been outside of the scope of the agreed-upon wayland protocols. If we don’t run on top of an X server, who’s doing the actual hardware setup? Our answer is: KWin, the compositor.

KWind

KWin plays a more central role in a Wayland world. For rendering and compositing of windows, it interacts with the hardware. Since it already initializes hardware when it starts a Wayland server, it makes a lot of sense to put screen configuration also exactly there. This means that we will configure KWin at runtime through an interface that is designed around semantics of atomic mode setting, and KWin picks a suitable configuration for connected displays. KWin saves the configuration, applies it on startup or when a display gets switched off, connected or disconnected, and only then tells the workspace shell and the apps to use it. This design makes a lot of sense, since it is KWin that ultimately knows of all the constraints related to dynamic display configuration, and it can make concert how the hardware is used and how its changes are presented to the applications and workspace.

KWayland and unit testing

Much of Kwin/Wayland’s functionality is implemented in a library called KWayland. KWayland wraps the wayland protocol with a Qt-style API for wayland clients and servers, offers threaded connection and type-safety on top of the basic C implementation of libwayland.
KWayland provides a library that allows to run wayland servers, or just specific parts of it with very little code. The KWayland server classes allow us to test a great deal of the functionality in unittests, since we can run the unit tests on a “live” wayland server. Naturally, this is used a lot in kwayland’s own autotests. In the libkscreen wayland backend’s tests, we’re loading different configuration scenarios from json definitions, so we can not only test whether the library works in principle, but really test against live servers, so we cover a much larger part of the stack in our tests. This helps us a lot to make sure that the code works in the first place, but also helps us catch problems easily as soon as they arise. The good unit test coverage also allows much swifter development as a bonus.

Output management wayland interface design

The output management wayland protocol that we have implemented provides two things:

  • It lists connected output hardware and all of their properties, EDID, modes, physical size, and runtime information such as currently used mode, whether this output device is currently enabled for rendering, etc.
  • It provides an interface to change settings such as mode, rotation, scaling, position, for hardware and to apply them atomically

This works as follows:

  1. The server announces that the global interfaces for OutputManagement and a list of OutputDevices is available
  2. The configuration client (e.g. the Display Settings) requests the list of output devices and uses them to show the screen setup visually
  3. The user changes some settings and hits “apply”, the client requests an OutputConfiguration object from the OutputManagement global
  4. The configuration object is created on the server specifically for the client, it’s not exposed in the server API at this point.
  5. The client receives the config object and calls setters with new settings for position, mode, rotation, etc.
  6. The server buffers these changes in the per-client configuration object
  7. The client is done changing settings and asks the server to apply them
  8. The compositor now receives a sealed configuration object, tests and applies the new settings, for example through the DRM kernel interface
  9. The compositor updates the global list of OutputDevices and changes its setup, then it signals the client failure or success back through the configuration object

The output management protocol, client- and server-side library, unit tests and documentation are quite a hefty beast, combined they come in at ca. 4700 lines of code. The API impact, however, has been kept quite low and easy to understand. The atomic semantics are reflected in the API, and it encourages to do the right thing, both for the client configuring the screens, and the compositor, which is responsible for applying the setup.

Next steps

I am currently working on a libkscreen module for screen configuration under wayland, that implements atomic mode setting semantics in libkscreen. It uses a new wayland protocol which Martin Gräßlin and I have been working on in the past months. This protocol lands with the upcoming Plasma 5.5, the libkscreen module may or may not make the cut, this also depends on if we get the necessary bits finished in KWin and its DRM backend. That said, we’re getting really close to closing the last gaps in the stack.

On the compositor side, we can now connect the OutputManagement changes, for example in the DRM backend and implement the OutputDevices interface on top of real hardware.

September 11th, 2015

Wall drawing

Sol Lewitt's wall drawing at Stedelijk Museum Amsterdam

Sol Lewitt’s wall drawing at Stedelijk Museum Amsterdam

A few weeks ago, I visited the Stedelijk Museum in Amsterdam for an exhibition of Henry Matisse’s works. What stuck is not a painting or collage by Matisse, but a wall drawing by Sol Lewitt. I took a photo with my phone and have since then used it as wallpaper for it, and it works well, colors are nicely toned, everything provides enough contrast, and I like how it looks on a screen.

Yesterday, when I needed a break from hacking on Plasma’s Wayland integration, I remade the photo into vector art to use it as wallpaper. You can download the result here (the wallpaper package has versions for all resolutions, including phone and landscape versions, you can just unzip the package into /usr/share/wallpapers).