Five Musings on Frameworks Quality

Gratuitious ape photo

Musing…


In many cases, high-quality code counts more than bells and whistles. Fast, reliable and well-maintained libraries provide a solid base for excellent applications built on top of it. Investing time into improving existing code improves the value of that code, and of the software built on top of that. For shared components, such as libraries, this value is often multiplied by the number of users. With this in mind, let’s have a closer look of how the Frameworks 5 transition affects the quality of the code, so many developers and users rely on.

KDE Frameworks 5 will be released in 2 weeks from now. This fifth revision of what is currently known as the “KDE Development Platform” (or, technically “kdelibs”) is the result of 3 years of effort to modularize the individual libraries (and “bits and pieces”) we shipped as kdelibs and kde-runtime modules as part of KDE SC 4.x. KDE Frameworks contains about 60 individual modules, libraries, plugins, toolchain, and scripting (QtQuick, for example) extensions.

One of the important aspects that has seen little exposure when talking about the Frameworks 5 project, but which is really at the heart of it, are the processes behind it. The Frameworks project, as it happens with such transitions has created a new surge of energy for our libraries. The immediate results, KF5’s first stable release is a set of software frameworks that induce minimal overhead, are source- and binary-stable for the foreseeable future, are well maintained, get regular updates and are proven, high-quality, modern and performant code. There is a well-defined contribution process and no mandatory copyright assignment. In other words, it’s a reliable base to build software on in many different aspects.

Maturity

Extension and improvement of existing software are two ways of increasing their values. KF5 does not contain revolutionary new code, instead of extending it, in this major cycle, we’re concentrating on widening the usecases and improving their quality. The initial KDE4 release contained a lot of rewritten code, changed APIs and meant a major cleanup of hard-to-scale and sometimes outright horrible code. Even over the course of 4.x, we had a couple of quite fundamental changes to core functionality, for example the introduction of semantic desktop features, Akonadi, in Plasma the move to QML 1.x.
All these new things have now seen a few years of work on them (and in the case of Nepomuk replacing of the guts of it with the migration to the much more performant Baloo framework). These things are mature, stable and proven to work by now. The transition to Qt5 and KF5 doesn’t actually change a lot about that, we’ve worked out most of the kinks of this transition by now. For many application-level code using KDE Frameworks, the porting will be rather easy to do, though not zero effort. The APIs themselves haven’t changed a lot, changes to make something work usually involve updating the build-system. From that point on, the application is often already functional, and can be gradually moved away from deprecated APIs. Frameworks 5 provides the necessary compatibility libraries to ease porting as much as possible.
Surely, with the inevitable and purposeful explosion of the user-base following a first stable release, we will get a lot of feedback how to further improve the code in Frameworks 5. Processes, requirements and tooling for this is in place. Also, being an open system, we’re ready to receive your patches.
Frameworks 5, in many ways encodes more than 15 years of experience into a clearly structured, stable base to build applications for all kinds of purposes, on all kinds of platforms on.

Framework Caretakers

With the modularization of the libraries, we’ve looked for suitable maintainers for them, and we’ve been quite successful in finding responsible caretakers for most of them. This is quite important as it reduces bottlenecks and single points of failure. It also scales up the throughput of our development process, as the work can be shared across more shoulders more easily. This achieves quicker feedback for development questions, code review requests, or help with bug fixes. We don’t actually require module maintainers to fix every single bug right away, they are acting much more as orchestrators and go-to-guys for a specific framework.

More Reviews

More peer-review of code is generally a good thing. It provides safety nets for code problems, catches potential bugs, makes sure code doesn’t do dumb thing, or smart things in the wrong way. It also allows transfer of knowledge by talking about each others code. We have already been using Review Board for some time, but the work on Frameworks 5 and Plasma 5 has really boosted our use of review board, and review processes in general. It has become a more natural part of our collaboration process, and it’s a very good thing, both socially and code-quality-wise.
More code reviews also keeps us developers in check. It makes it harder to slip in a bit of questionable code, a psychological thing. If I know my patches will be looked at line-by-line critically, it triggers more care when submitting patches. The reasons for this are different, and range from saving other developers some time to point out issues which I could have found myself had I gone over the code once more, but also make me look more cool when I submit a patch that is clean and nice, and can be submitted as-is.
Surely, code reviews can be tedious and slow down the development, but with the right dose, in the end it leads to better code, which can be trusted down the line. The effects might not be immediately obvious, but they are usually positive.

Tooling

Splitting up the libraries and getting the build-system up to the task introduced major breakage at the build-level. In order to make sure our changes would work, and actually result in buildable and working frameworks, we needed better tooling. One huge improvement in our process was the arrival of a continuous integration system. Pushing code into one of the Frameworks nowadays means that a it is built in a clean environment and automated tests are run. It’s also used to build its dependencies, so problems in the code that might have slipped the developer’s attention are more often caught automatically. Usually, the results of the Continuous integration system’s automated builds are available within a few minutes, and if something breaks, developers get notifications via IRC or email. Having these short turnaround cycles makes it easier to fix things, as the memory of the change leading to the problem is still fresh. It also saves others time, it’s less likely that I find a broken build when I update to latest code.
The build also triggers running autotests, which have been extended already, but are still quite far away from complete coverage. Having automated tests available makes it easier to spot problems, and increases the confidence that a particular change doesn’t wreak havoc elsewhere.
Neither continuous builds, nor autotests can make 100% sure that nothing ever breaks, but it makes it less likely, and it saves development resources. If a script can find a problem, that’s probably vastly more efficient than manual testing. (Which is still necessary, of course.)
A social aspect here is that not a single person is responsible if something breaks in autobuilds or autotests, it rather should be considered a “stop-the-line” event, and needs immediate attention — by anyone.

Continuous Improvement

This harnessing allows us to concentrate more on further improvments. Software in general are subject to a continous evolution, and Frameworks 5.0 is “just another” milestone in that ongoing process. Better scalability of the development processes (including QA) is not about getting to a stable release, it supports the further improvement. As much as we’ve updated code with more modern and better solutions, we’re also “upgrading” the way we work together, and the way we improve our software further. It’s the human build system behind software.

The circle goes all the way round, the continuous improvement process, its backing tools and processes evolve over time. They do not just pop out of thin air, they’re not dictated from the top down, they are rather the result of the same level of experience that went into the software itself. The software as a product and its creation process are interlinked. Much of the important DNA of a piece of software is encoded in its creation and maintenance process, and they evolve together.

5 Responses to “Five Musings on Frameworks Quality”

  1. There is a major fundamental problem with KDE. We think that somehow exposing every possible setting, implementing every possible feature and use case in Plasma Desktop itself would make it surprisingly better. The two examples that come to my mind are as follows.

    1. Desktop settings – there are 6 different types of layouts and there are 9 different types of wallpapers that can be used. Some of them don’t even work, for example picture of the day. When developers introduce 6×9 different combinations they and users tend to forget about the things that are bad design, for example when you change layout all your wallpapers are gone from the desktop settings.

    2. System settings – somehow someone thought that system settings should not be system settings for an editor for different stuff like a full fledged theme editor – you could make every possible change to the current theme. We such a powerful tools at our hands yet Plasma is shipped with Windows 98 colors. To change the overall look of your system you have to visit 10 different places in the system settings.

    Unless there is this fundamental change is thinking of KDE where we only ship product that largely bug free and all stock components are fully supported I don’t think we could call KDE as the best.

    • Relax, we’re on it!
      – About wallpaper types not working – report a bug, please!

      – About appearance settings being spread over a dozen different settings modules: We are aware of that, and the plan is to have “megathemes” which change everything with one click, plus options to fine-tune them (yes, there are many users who want to do that!). See also my blog post and Jens’ blog post for even bolder approaches to this.
      And Heiko’s blog post for ideas for a better organized System Settings.

      – About “Windows 98 colors”. First of all: Oxygen does not use Windows 98 colors. Windows 98 was plain grey, Oxygen isn’t. The other point: Since presumably you follow Planet KDE, you should have seen the screenshots of the new design already. Does this look like Windows 98 to you???

      • I will wait for Plasma 5 to come and I intend to switch to it on Fedora 21 as default this October. If the problem still persist then I will be report a bug. Bugs have tendency to go stale.

        What do you think about including theme editor in system settings? I find this paradigm very dangerous because then all aspects of KDE want to have their fine control settings in system settings. For example, KWin’s animation speed which are as follows Instant > Very Fast > Fast > Normal > Slow > Very Slow > Extremely Slow. I wonder who in their right mind would want to run their system in Extremely Slow animation speed.

        Here is an example of what we miss on when we implement such redundant features. Launch Feedback is set to 30 seconds – you click something it will start in 2 seconds but the icon will keep jumping for 30 seconds. I think more features just make code unmaintainable. We miss on important things.

        BTW I do read all KDE Planet articles. I am learning C++ and hope to resolve some of my problems on my own so that I don’t start sounding like an ass. lol.

    • sebas says:

      Thanks for your comment, Sudhir.

      It would be nice however, if you actually referred to the article in the comments. Your comments are not about Frameworks 5, but about the previous version of Plasma. We’re open for that, mind you, but your comment is vastly offtopic here.

      You could attempt reading the above article and think how these quality process changes related to your concerns. This is not the place, however, to just give a random opinion.

      • My apologies I didn’t intend to divert the discussion. All I was trying to say was that although big things like Frameworks 5 or knowledge management system like PIM are happening in KDE yet usability still takes a back seat. It wasn’t meant to demean any of the courteous work rather just wanted to point what matters to a general user.