Five Musings on Frameworks Quality

Gratuitious ape photo
Musing…

In many cases, high-quality code counts more than bells and whistles. Fast, reliable and well-maintained libraries provide a solid base for excellent applications built on top of it. Investing time into improving existing code improves the value of that code, and of the software built on top of that. For shared components, such as libraries, this value is often multiplied by the number of users. With this in mind, let’s have a closer look of how the Frameworks 5 transition affects the quality of the code, so many developers and users rely on.

KDE Frameworks 5 will be released in 2 weeks from now. This fifth revision of what is currently known as the “KDE Development Platform” (or, technically “kdelibs”) is the result of 3 years of effort to modularize the individual libraries (and “bits and pieces”) we shipped as kdelibs and kde-runtime modules as part of KDE SC 4.x. KDE Frameworks contains about 60 individual modules, libraries, plugins, toolchain, and scripting (QtQuick, for example) extensions.

One of the important aspects that has seen little exposure when talking about the Frameworks 5 project, but which is really at the heart of it, are the processes behind it. The Frameworks project, as it happens with such transitions has created a new surge of energy for our libraries. The immediate results, KF5’s first stable release is a set of software frameworks that induce minimal overhead, are source- and binary-stable for the foreseeable future, are well maintained, get regular updates and are proven, high-quality, modern and performant code. There is a well-defined contribution process and no mandatory copyright assignment. In other words, it’s a reliable base to build software on in many different aspects.

Maturity

Extension and improvement of existing software are two ways of increasing their values. KF5 does not contain revolutionary new code, instead of extending it, in this major cycle, we’re concentrating on widening the usecases and improving their quality. The initial KDE4 release contained a lot of rewritten code, changed APIs and meant a major cleanup of hard-to-scale and sometimes outright horrible code. Even over the course of 4.x, we had a couple of quite fundamental changes to core functionality, for example the introduction of semantic desktop features, Akonadi, in Plasma the move to QML 1.x.
All these new things have now seen a few years of work on them (and in the case of Nepomuk replacing of the guts of it with the migration to the much more performant Baloo framework). These things are mature, stable and proven to work by now. The transition to Qt5 and KF5 doesn’t actually change a lot about that, we’ve worked out most of the kinks of this transition by now. For many application-level code using KDE Frameworks, the porting will be rather easy to do, though not zero effort. The APIs themselves haven’t changed a lot, changes to make something work usually involve updating the build-system. From that point on, the application is often already functional, and can be gradually moved away from deprecated APIs. Frameworks 5 provides the necessary compatibility libraries to ease porting as much as possible.
Surely, with the inevitable and purposeful explosion of the user-base following a first stable release, we will get a lot of feedback how to further improve the code in Frameworks 5. Processes, requirements and tooling for this is in place. Also, being an open system, we’re ready to receive your patches.
Frameworks 5, in many ways encodes more than 15 years of experience into a clearly structured, stable base to build applications for all kinds of purposes, on all kinds of platforms on.

Framework Caretakers

With the modularization of the libraries, we’ve looked for suitable maintainers for them, and we’ve been quite successful in finding responsible caretakers for most of them. This is quite important as it reduces bottlenecks and single points of failure. It also scales up the throughput of our development process, as the work can be shared across more shoulders more easily. This achieves quicker feedback for development questions, code review requests, or help with bug fixes. We don’t actually require module maintainers to fix every single bug right away, they are acting much more as orchestrators and go-to-guys for a specific framework.

More Reviews

More peer-review of code is generally a good thing. It provides safety nets for code problems, catches potential bugs, makes sure code doesn’t do dumb thing, or smart things in the wrong way. It also allows transfer of knowledge by talking about each others code. We have already been using Review Board for some time, but the work on Frameworks 5 and Plasma 5 has really boosted our use of review board, and review processes in general. It has become a more natural part of our collaboration process, and it’s a very good thing, both socially and code-quality-wise.
More code reviews also keeps us developers in check. It makes it harder to slip in a bit of questionable code, a psychological thing. If I know my patches will be looked at line-by-line critically, it triggers more care when submitting patches. The reasons for this are different, and range from saving other developers some time to point out issues which I could have found myself had I gone over the code once more, but also make me look more cool when I submit a patch that is clean and nice, and can be submitted as-is.
Surely, code reviews can be tedious and slow down the development, but with the right dose, in the end it leads to better code, which can be trusted down the line. The effects might not be immediately obvious, but they are usually positive.

Tooling

Splitting up the libraries and getting the build-system up to the task introduced major breakage at the build-level. In order to make sure our changes would work, and actually result in buildable and working frameworks, we needed better tooling. One huge improvement in our process was the arrival of a continuous integration system. Pushing code into one of the Frameworks nowadays means that a it is built in a clean environment and automated tests are run. It’s also used to build its dependencies, so problems in the code that might have slipped the developer’s attention are more often caught automatically. Usually, the results of the Continuous integration system’s automated builds are available within a few minutes, and if something breaks, developers get notifications via IRC or email. Having these short turnaround cycles makes it easier to fix things, as the memory of the change leading to the problem is still fresh. It also saves others time, it’s less likely that I find a broken build when I update to latest code.
The build also triggers running autotests, which have been extended already, but are still quite far away from complete coverage. Having automated tests available makes it easier to spot problems, and increases the confidence that a particular change doesn’t wreak havoc elsewhere.
Neither continuous builds, nor autotests can make 100% sure that nothing ever breaks, but it makes it less likely, and it saves development resources. If a script can find a problem, that’s probably vastly more efficient than manual testing. (Which is still necessary, of course.)
A social aspect here is that not a single person is responsible if something breaks in autobuilds or autotests, it rather should be considered a “stop-the-line” event, and needs immediate attention — by anyone.

Continuous Improvement

This harnessing allows us to concentrate more on further improvments. Software in general are subject to a continous evolution, and Frameworks 5.0 is “just another” milestone in that ongoing process. Better scalability of the development processes (including QA) is not about getting to a stable release, it supports the further improvement. As much as we’ve updated code with more modern and better solutions, we’re also “upgrading” the way we work together, and the way we improve our software further. It’s the human build system behind software.

The circle goes all the way round, the continuous improvement process, its backing tools and processes evolve over time. They do not just pop out of thin air, they’re not dictated from the top down, they are rather the result of the same level of experience that went into the software itself. The software as a product and its creation process are interlinked. Much of the important DNA of a piece of software is encoded in its creation and maintenance process, and they evolve together.