Ok, that is a bit harsh, but still. Release parties are organised all around the world to celebrate the release of KDE 4.8. You should go to one, most of them are relaxed, friendly and allows you to meet your local developers, contributors or just fellow KDE users.

There is also a Release Party planned in Culemborg, The Netherlands. It’s just south of Utrecht, a beautiful city in the center of the country. In the middle of the region known for the jam-production. The release party won’t have any jam though, we will stick to Pizza, cola, a soccer table, a presentation done by Sebastian Kügler and a demo of Plasma Active.

Entrance is free and everyone who wants to come is welcome. All you have to do is sign up.

During KDE’s (one of the largest open source communities around, with about 2500 active developer accounts with direct write access to many millions of lines of code across dozens of products, and large numbers of external contributors) ongoing migration from SVN to Git, GitHub was never considered as an option because the community considers it unacceptable for an open source community to throw their weight behind a proprietary solution.

Instead we seriously considered the open source alternative Gitorious.org for a while, but ultimately were unable to come to an agreement with Shortcut AS with regard to a satisfactory hosting plan, and also had troubles making its ACL system work for our needs in some exploratory migrations to the platform.

Self-hosting the Gitorious software also fell out of the running fairly quickly due to the ACL problems and other concerns about the software’s features and implementation; while KDE probably would have been able to put together a squad of coders to work on the software in its interest, Shortcut at the time had failed to get a dev community around Gitorious off the ground and was unacceptably slow at processing merge requests – meaning a fork might have become necessary, which is always an icky proposition not to mention a big burden on resources.

Ultimately we ended up banging together our own Git infrastructure out of several smaller components, some of which we wrote ourselves:

• To handle access controls as well as the developer-facing repo management knobs (personal server-side clones, personal scratch repos, access management for those, a trashcan for repos, the works) we chose gitolite, which was definitely the best choice we could have made. It’s reliable, well-documented, fast and rich in features, and its principal developer Sitaram Chamarty has been an incredible aid to our efforts both with many hours of work spent coding additional features we needed and tons of advice. We did write a couple of add-ins to gitolite ourselves to provide some additional commands to developers, which we found pleasant to do.
• As web-based repository viewer we use an inhouse-modified version of Redmine. As with gitolite, we’ve found the Redmine crew to be easy to approach and get help from, and while we ran into some trouble with the scalability of its Git support, it proved easy enough to modify without requiring massive forking. Redmine’s database of projects serves a vital role in the system in that the gitolite config is generated from Redmine’s database (e.g user access roles), and we also generate a custom-format XML from it that is used by end-user clone/build tools, our translation infrastructure, LXR and many other bits and pieces that have a need to discover projects and repositories and their metadata (which can be modified by project managers in the Redmine web UI, e.g. setting description texts or translation branches).
• For review, we use ReviewBoard, continuing on from SVN. This is definitely the weak spot in our setup right now; ReviewBoard just doesn’t support Git very well since it’s designed to handle single patches rather than series of them and thus also can’t track branches. We’ve been looking at Gerrit on and off to replace it, but so far can’t really bring ourselves to commit to it since it’s a fairly unwieldy beast.
• We wrote our own repository hooks in Python, which are in charge of things like sending out commit mails to mailing lists, CIA.vc, the bug tracker and people mentioned in commit messages behind special keywords, evaluating other keywords to close bug tickets or review requests, check for trivial errors (EOL style, license headers, that sort of thing), ref backups and a bunch of other tasks. Some of which run through asynchronous job queues as required to avoid blocking.
• We wrote our own mirroring system to provide several read-only git:// mirror servers behind a geo-ip DNS load balancer. The mirrors get updated whenever a push to the master happens.
• KDE has an LDAP-based single sign-on system in use at many of its web properties including Redmine and ReviewBoard, currently using an inhouse-modified (mostly expanded with our own modules) version of GOsa as web frontend. GOsa is also used to request developer privileges and manage the SSH keys for Git write access.
• There are a couple of other minor bits and pieces flying about, like a commits.kde.org/ web app written in Ruby+Sinatra that forwards to the Redmine page for the commit; those URLs pop up in various places, e.g. in a developer’s terminal output after pushing, in mail headers, CIA.vc postings, etc.

KDE’s migration to Git is still ongoing, with many large and actively developed codebases still remaining in SVN at this time, as well as the entire translation effort. To actually migrate things we use our own tool called svn2git which takes raw SVN repo data, an account conversion map (supplied as plain text generated from our LDAP) and a plain text file with regex-based rules as input and produces git repositories. Writing the necessary rules files is unfortunately a time-intensive task especially given some very complex project histories originally imported from CVS, but additional tools exist to partially auto-generate them by analyzing SVN history programmatically.

Here’s a couple of links of interest:

• The developer-facing user manual for KDE’s Git infra: http://community.kde.org/Sysadmin/GitKdeOrgManual
• The initial posting of the sysadmin working group’s plan for the above setup, and our evaluation of this stack vs. the Gitorious software (ODF of the latter is attached to the first reply): http://mail.kde.org/pipermail/kde-scm-interest/2010-June/001…
• The XML export from Redmine mentioned above: https://projects.kde.org/kde_projects.xml

But the thing is, this feature was discussed a long time ago, see for example the famous sysadmin advice. The xml-file making it possible for kdesrc-build to work like it does was developed shortly after the first extragear applications started the migration to git. With great input from Eike Hein and implemented by Ben Cooksley. I want to give them both a big Thank You as well! Well done.

The other existing search engine is lxr.kde.org, that one is a bit underrated, but it is a great tool to find examples on how you can use a certain class.

But sometimes you just want a free search, especially when you don’t know if there is a class available you need. So, to fill that gap, I’ve done a full text index of api.kde.org and whipped up a small interface to search it. It also provides a xml-output, rumor is that Nicolás Alvarez will make a nice krunner out of it. The search is powered by Xapian/Omega which is super fast it seems.

Currently I would appreciate some feedback if you try it out; about if this would be a usefull addition, if it should be merged with the default search on api.kde.org, if it would make lxr useless, comments about the interface, ah well, everything you think is welcome in the comments.

Try it here: http://api.kde.org/search.

The other option is that you are using different passwords for each site, but that means you are probably writing them down, on paper, a text file or in a application as KWallet or KeePassX. KeePassX is very convenient as you can generate large passwords easily. As you never ever have to type them, who cares about how long they are. And in general, how longer the password the better it becomes (Reliable proof).

But there is a catch, whenever your password is intercepted, the interceptor can use this password to login to the site without your knowledge. This can very well be a man-in-the-middle attack; someone eavesdropping your connection. The government can do that, your telephone company which might do deep packet inspection to see if you are using skype instead of their network, a key-logger/virus on your computer (is there still anyone who believes those don’t exist on Linux?) or some malicious code running on the server which serves the website.

Https based sites partially solves some of the listed problems, but key-loggers or malicious webmasters on the other hand can still get to the passwords if they want to. If you are using the same password on multiple sites, you get more vulnerable very fast.

Some banks use a calculator, you have to punch in some codes on a small device, it gives you back a code and you type it in on the website and get access. But such a calculator alone is not safe. It can get stolen easily. But you don’t get access only by the calculator, but you need a password too. That’s called Two factor authentication: it is “something you know” (password) and “something you have” (calculator).

Say your password gets compromised, they still need the calculator to get access. The other way around, say someone steels your calculator, they can not get into your account, as they don’t know the password. As both are needed, you are dramatically increasing the safety around the access to the data on the website.

Most sites don’t offer two factor authentication in any way (there are work arounds, you can look at yubikey for example). There is one important site that offers two factor authentication though: Google. Google is getting increasingly important. For some people it gives you access to your secret circles with private info, it holds the all important spreadsheet with financial informations and also holds all your private mail (/me waves at all the eV-members!). Think of how you feel if that data ends up on a public website… (thinking it won’t happen to such a big company is very naive).

I’ld like to call out to everyone to activate Googles two factor authentication. It works like this: you can login normally, with your username and password, only after you’ve done that, you are asked for an additional code. This code can be generator by a special app running on your android or iPhone-phone, it can be send as a text/sms message to your phone, or you can use codes that you have previously printed. You only have to it once in the 30 days, and face it, most people have their phone with them at all times, so it’s hardly a problem.



Setting it up is pretty straight forward. Go to your account settings page, click the link and follow the instructions. It will make your life a bit more safe. It’s worth the trouble. Now it’s time to think how we can integrate something like that to identity.kde.org.

I’m very, very, very happy with the award. I also find it awesome that fellow members of the sysadmin-team are mentioned. Although it is assigned to me, it is one for the team.

The last years I’ve shifted from translator, to developer to sysadmin. I started as a translator to dutch, then hacked on Digikam, started a very good mailclient, which never picked up somehow, started RSIBreak, which is picking up and then became Sysadmin after Dirk asked me to get involved.

I know the current infrastructure we currently have is quite good. It has improved the last years and we have almost no downtimes (knock on wood). Even the major release of 4.7 went fine, we saw some amazing stats like the www.kde.org-server, which had over 100GB traffic on the day after the release. We stll have some nice plans for the future, like a continious build cluster. These are no tasks for an individual, you need a group for that.

Getting the award makes me feel happy, it is a recognition of the work done by the team, so I want to congratulate them too: Ben, Eike, Dirk, David, Ingo and Jeff. Also the other winners of the awards, Dario and Martin, well deserved! You are doing amazing work, it feels a bit silly to be mentioned in one sentence with you two heros.

I hope I will be able to serve the community as best as I can for the times to come! Thanks all.
Ps. Do winners of the award still get a Qt green phone? :-)

Have you ever donated? You might find your self in the list of donations.

There are some area’s we need some further work, like the spam setup and we see some bounces which could be valid or not valid. Seems like the bounce processing of mailman in the previous setup might not have been working correctly, so we see an above average of automatic unsubscribes from mailinglists to catch up with that.

The old server was qmail based, a small mail server, which has enormous speed and it served us well for the last ~decade, but it showed its age. We have now switched to Postfix, a modern mail server, which gives us a bit more control to the queues, better logging and it’s well known, so easy to get support for.

Enjoy the new mail server, and if there are any problems, file a sysadmin bugreport as usual. Dirk did the work on the new server, so hug him when you see him.

KTown was a multi-function server, it performed a whole range of services, ranging from serving some sites, being the main mailserver for all kde.org traffic, being a stealth nameserver and serving the master source tarballs for everything we produce.

Once we decided to decommission the server we drafted a plan to move all the services around and organize them better. We moved the sites to our master webserver and rethought about the nameserver setup. Historically domain names were mostly registered by individuals, we have transferred most of those now to the KDE e.V.. Registering domainnames privately is just a bad idea. Times change, people come and go. In the end getting changes done from people who have been inactive for years already is no fun. I’m happy we have solved this now. Last week we even transferred kde.org, it was registered by Trolltech in the past. Though it was in good hands with the sysadmins of Nokia, we felt it was better to keep everything together.

For all those domains we need nameservers. bytemark.co.uk was kind enough to provide us some nameservers we can use.

Another server we worked on was Immanuel. The hosting arrangement is a bit dodgy, so we can not rely on it being repaired quickly after it crashes. That means we can not put mission critical stuff on there anymore. That means we needed to move stuff around. That means bugzilla had to move, we originally planned that to happen while upgrading to bugzilla4, but we had to move it before that was ready. Immanuel also served some websites which we moved around.

Next to that some remarkable things are: move of kdedevelopers.org, including a new layout, usual drupal updates and the update of the mediawiki version on our wiki’s.

Since that’s all done, we are looking towards new things. We have two somewhat bigger wishes: deal with api.kde.org / ebn and we want to start building a build farm, not for our own pleasure, but to have continuous builds of our repository and have reports when the unit tests breaks.

Call for a server

But I first want to start with api.kde.org, Allen Winter is currently maintaining it, but I think he needs some help to get it in a better shape. For that to happen, I would like a new server. It needs to have some hard disk space (like 120GB, as it needs to have a full copy of the svn repo + everything in git), a somewhat powerful processor and some memory.

If you are a hosting company and want to donate something like that to us, drop us a mail at sysadmin@kde.org or talk to us on #kde-sysadmin at freenode…

The system relied on the mails which were sent by PayPal after a donation was made. It parsed the content, extracted the information and then generated a piece of html from it and committed that, after that a signal was given to svn up so the donation would show up. At the beginning that probably worked fine. But PayPal was not very consistent with the mails it send out. Format changed, the script was adapted. Sometimes Paypal mails arrived in French or other languages. It became messy and at one point beyond fixing.

So, this week I sat down and have rewritten the stuff. We now use the PayPal API which provides a method to receive the payment details. This means the donation gets directly recorded and we put it in a database table. The user is redirected back to the kde site and can see his donation and the message he left instantly. Pretty straightforward in the end, just needed to get done. See the result (including a graph) here.

If you wish to test the system and make a donation to KDE e.V. at the same time, go here, click on the yellow Donate button and don’t forget to leave a nice message when you make the donation!