After the couple hours sleep (’Is this good for productivity in any way?), we started the last bit of the API review. We reviewed the akonadi changes between the KDE SC 4.4 release and current trunk, renamed method names, watched for const’s and made sure we did not do any binary incompatible changes.

After lunch, we had a small talk about deprecating all old API KDE has which deal with kresources. As Akonadi is the successor of those and that is approaching rapidely, it is time for developers to port all the remaining kresources usages. With marking that API as deprecated, they get nice little warnings while compiling….

After that we talked a bit about the development of kdepim in the next months, where in SVN that will happening, and how that matches the KDE policy around the upcoming freezes. More news on that later. We first need to talk to some more people.

In the meanwhile I adapted the accountwizard to the changes i described yesterday. And with good progress. We are now beta testing it and fixing the remaining bugs. Should be ready for the KDE-PIM 4.5.0 release.

Steven hacked on unit tests and proxymodels (what else, whahaha). Tobias fixed most issues brought up by the API-review and some small fixes in KAdressBook (of course). Volker fixed a couple of KMail bugs, answered loads of questions and stated just now ‘Now that you asked me, i did not really do that much’.

Kevin Krammer worked on the conversion tool for transferring the old data from kmail to the new akonadi based akonadi. It has challenges like reading the old cache folder, which can contain a mix between mbox and maildir files and directories. He promised to blog about that later on.

Sérgio spended his time on KOrganiser, fixed a bunch of bugs and made the journal editor working again. Matthew fixed the default layout for KMail and a bunch of bugs, for example he fixed the hated bug about the ‘kontact special date summary plugin’ (if scrabble had spaces, this would have been a winner) hang now and then. Now ported to Akonadi properly.

Thomas deprecated the KResources API and bugxing and porting on KMail. Kevin Ottens worked the past days on the kimap implementation, writing unit tests, heck a whole framework. Users can activate a logfile to provide debug output about the imap process. That logfile can then be replayed with his framework, to exactly reproduce bugs.

You can see, that we are working in very many area’s at the same time. preparing our software for the next release. More tomorrow…

After lunch we started fixing bugs.For the first time in a while we managed to sync with my imap server. It is so stupid how tiny little things prevent huge things from operating properly. Change a few bits and everything starts to work again. It is so great that at such a meeting everyone is available, we have the imap guy, the akonadi guy and the guy with the problem. there is no escape possible, a solution will be found.

After that I actually was able to run KMail, saw that it wasn’t as bad as I had made it in my mind, and even fixed a tiny bug in there. Imagine that, me fixing bugs in KMail. Don’t get used to it. The rest of the day was filled with everyone hacking and fixing bugs. With this speed stuff should get working for a lot people Real Soon Now.

My blog about accountwizard triggered some nice comments. Some people pointed out security issues, which we had not given much thought, and some people also pointed to Mozilla’s ISPDB. I’ve read the documentation, which is conflicting here and there and unclear in some area’s, but I was pretty quickly convinced that was the right way to go.

I started to create code which can communicate with ISPDB and finished that just now, pretty straightforward overall. The idea is that the user provides his e-mail address and in that will be checked in an online database. If it is the database, the proper settings for that provide are returned. Exactly what we wanted to do with the GHNS integration. The mozilla team has setup some security measures to make sure no attacker can upload malicious scripts. So using that makes sense.

The next days I will adjust the accountwizard to first use this ISPDB primarily. Needs a bit of re-factoring now, but that’s ok. Stay tuned….

This resulted in Volker writing code to make this wizard based. You ask the user what there full name is, etc. guiding them though the whole process. He also made it so that it would be possible for providers to provide a wizard on their own. That means providers could provide a script which only asks the needed information and fills in the other needed data. Like TLS / SSL / CRAM-MD5, etc. They obviously know the correct values to connect to their servers, and it leaves a limited set of questions the end user has to fill in. In the most ideal situation only their username and password.

While Volker did all that work, I kind of picked it up where he left it and decided this system would be an excellent candidate for get hot new stuff. My idea was that these provider based scripts should be downloadable by the end user, although I wanted it to be transparent. That means that when the user enters the wizard, it asks: which provider are you using? The list with providers is automatically fetched from the ghns-server and displayed. If their provider is not available they will be ale to enter the account details manually. That’s now implemented, see this screenshot.

What will strike your attention is, that there is only one provider available. So true. And that’s where my new project will kick in. I want to automate this process. In the idea work flow, the very first user of a provider will have to enter their data manually. After this process it is possible to use that account to create a new template script for that provider, and upload it back to the ghns-server. The second ever user immediately sees that new provider in the script and only has to fill in just a few details. Of course we will do so for some well know providers like gmail, gmx, etc.

What’s particulary interesting in this setup is that the provider wizard are scripted, that means it uses kross and the files are javascript. So each provider can provide their own scripts without needing a development environment or knowledge of C++. Everyone can do it. Companies can provide their own wizard for easy setup for new employees or whatever….

I hope to get started on this somewhere during this meeting. If you want to help, jump in the #akonadi channel. I probably need to create the foundations of this application first, after that help is appreciated, as I probably become distracted again soon after the meeting is over.

In Berlin there is the Akonadi meeting again. We do it twice a year and that seems to be a good interval. Just before each freeze (well, in this case, slightly after). Ah freeze, that reminds me of a) the temperature inside this cabin and b) the silence which arrived after the small argument… Back to topic. I’ve not hacked much the last couple of months, work work gets in the way sometimes. The spare time I had, I spent mostly on sysadmin tasks, of which Eike Hein is now doing most.

The Akonadi team has a few problems, or challenges if you like. Not really technical, but rather on the marketing side of stuff. I think the last couple of months have shown that we are not really communicating a lot with ‘the outside’, and in the days before this meeting, I’ve heard a lot of questions about the current state of KMail and other Akonadi apps. As with each meeting, I again hope to have time to report from the meeting and give you updates about the state of Akonadi within KDE.

I also will explain what I did manage to do last months and I hope to get started on a project related to it. More about that later.

One of the things I do want to talk about is Mailody. It is dead. With tears in my eyes I’ve deleted it a month ago. Why? Because I decided to do so. This was triggered by a flow of people wanting to use Mailody. Still puzzled? Yes? Ok. Let me explain. I created Mailody years ago, because I wanted to change the mail reading experience of my users. I never ever wanted to go after KMail users. They do the things they do their way and that’s fine. I wanted to create a mail client which had an alternative interface, alternative way of doing things, like saving the sent mail not in a separate folder, but in the same folder to preserve threading, having the ability to do a quick reply to a message, without launching a full blown composer, etc. (I can go on, but it’s pretty pointless for a dead app). I’ve accomplished some of my goals, but failed in some area’s, like seducing other developers to help me.

I’ve also failed to create a substantial userbase to keep it alive. I think I was the only user. I never made the switch to Mailody4, because it was never ready for production use. I feel the users approaching me to start using Mailody4 were doing it for the wrong reasons. Not because they liked what I created, no, they wanted it because KMail in svn trunk isn’t ‘ready’. Although everyone is Free to use their own motivation, I don’t think Mailody4 should be used for production. It will reflect badly on Mailody, Akonadi and me. Although I told people not to use it, they still insisted, ‘it is in keg, so must work somehow’. Right. The only option I had left was to kill it from svn, and so I did.

Is it really dead, dead? No. There still is a chance I will pick it up. I’ve still some unfinished business there. But it won’t be anytime before KMail trunk has been released and everyone is happy using that again. That means the whole stack should be ready for usage again (imap, mailtransport, identies, kmime, akonadi). Then I will make sure the glue between them (Mailody) is working fine again. But not now.

As said, I’ll be a frequent blogger the next days, hoping to bring you some news from what’s happening in Berlin at the Akonadi Meeting.

That’s my result of a weekend hacking in Essen. You probably don’t understand it. So I’ll add some explanation. You are looking at akonadiconsole, the debugging application of Akonadi. I’ve extended an existing resource where you can configure the filesystem path of your images. It will list all subdirectories and when clicked on a folder, it will list all images. When you click on an image, it will fetch the thumbnail of the image and show it.

That covers the most basic functionality that is needed for Digikam as far as I can tell. Of course a lot of work has to be done in Digikam to ever support Akonadi resources.

It’s just a start of a resource which shows how the Digikam and KIPI-plugin people can integrate Akonadi in their application. I know Colin Guthrie (the original author of the resource) and Luka Renko are interested in working on this more. I hope this will give them a head start, and I’m happy to assist where possible and add some code here and there.

I arrived there in the afternoon. It was nice to finally meet the people behind Digikam and the kipi-plugins. A few years ago I learned C++/Qt hacking while working on Digikam. It was a wonderful time. Since then I’ve moved on, but I’ve returned to this sprint to give a presentation about Akonadi. As Akonadi is fully type independent, it is able to be used for Digikam. It would open a nice new set of oppertunities.

For example, you could organize your images while your offline. Not only the images on your local hard disk, but also on your Facebook, Flickr or Gallery based website. As soon as you connect to the internet again, it would execute all the moves. Can make that train trip to Berlin so much more productive, right?

Another feature would be to use your image based resources to attach an image as an attachment to an email. No longer select an image from disk, but select an image from an album, as you would see them in Digikam.

I’ve not realized that at a Digikam meeting there would be a lot of people with digital camera’s around. We already decided that it looks a lot more like Digikam Next Top Model, instead of a Digikam sprint….

After that I worked to get the manual expunge code, which I explained on Friday. The DBus interfaces were added by Kevin and I adapted Mailody to use them. So that was one of the last goals I had for the meeting. In the meanwhile Thomas McGuire yelled that his rewrite of the pop3-resource was pretty much complete and passed his unit tests.

Day 3 was somehow a bit more relaxed than the other days. We briefly looked over the last bit of API we had to review. It was coming from the KMime library. KMime is a library that has been around for ages and was written with the intend to replace mimelib which is used by KMail. But that never happened.

KMime is the library that is responsible for creating the actual message after you press the send button in an email composer, but also deals with extracting the different parts from a message to display them, extracting the plain part, the HTML part, maybe some attachments, but also dealing with all kinds of encoding and decoding problems which are used to send the actual mail in.

When I started Mailody, I first tried to do that all on my own, but I soon realized that was not something I wanted to do, and Volker pointed me to the – at that moment unused – KMime library. After we removed the dust, we discovered some bad API and for KDE4, we removed most of the ugly API (like methods returning QList* (yes returning pointers to a list with pointers). Mailody then turned into a happy user with KMime. With the akonadification of KMail, it is needed that KMail also used KMime internally. Akonadi delivers the messages in that form to KMail, so it needs to. Andras Mantia has been actively working on this conversion for KMail. I’ve big respect for that work. I know it is a big task and probably not the most fun to do.

That also mean there is a new user of that stuff, and that means some API changes are wanted. We are still not completely happy with existing API, so that makes the extensions to that API somewhat less critical to get absolutely perfect. This is one of the candidates for a rewrite for KDE5. Still reading? Sorry to be a bit verbose on this stuff, just want to make sure you get a feeling why an akonadified KMail is not available yet. From one big monolithic application, it is being teared down to separate components, which will make the codebase a lot easier.

You have to have bad api’s to realize the value of good API’s.

After this mandatory part of the meeting, we quickly went over to the other mandatory part and did the Group Photo. I’ve not seen it yet, but I’m sure it will appear on the Dot or Planet soonish. There were a couple of discussions between various people. It’s fun to see that the base of Akonadi works nicely for a lot of people and we are slowly arriving at the phase were we can extend it further. Hot Topics in that area are:

• Search. You can basically split up two types of searches. The first one is in the messages within Akonadi. We will use Nepomuk for that. Especially now Virtuoso back-end for Nepomuk seems to be performing nicely, we can go ahead and implement them further. The ground work is already present; Akonadi supports creating virtual search folders, containing links to messages in other folders. Also each message passed into Akonadi is being indexed by Nepomuk. Just need to finish up that work as far as I know, like creating a user interface for creating such a search folder. The second type of searches are server side searches. Take IMAP. It allows you to search for messages directly on the server. We need to support that too.
• Filtering. Basically you have the same difference. There is a local filtering of messages, which is implemented by a summer of code student. It means that a special type of resource gets the messages before it gets added to the final destination folder. That special resource can filter it to another folder for example. This should all be finished, but it’s not completely ready yet. There is also filtering on the server side. For example IMAP filtering is (and should) be done on the server side, for example with Sieve. Volker and I briefly brainstormed about the possibility to create an application which can deal with that. Should not be too much work if we can base it on the work already done by the SOC student.

I briefly brought up Google Wave. I think Akonadi can easily provide a resource for it, although there wasn’t much interest from the participants of the meeting to work on it, so we probably will not work on it ourselves. Of course if the resource is created, you’d still need an application to edit the waves. So if anyone wants Google Wave integration into KDE, please use Akonadi as base. We can help you with the stuff, we just don’t have time to do it ourselves.

That pretty much concludes my coverage from the meeting. I know there will be a dot story outlining more stuff that has been done during the meeting, outside the little box I have lived in. It was fun to be there again.

After that we started with API-Review of the Mailtransport classes. Around 1920 everyone used to use their local sendmail application to deliver mail. That was ordered to deliver the mail as soon as the 14k4 modem has made a connection to the internet. A bit later people started to depend on the SMTP method to send mail. The application send the email to the provider which provides your internet connection and then it’s done. But now we are starting to support Microsoft Exchange Servers, we need a new way of sending mail. The feature Microsoft Exchange offers is that it also deals not only with incoming mail but also with outgoing mail. That means we needed to adapt our Mailtransport library to know which Akonadi Resources are capable of sending email.

When you compose a mail you have a selection bar, where you can indicate which provider/transport should be used for sending. That now has been extended to also show the resources that are capable of sending it. And that required a lot of changes. We combined it with nice features as sending messages after a specific date and time and moving messages after they have been send to a certain folder. To our surprise the whole morning was filled with reviewing the API and we have a pretty long list of stuff that has to change there.

After lunch we continued with the API review of IdentityManager. In this KDE cycle support has been added for images within signatures. Signatures in the context of mail messages which can hold company disclaimers for example. A much requested feature was that people (or company policy) needed to insert there company logo in the signature. A couple of KDE releases ago we added support for html based signatures and now we added the support for images. And those images need to be stored somewhere. But in a application independent way. The identities you setup are shared between KMail and Mailody and we need to prevent that breaks with the set image in the signature when you switch between those clients.

Continuing with yet more API review (really a weekend is to short for this). We continued with the API review of the widgets and classes provided by the new address book application. The classes are all in a library so other application can make use of them. One of the discussion points was about the dialog. When you for example click on a phone number and you have a voip phone, it can actually dial that phone number, the same goes for an email address. When you click it the default mail client is opened and the composer is launched. But if you show the dialog in an open composer, you want to add that address to the addressee list of that composer. Implementing that without making the API ugly needed some thinking.

In the evening Brad showed us the progress on Exchange support. He demonstrated Akonadi fetching mails, addresses and appointments from the Exchange server. Although it is in pre-alpha stage, it already shows that the resource is on the right track. From his side, there are pending requests for free-busy listing support and support for filtering. Both will be implemented sooner or later. His biggest problem, the mail sending part, is solved recently.

After that, we did an overview of the pim current situation:

• KOrganizer: Porting is going on, but we will not manage to release an Akonadi based version for KDE 4.4, so that will show up not earlier than 4.5
• KJots: A relatively new addition to the pim module is KJots. It is fully Akonadified and will be released with 4.4 as usual.
• Work on the SyncML client is improving steadily. It will be merged into KDE 4.4. It is an Akonadi Agent and I think we still need an GUI for it, but the framework will be there. I’ll blog about it more when we have had the demo.
• We have little trust OpenSync will ever deliver what we need. That means all the code we have using that, will move to playground. That includes the KitchenSync application.
• KAddressBook is Akonadified and will be released with KDE 4.4. We still need to port some PIM application to use the new KAddressBook API instead of the old one, but that’s work in progress.
• KMail: Porting is steadily continuing and progressing nicely. Though it won’t be ready for KDE 4.4.
• KNotes, it is largely unmaintained I think. The functionality is probably taken over by KJots and the notes plasmoid. So we will remove it from kdepim.
• Wizards. Volker has a prototype for configuring your mail client, based on kross script / get hot new stuff. Basically the idea is that you can select that you have a GMail account for example. All the right settings are then provided and the user only has to enter the username and the password. GHNS can provide new or updated scripts and this makes it possible for companies to deploy there own scripts to their users.

We also discussed the possibility to get rid of KUniqueApplication for kdepim applications. With the move to Akonadi the technical problems of the back-end is gone, so we can actually can rid it of it. It has some consequences for the communication in relation to standalone vs. apps in Kontact, so that requires further investigation.

Also we decided that we want to get rid of all DBus calls which control other application. Instead of launching KAddressBook and order it over DBus to show a contact. Apps should simply show a dialog which holds the contact data, much more friendly for the users. Every application that is getting ported to Akonadi should keep that in mind; provide widgets that can be reused by other application.

After we went out for dinner, we started on the API review. Due to a lot of KDAB activity in combination with the work of some SOC students, we have an enormous amount of API to review. We started with the stuff from Akonadi core and finished way after midnight.

Hm, now the speaker is speaking about a deviation, who knows where I’ll end up now. Because people are talking loudly in this cabin, I can not decipher the German announcement yet.

I’m traveling to Berlin because the KDAB building in that city once again hosts an Akonadi Sprint. I wanted to organize it this time in my beautiful office, but considering that KDAB employes lots of people that work on Akonadi, it does not make much sense to let them all come over to the NL. That would be a waste of money.

I’ve set a few goals for this meeting. First I want to see if I can adapt the imap Akonadi resource (the thing that feeds Akonadi with data from imap servers) to properly deal with expunge. The IMAP protocol deletes messages in two stages. First the message gets a flag as being deleted and then you expunge the mailbox, which permanently removes the messages which are tagged for deletion.

This is one area KMail and Mailody differ. Mailody uses the above method to. When you delete a message, it is removed from the message list, but it is still there, you can undelete it when you hit the menu item to show the deleted messages. KMail does not work that way. I’m not sure what it does currently, but in the past it moved deleted message to the trash folder and immediately expunges the mailbox. We need to discuss how we can make the resource support both systems, without biting each other. Maybe I can convince Thomas to adapt to the Mailody method ;-).

Another goal is to see if I can get the IMAP resource to work with my mail server, somewhere there is a bug in the IMAP library or my mail server which prevents this. And that is not only blocking my implementation of the goal above, but it also prevents me from using Mailody4 full time. At the end of the meeting, I would like to be able to use Mailody4. I know I’ve set that as goal for previous meetings, but who knows, maybe it will work out this time.

There will also be some api reviews for the changes between 4.3 and 4.4. And there are a lot of changes, so that will take a lot of time I guess. Also, some of the API changes done to the identity manager (the thing that manages your identities in both KMail and Mailody) are not how I would have done them. I like them to be changes and I would like to work on them to correct those problems.

The speaker is still rambling about the deviation we are going to take. I don’t think that’s of any interest for me though.

Last goal is that I would like to know the current state of the porting of the applications to Akonadi and what the time line will be. That’s of course the stuff I will be blogging about this weekend. I know you are waiting for that info ;-)