I noticed that the video and slides for my Qt Dev Days 2011 talk in Munich are now online. Here’s the video and here are the slides.

I couldn’t attend the event in San Francisco because I had to attend to some personal matters. Johannes and Donald covered for me there on very short notice (thanks guys!). For that matter, I have been mostly ‘offline’ for the last 2 months or so and will continue to be offline for atleast end of this month. So, if you sent me mail, I will get back at some point

I am just about to catch my flight to Munich to attend Qt Developer Days 2011 where I will be giving a talk on Qt Quick Best Practices. I have been working with QML exclusively for over a year now and this talk is in essence a summary of all the things I have learnt about it. This will also be my 6th developer days as attendee and third time as a speaker.

Am really looking forward to interesting conversations about Qt5, Qt Quick and the Qt project

If you have worked with QtSql, you might have hit the dreaded “Parameter count mismatch” for your perfectly valid SQL query. The problem is excruciatingly hard to debug because the query itself works perfectly fine with the sqlite3 tool.

Here’s the solution: Compile Qt with -system-sqlite.

The problem: Qt uses it’s own sqlite3 headers under src/3rdparty by default which are completely out of date. Qt 4.7 has sqlite3 header from 2009-10-14 version 3.6.18. Almost 2 years old and current sqlite version is 3.7.7! That’s like using Qt 4.5.3 in 2011 FTS3/4 table queries fail consistently when using Qt’s own headers.

I have opened QTBUG-21040.

Ever heard of WebKit2 and wondering what it means from a Qt perspective? Here’s an attempt to explain QtWebKit and QtWebKit2 in simple terms. I make no attempt to be completely technically correct, it’s meant to be able to explain terminology to the WebKit uninitiated.

In WebKit lingo, “WebCore” is the thing that takes of parsing/layouting/rendering of various css/svg/html documents, providing DOM bindings etc. “JavaScriptCore” implements JavaScript support and is also referred to as SFX (Squirrel fish Extreme). JavaScriptCore can be used as a stand alone JavaScript engine and has no dependencies on WebCore. WebCore uses JavaScriptCore to support JavaScript in web pages. WebCore also contains support for NPAPI plugins (like flash). “WebKit” uses the WebCore to build a platform/toolkit specific API. For example, the Qt “WebKit” port provides QWebElement which exposes the WebCore’s DOM. By definition, WebKit is platform/toolkit/port specific. The Qt port is simply called QtWebKit.

The QtWebKit port is released periodically independent of Qt releases. These ports have the number QtWebKit 2.0, QtWebKit 2.1, QtWebKit 2.2 etc. QtWebKit 2.0 is identical to what was shipped with Qt 4.7.0. QtWebKit 2.1, intended to be mobile friendly, is not part of any shipping Qt release. QtWebKit 2.2, which will be shipped as part of the upcoming Qt 4.8.0, is yet to be released.

Now for WebKit2. The first and most important thing you should know about WebKit2 (even before you know what WebKit2 is) is that WebKit2 is NEITHER AN UPGRADE NOR A NEWER RELEASE of WebKit. It is a parallel port that can happily co-exist with “WebKit”. Let me reiterate: Stop trying to think of WebKit2 as WebKit version 2 Think of it as a completely different API from existing WebKit.

WebKit is the traditional in-process renderer. If you create 100 web pages, they all reside in one process. If one page causes a crash, it brings everything down. WebKit2 provides a system and an API to make it possible to render a page in a separate process. The process management is taken care of by WebKit2. The actual rendering of the page happens using WebCore. WebKit2, therefore, spawns out processes, renders pages in these processes and makes the end result available to the application. It provides mechanisms deliver events from the application to the rendering process. The Qt port of WebKit2 is simply called QtWebKit2. QtWebKit2 is what is used in the N9 browser.

White-space has never been more important. QtWebKit 2.x is a completely different beast from QtWebKit2. QtWebKit 2.x is plain old QtWebKit releases. QtWebKit2 is Qt’s port of WebKit2. This unfortunate naming is a result of Apple announcing WebKit2 shortly after the Qt guys deciding to call their releases QtWebKit 2.x.

WebKit2 and Chromium are similar in their goal. Chromium does not use WebKit2 and probably never will. The Chromium code was intended for the chromium browser specifically. The WebKit2 code was designed upfront to be an API. This difference in motivation resulted in different implementations. See this page for more details.

Because of the multi-process nature of QtWebKit2, many APIs that existed in QtWebKit simply don’t exist anymore. WebKit2 design lends itself to an asynchronous API compared to WebKit where most API was synchronous. For example, DOM introspection of web pages using QWebElement is not possible since the web page’s DOM resides in another process.

QtWebKit2 has a hard dependency on Qt5 and is very much a moving target like Qt5. QtWebKit will probably not work well with Qt5, we have to wait and see.

Current status: Nokia’s Qt WebKit team has decided to focus on QtWebKit2. They have decided to pass on maintainership of QtWebKit to someone else. At the time of writing, there is no publicly announced appointed maintainer to QtWebKit.

Update: Mentioned about QtWebKit 2.x releases based on Jocelyn’s comment.

At the Qt Contributors Summit, Johannes‘ showed me his Qt/Caca Lighthouse plugin. Caca is a graphics library to output text instead of pixels. So this plugin lets you run Qt programs on the console

His code needed some love, so I forked it and cleaned it up. Caca does not provide a event fd and so we have to keep polling caca for events. Since this wasn’t ideal, I moved the event handling to a separate thread and blocked for events. Unfortunately, I found that the caca library is not thread-safe and rendering and processing events in separate threads makes caca crash at randomly. So, I ended up moving the rendering to the event processing thread and having to resurrect the 20ms event timer again The cool thing though is that now Qt renders to QImage in the main ui thread and hands it off to caca. Caca opens a X connection (or similar), converts the image into text, displays a window and handles events in another thread. With some refactoring and thanks to QMetaObject::invokeMethod, the threaded and non-threaded rendering are pretty much the same and can be switched using an environment variable (THREADED_CACA=1).

Animated tiles:

If you want to hack further, code is on gitorious. (Caca doesn’t seem to deliver gpm events with ncurses, so that would be a nice fix to have)

Update: Welcome slashdot readers

Disclaimer: I am an ex-troll and my company does a lot of Qt business with Nokia

I have been reading the largely negative comments in the blogs by Aron and Daniel about the future of Qt. Sigh, the anonymous internet has made it all too simple for people to post abusive comments and suggest conspiracy theories.

I empathize with the anger; my own business relies on the Qt ecosystem. However, this decision appears to be quite logical to me. On one hand, you have Symbian. Nobody in their right mind would want a Symbian future, let alone pitch it as the competitor for Android or iOS. If you think that line requires justification, you shouldn’t be reading this article, move along Second, MeeGo. I am going to speculate here since I have not seen the actual harmattan/MeeGo UI. So, let’s say we have something like the Intel MeeGo tablet shown at MWC. Shocking, no? They are “working” on Copy&Paste, zooming is slow, opening the app causes lots of flicker, the scrolling looks laggy and the presenter is defensive. Continuing my speculation, assuming Nokia’s UX is in a similar state, what would you as a CEO do? I mean, this is the state _today_, imagine 5 months back. I would just drop MeeGo and try shopping for the software elsewhere.

I believe that’s what has happened. Nokia had to make a tough call because MeeGo doesn’t appear to be shippable anytime soon. Some people are of the opinion that choosing an alternate OS is pointless because by the time one adapts to the new OS, one can clean up MeeGo. I think one reason here could be that Elop&Co simply lost confidence in their developers after seeing the state of MeeGo. Another reason probably is that Maemo has always been a research project inside Nokia. MeeGo was announced exactly a year back at MWC 2010. I don’t think Nokia internally ever took it out of the ‘research project’ mode. Yes, they had a little flirt with it trying to make it the main platform but they quickly seem to have discovered it’s not ready (Ballmer mentions that they started talks back in Nov 2010).

So, they now have to choose between Android and WP7. I simply don’t see how Nokia can compete with the existing vendors who have a big head start with Android phones. Back here in India, everybody and their great grandmother are _shipping_ Android phones. Motorola, LG, Samsung, HTC, Videocon (yes, that washing machine company), Dell, Sony, Acer, Micromax, Olive (yes, 10, that I know of!) are already shipping Android phones across all price ranges. With the upcoming cricket world cup and IPL, I expect lot more Android phones to be advertised. A Nokia android phone will be indistinguishable in this crowd. So, personally, I would go with WP7 too as there is some hope for differentiation. With WP7, Nokia can hopefully put pressure on MS (who wants this whole thing to succeed badly for their own future) to deliver on the software.

All this talk of conspiracy theories is quite baseless. Elop cannot make unilateral decisions, that’s not how companies work. He obviously requires the support of the directors. If the share holders think this is a bad decision, they can fire the board of directors but I don’t see this happening. If my speculation about MeeGo’s current state is correct, all it takes is to show the share holders the current MeeGo prototype and that would be the end of discussion.

It’s also a good decision for Nokia for not considering Qt port on WP7. Heck, Qt/Symbian local compilation support on Linux/Mac isn’t there (for what 2 years?) with Nokia having complete control over all the software layers – the toolkit, IDE, OS. Qt on WP7 is a massive massive investment. It is probably a worthwhile undertaking that project after Nokia/WP7 is successful.

FWIW, we all have to be happy that Nokia has been open about this even before it has done anything about it. I, for one, totally appreciate their Openness. So, before you pour out your hate for Nokia, please remember that this is just business as usual. At the end of the day, they have to pay salaries. They have had to disappoint us developers for their own survival. If you are going to argue that MeeGo was truly groundbreaking and what not – please put your money on MeeGo, start your own company and ship MeeGo devices instead of pointing fingers at Nokia.

Where does this leave Qt? Qt has taken a bit big hit because of this decision. This decision means that Qt has suddenly become lot less relevant. I expect the MeeGo phone to be only as successful as the n900. I don’t expect MeeGo or Qt to die inside Nokia until WP7 phones are wildly popular. One single phone, that has been delayed over and over again and that has been sidelined into research does not give me a lot of confidence. Personally, I was hoping for this uber-awesome device for which I can build and _sell_ great applications.

My view is that Qt’s future lies outside Nokia. The Qt fanboy I am, I will do everything I can to keep Qt going. Qt has a very good future in the embedded space (settop boxes, IVI etc). Many companies I met at CES this year were committed to using Qt. For them to continue using Qt, the open governance model simply has to happen. Now. Qt has to be seen as a toolkit that has constant progress with an active community. Ports of Android, iOS can then become part of main stream. If this does not happen soon, future companies are just going to switch over to Android for their devices.

To my knowledge, there are 3 Qt based JSON parsers out there – QJson, JsonQt and this. QJson uses bison for parsing and JsonQt is hand-written. I have used QJson before and it works perfectly fine.

If you are like me, you will sense an opportunity here to write a qlalr based parser So, here it is – qjsonparser. The grammar is from RFC4627. It behaves very much like QJson – it returns a QVariant for the parsed JSON, uses QVariantMap for objects and QVariantList for arrays. Unlike the others, code is meant to be compiled in place (instead of a library). So, there’s just 3 files overall that you need to drop into your code (README).

If you haven’t used qlalr before:
1. qlalr generates reentrant parsers out of the box. With flex/bison in C mode, one needs to do all sorts of stuff to create a reentrant parser. (QJson uses bison in c++ mode, so it’s reentrant)
2. Complete control over shift/reduce. AFAIK, parsing incrementally using bison isn’t possible easily. With qlalr, you have to write the equivalent of yyparse() yourself and this gives a great deal of control over the shift/reduce steps.
3. It’s completely undocumented. With bison, you don’t really need to understand how LALR parsers work. qlalr, on the other hand, will make you pull out your compiler book. Most of the yyparse() equivalent code that I mentioned can be copy/pasted. But if you are averse to copy/pasting seemingly obfuscated/random code, you absolutely have to understand how LALR parsers work before touching qlalr. Which is why you shouldn’t believe those posts which say that you will be at home with qlalr if you have used bison before. And oh, I don’t intend to spoil the fun of using it by documenting it .
4. qlalr is used by QXmlStreamReader and the old QtScript code. I think it’s the best parser generator for Qt projects.

PS: Currently, I use a hand made lexer, but you can use lexgen which is a Qt friendly scanner. I would have used it but it would add to the number of files. lexgen is used in the Qt’s CSS scanner. The CSS parser, however, is hand made since qlalr didn’t exist then.

UPDATE: This project has moved to http://gitorious.org/qjsonparser/qjsonparser

I have been working on getting Flash working well in Qt/Webkit on the N900. The hard work is done, you can find all patches here.

1. XEmbed support for the Flash on the N900 does not work very well. So, we just force windowless mode on all Flash.
2. Transparent flash is not supported and behaves the same as in opaque mode.
3. Opaque mode is implemented using the local rendering extension.

There’s some optimization work to be done which I plan to do next week.

Maemo5 PR1.2 for the N900 was released a couple of weeks back. You can see the features and bug list here – the most interesting for me being Qt 4.6.2 is now shipped as part of Maemo and Skype supports video calls and multiuser chat.

There are many tutorials on how to install but they drop terms like NAND, eMMC, FIASCO which made little or no sense to me. What better way to make the web a better place ™ than by writing yet another tutorial So, here’s how you go about manually installing PR 1.2.

0. I like to understand what I am doing, so here’s what I found The N900 has a 256 MB NAND Flash which contains the rootfs (of type ubifs). The boot loader and kernel goes into this NAND flash. The N900 also has a 32GB built-in eMMC. It contains a 2GB /home partition (ext3), 27GB /home/user/MyDocs (vfat) and swap. Installed apps go into /home/opt and thus the user is limited to 2GB of apps. /home/user/MyDocs is what you see when you usb mount your N900 on your PC. The N900 also has a external micro SDHC slot (remove the cover to see). This ‘external’ card remains unaffected by what we are about to do (it’s at /media/mmc1).

What we are going to do now is to reflash the NAND flash with the new PR 1.2 OS. We will then flash the eMMC. It’s optional to flash the eMMC. It’s only required when some new program that gets installed in the NAND misbehaves with old configuration files in the eMMC. I went ahead and flashed the eMMC too (for the heck of it). Important : The N900 will move content from rootfs to the eMMC’s /home/opt on it’s first boot. This means that you have to power on the N900 only after you are done flashing both of them.

Lastly, the term FIASCO stands for “Fractal Image And Sequence Codec” which I found after some very intense googling. It is just a compressions method (like gzip) used for the images.

Charge your n900′s battery so it will atleast last for 30 mins before you begin. Actually, it’s only a 5 min process but why take the risk.

1. If you plan to flash the eMMC, first back things up. There is no complete backup/restore utility on the N900. There is a ‘backup’ utility under applications. Use this to create a backup of your contacts/calendar, bookmarks and some apps. Very important : this backup resides in eMMC itself (under /home/user/MyDocs/Backups). This means that unless you copy this backup folder to your PC, your backup is lost. I don’t have any apps installed; if you do now is a good time to make note of the custom repos you added and list of custom apps. I just usb-mounted and copied over all my pictures  and the Backups folder.

2. Get the flasher from http://tablets-dev.nokia.com/maemo-dev-env-downloads.php (maemo_flasher-3.5_2.5.2.2.tar.gz). Works just fine on 64-bit linux too.

3. Get the images to be flashed from http://tablets-dev.nokia.com/nokia_N900.php. RX-51_2009SE_10.2010.19-1_PR_COMBINED_MR0_ARM.bin (for the rootfs on NAND) and RX-51_2009SE_10.2010.13-2.VANILLA_PR_EMMC_MR0_ARM.bin (for the eMMC). Important – I used the Global release instead of India since some reports suggest that Indian release does not have Skype Video.

The IMEI of your phone is under Settings->About (or power off and look under the battery).

4. Switch off n900. Press and hold the u key and connect usb to PC. You will see a usb sign on the top-right. Now run flasher,

flasher-3.5 -F RX-51_2009SE_10.2010.19-1_PR_COMBINED_MR0_ARM.bin -f

<success message>

flasher-3.5 -F RX-51_2009SE_10.2010.13-2.VANILLA_PR_EMMC_MR0_ARM.bin -f -R

<more success>

That’s it!

Reboot the N900. Dialing *#0000# will tell you the firmware version. If you go to a skype contact, you will see ‘video call’ What about Qt? It’s all under /usr/lib. Here are the sizes

libQtCore.so.4.6.2 (2.8M), libQtDBus.so.4.6.2 (611.5k), libQtMaemo5.so.4.6.2 (81.1k), libQtMultimedia.so.4.6.2 (176.8k), libQtNetwork.so.4.6.2 (1.3M), libQtOpenGL.so.4.6.2 (468.5k), libQtSql.so.4.6.2 (259.9k), libQtSvg.so.4.6.2 (396.1k), libQtXml.so.4.6.2 (325.6k), libphonon.so.4.3.1 (336.2k)

I guess they were concerned about using too much space in rootfs So, the QtGui, WebKit and XmlPatterns are actually in /home/opt/lib (probably got moved from the rootfs on first boot)

libQtGui.so.4.6.2 (10.7M), libQtWebKit.so.4.6.2 (17.5M), libQtXmlPatterns.so.4.6.2 (5.5M)

I managed to get my hands on the N900 after a long long wait. But the path to getting the device was long and tragic. Here’s the story.

Back at foss.in (Dec 1st week 2009), I heard of a contest held by the Nokia maemo-bangalore team.  The sexy 3d pic on the blog got me all excited. So, at foss.in, including all my customer commitments, I worked 3 days/nights at a stretch to write a youtube video browser, cilantro (anagram of the irc nick of a good friend and lead of qt/webkit team to whom the app is dedicated to ).  In all honestly, I put myself under some pressure to participate in the contest – I was already overloaded with work but I thought that if I can get my hands on the device I can improve QtWebKit Flash performance  since I was working on that at that point. So, you can understand my excitement when awaiting for the results.

Except it was not announced (we were told informally that it will be announced at end of foss.in). We were told it was postponed to be on the 8 Dec 2009. Minor disappointment. But, fine. That didn’t happen either. After a few tweets and poking a nokia friend, the results were announced on 11 Dec 2009. No mail to winners, no notification on when we will get the devices. Just the blog.  I mean no “thanks for participating, we will announce winners on xx date” even. Oh well. I forgive, only because I won. But my joy wasn’t long lived.

To my shock, the winners had folks who had not written their app for the contest. Atleast following the links on the blog suggests that the apps were published way back. Surely, one expects code to be written around the time of the contest? I must be wrong, so to make a more judicious decision, I went around looking for the source. Except, with all my googling powers, I cannot find the source for any (except one which had a commit long before contest announcement). Still convinced that my assumptions were just wrong, I left a comment on the blog asking for the code. No response. I sent a mail asking for the code. No response. So, I poked the Nokia friend yet again. This time I got a response saying they *cannot* publish the code and they will send a mail to just let people upload code to maemo garage if they care. HUH!? So, here we have a contest for foss.in where the winners have been adjudged with no published code. Sad, really sad.

On dec 18, I was told it *might* take a month for the device to arrive (and only after I asked them). 1 month!? I mean I wrote this app to get my hands on it immediately. Maybe I am expecting too much. I wait. On Jan 8, I asked them what the status was. No response (see the trend?). So I poked Nokia friend again and I got a response on Jan 14, saying they will intimate me by phone. As, you can guess, nothing was heard from them ever again. On Feb 3, I mailed them again. No response as usual. This time I didn’t ask my Nokia friend. I had given up on the people running the contest and I was going to take this outrageous behavior public.

Amazingly on Feb 16 2009, I got contacted by the maemo team for the first time ever (i.e first time initiated by them). And I did end up getting the N900 but the whole thing was such a sorry experience that I will never enter a contest that involves the maemo-bangalore team. I am not even sure if I deserve the award and if the contest was a FOSS contest (we can never know until we see the source). I had given them the link to my source but they haven’t bothered updating the blog post.

@maemo-bangalore – you need to learn to communicate and in the least, atleast respond when asked questions. You should be ashamed of treating the community and contest winners this way. If you don’t have resources to do a good job, just don’t run contests. What you have done is a mockery of FOSS. I am not actually upset with the late delivery of the device but the way you ran the contest.

If I were Nokia, I would fire the incompetent person who ran the contest.