I couldn’t attend the event in San Francisco because I had to attend to some personal matters. Johannes and Donald covered for me there on very short notice (thanks guys!). For that matter, I have been mostly ‘offline’ for the last 2 months or so and will continue to be offline for atleast end of this month. So, if you sent me mail, I will get back at some point

Am really looking forward to interesting conversations about Qt5, Qt Quick and the Qt project

Here’s the solution: Compile Qt with -system-sqlite.

The problem: Qt uses it’s own sqlite3 headers under src/3rdparty by default which are completely out of date. Qt 4.7 has sqlite3 header from 2009-10-14 version 3.6.18. Almost 2 years old and current sqlite version is 3.7.7! That’s like using Qt 4.5.3 in 2011 FTS3/4 table queries fail consistently when using Qt’s own headers.

I have opened QTBUG-21040.

In WebKit lingo, “WebCore” is the thing that takes of parsing/layouting/rendering of various css/svg/html documents, providing DOM bindings etc. “JavaScriptCore” implements JavaScript support and is also referred to as SFX (Squirrel fish Extreme). JavaScriptCore can be used as a stand alone JavaScript engine and has no dependencies on WebCore. WebCore uses JavaScriptCore to support JavaScript in web pages. WebCore also contains support for NPAPI plugins (like flash). “WebKit” uses the WebCore to build a platform/toolkit specific API. For example, the Qt “WebKit” port provides QWebElement which exposes the WebCore’s DOM. By definition, WebKit is platform/toolkit/port specific. The Qt port is simply called QtWebKit.

The QtWebKit port is released periodically independent of Qt releases. These ports have the number QtWebKit 2.0, QtWebKit 2.1, QtWebKit 2.2 etc. QtWebKit 2.0 is identical to what was shipped with Qt 4.7.0. QtWebKit 2.1, intended to be mobile friendly, is not part of any shipping Qt release. QtWebKit 2.2, which will be shipped as part of the upcoming Qt 4.8.0, is yet to be released.

Now for WebKit2. The first and most important thing you should know about WebKit2 (even before you know what WebKit2 is) is that WebKit2 is NEITHER AN UPGRADE NOR A NEWER RELEASE of WebKit. It is a parallel port that can happily co-exist with “WebKit”. Let me reiterate: Stop trying to think of WebKit2 as WebKit version 2 Think of it as a completely different API from existing WebKit.

WebKit is the traditional in-process renderer. If you create 100 web pages, they all reside in one process. If one page causes a crash, it brings everything down. WebKit2 provides a system and an API to make it possible to render a page in a separate process. The process management is taken care of by WebKit2. The actual rendering of the page happens using WebCore. WebKit2, therefore, spawns out processes, renders pages in these processes and makes the end result available to the application. It provides mechanisms deliver events from the application to the rendering process. The Qt port of WebKit2 is simply called QtWebKit2. QtWebKit2 is what is used in the N9 browser.

White-space has never been more important. QtWebKit 2.x is a completely different beast from QtWebKit2. QtWebKit 2.x is plain old QtWebKit releases. QtWebKit2 is Qt’s port of WebKit2. This unfortunate naming is a result of Apple announcing WebKit2 shortly after the Qt guys deciding to call their releases QtWebKit 2.x.

WebKit2 and Chromium are similar in their goal. Chromium does not use WebKit2 and probably never will. The Chromium code was intended for the chromium browser specifically. The WebKit2 code was designed upfront to be an API. This difference in motivation resulted in different implementations. See this page for more details.

Because of the multi-process nature of QtWebKit2, many APIs that existed in QtWebKit simply don’t exist anymore. WebKit2 design lends itself to an asynchronous API compared to WebKit where most API was synchronous. For example, DOM introspection of web pages using QWebElement is not possible since the web page’s DOM resides in another process.

QtWebKit2 has a hard dependency on Qt5 and is very much a moving target like Qt5. QtWebKit will probably not work well with Qt5, we have to wait and see.

Current status: Nokia’s Qt WebKit team has decided to focus on QtWebKit2. They have decided to pass on maintainership of QtWebKit to someone else. At the time of writing, there is no publicly announced appointed maintainer to QtWebKit.

Update: Mentioned about QtWebKit 2.x releases based on Jocelyn’s comment.

At the Qt Contributors Summit, Johannes‘ showed me his Qt/Caca Lighthouse plugin. Caca is a graphics library to output text instead of pixels. So this plugin lets you run Qt programs on the console

His code needed some love, so I forked it and cleaned it up. Caca does not provide a event fd and so we have to keep polling caca for events. Since this wasn’t ideal, I moved the event handling to a separate thread and blocked for events. Unfortunately, I found that the caca library is not thread-safe and rendering and processing events in separate threads makes caca crash at randomly. So, I ended up moving the rendering to the event processing thread and having to resurrect the 20ms event timer again The cool thing though is that now Qt renders to QImage in the main ui thread and hands it off to caca. Caca opens a X connection (or similar), converts the image into text, displays a window and handles events in another thread. With some refactoring and thanks to QMetaObject::invokeMethod, the threaded and non-threaded rendering are pretty much the same and can be switched using an environment variable (THREADED_CACA=1).

Animated tiles:

If you want to hack further, code is on gitorious. (Caca doesn’t seem to deliver gpm events with ncurses, so that would be a nice fix to have)

Update: Welcome slashdot readers

Incredibly, the German consulate was kind enough to not only send me my visa a couple of hours of back but they also gave me a visa starting from 16th June! With very many things working to my favor (like just a 100 EUR rescheduling fee for the ticket), I am very happy to say I will arrive on 16th afternoon!

Qt Contributors' summit

You can use these instructions if you:

• Want to publish a Qt app for a Nokia Symbian device to the Ovi Store
• Do not use any of these in TARGET.CAPABILITY in your pro file: NetworkControl, MultimediaDD, CommDD, DiskAdmin, AllFiles, DRM and TCB
• Have Nokia Qt SDK 1.0 (with Smart Installer) or Qt SDK 1.1 (beta/pre-release) in Windows platform to build the release executable
• Have 1 euro to spare on your credit card

Pre-submission instructions:
(Start on this at least 4 days before your app’s coding/testing is expected to complete)

1. Go to publish.ovi.com and register yourself (it’ll cost you 1 euro)

2. Make a request for Nokia UIDs by sending an email to publishtoovi.support@nokia.com with the following information:

• your Ovi Publish login id
• the company name under which you registered for Ovi Publish (if any)
• IMEI numbers of devices (upto 5) you intend to use for testing the app

3. You should receive a reply from Ovi Publish with a Terms and Conditions pdf (something like this (pdf)), asking you to print it, sign (or get signed by a company representative), scan and email that back to them. Sign on and scan all pages.

4. Once you email them the scan, you should receive the following:

• Five 8-digit hexadecimal numbers: Nokia UIDs (you can publish exactly one app with each UID)
• Three files (probably in a zip archive): <something>.cer, <something>.key and <somethingelse>.sis

5. Specify the version number of your app in the .pro file.

VERSION = 1.0.0

6. Create an icon for your app (256×256 pixels, PNG format) to be shown in the phone’s applications menu. Update your .pro file with a path to the icon.

symbian:ICON = icon.png

(Warning: Using PNGs as icon is undocumented afaik, but works. We’re supposed to use Tiny SVGs but the svgs I created out of Inkscape didn’t work – maybe they weren’t “tiny” enough.)

7. Identify the Symbian versions/devices you’re targeting the app for. If you support all Symbian versions, you can skip this step. If you support only Symbian^1 and ^3 (like me), include the following in your .pro file:

symbian {
  supported_platforms = \
    "[0x1028315F],0,0,0,{\"S60ProductID\"}" \ # only Symbian^1
    "[0x20022E6D],0,0,0,{\"S60ProductID\"}"   # and Symbian^3
  # remove default platforms
  default_deployment.pkg_prerules -= pkg_platform_dependencies
  # add our platforms
  platform_deploy.pkg_prerules += supported_platforms
  DEPLOYMENT += platform_deploy
}

For including support for other Symbian versions, pickup the appropriate hex codes from here. If a user tries to install the app in a version of Symbian not specified in the list, he will see a warning during installation (something like: “This software is not supported in this platform. Continue installation / Cancel?”).

8. Include the vendor name in the .pro file. (If you specified a company name in step 2, the vendor name should be the company name. Else, the vendor name can be the name of the author of the app.)

symbian {
  vendorinfo = "%{\"InsertVendorNameHere\"}" \
                      ":\"InsertVendorNameHere\""
  vendor_deploy.pkg_prerules += vendorinfo
  DEPLOYMENT += vendor_deploy
}

Submission instructions:
(Start on this after your app’s coding/testing is complete.)

9. Create a screenshot of the app. You can get it by PrintScreen-ing on Qt Simulator running at phone resolution. But in case your app doesn’t work well on the simulator (like mine), you can temporarily hack your QMainWindow to draw itself to a file on the device.

MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent)
{
    ...
    // grab a screenshot 5 secs after startup
    QTimer::singleShot(5000, this, SLOT(getScreenshot()));
}

void MainWindow::getScreenshot()
{
    QPixmap pixmap = QPixmap::grabWidget(this);
    pixmap.save("E:/app_screenshot.png", "PNG");
    qDebug() << "Saved screenshot";
}

One way to transfer this image from your phone to your PC is to choose Connectivity > USB > Mass storage in the phone and then connect the phone to the PC by USB cable.

Ovi accepts only square images as screenshots (why that is the case when my phone doesn’t have a square screen is one of those life’s little mysteries). So open up the screenshot in your favourite image editor (Gimp or Inkscape or MS Paint or whatever) and cut out a square portion or add a square background (256×256 or bigger, PNG or JPG format).

10. You now have two options to create the .sis file for submission to Ovi Publish. Option A is dead simple, but needs you to have a very recent Qt Creator. Option B is a tad complexer, but is also nicer as it lets us try out the .sis file before submission. Make your pick.

Option A:

1. Check your Qt Creator’s version (in Windows, where you will be compiling the app for shipping). If it’s anything less than 2.2.81, install the latest bleeding-edge snapshot version. (If you compile Qt Creator yourself from source, make sure you have commit 8d781af).
2. Open your project in this uber-recent version of Qt Creator, select the Symbian Device build target. Choose Build > Publish Project and follow instructions from the wizard that opens up.
3. In the wizard, change the Application UID field to one of the Nokia UIDs sent to you in step (4) by Ovi Publish.
4. Once you hit the Commit button in the wizard, you might have to wait a while until the Open Containing Folder button gets enabled. When it does, you can click that to pickup <yourapp>_installer.sis from that folder. Copy that file to someplace safe (maybe add it to your version control). This .sis is what you’ll be uploading to Ovi Publish. You cannot try out this .sis on the device, though.
5. During step iv, Qt Creator changes the TARGET.UID3 specified in your pro file. To continue testing the app on your device (during further development), you might want to revert the UID to the earlier value.

Option B:

1. Change the TARGET.UID3 specified your .pro file to one of the Nokia UIDs sent to you in step (4) by Ovi Publish.
2. Open Qt for Symbian Command Prompt from the Windows start menu and run the following commands. Use the local path to the .cer and .key files you received in step (4).
   

   make clean
   rm *.pkg
   rm *.sis
   qmake
   make release-gcce
   make sis QT_SIS_CERTIFICATE=path/to/something.cer QT_SIS_KEY=path/to/something.key

3. Open <yourapp>_installer.pkg in a text editor. You should find something like:

   #{"appname installer"},(0xA000D7CE),1,0,0

   where appname is the name of your app and 1,0,0 is the version number you specified in step (5). The UID has to be either 0×2002CCCF or 0xA000D7CE. If it’s anything else, change the UID to 0×2002CCCF.

4. You should also find a line specifying where it should get installed on the device:
   

   "C:/path/in/dev/env/to/app/appname.sis" – "c:\private\2002CCCE\import\appname.sis"

   Change the “c:\private\2002CCCE\import\” in the right hand side to “!:\private\2002CCCE\import\”. The “!” tells the Symbian packager to allow the user of the app to choose a drive during installation.

5. Go back to the open Qt for Symbian Command Prompt window and run:
   

   make installer_sis QT_SIS_CERTIFICATE=path/to/something.cer QT_SIS_KEY=path/to/something.key

   This creates a <yourapp>_installer.sis in the same folder. Copy that file to someplace safe (maybe add it to your version control). This .sis is what you’ll be uploading to Ovi Publish.

6. Pick a test device (whose IMEI you sent to Ovi Publish in step (2)), and install on it the .sis file you received in step (4).
7. You can now try out the <yourapp>_installer.sis on the test device. Apart from being able to make a test run of the actual binary you’re going to submit, you can get a feel for what the user experience would be during installation. (For eg., I noticed that the installation fails if there’s no internet, even if the device has the required version of Qt.) You can also make sure that your (or your company’s) name and the app’s version number are showing up correctly during installation.

11. Login to publish.ovi.com, and hit Create New Content. Choose Qt Based App, answer a few legal questions, and hit Continue. Fill the form that follows, making use of the icon you created in step (6) and the screenshot you created in step (9). Hit Create.

12. Click on the Content files tab. Click the button next to Qt file and upload the <yourapp>_installer.sis file you created in step (10).

13. Enter the Nokia UID that you used in step (10).

14. Select Requesting Nokia Signing as the Symbian Signed Type. Hit Upload & Continue

15. For the platforms/devices that you identified in step (7), specify the amount of testing done. Leave the rest of the platforms as is. Click Next

16. Select the countries to publish in, hit Next, select the languages that it works with, hit Save. Then click on the Submit to QA button under the Distribution tab. You might have to wait 6-8 working days for the QA to complete.

17. Be happy.

References:
[1] Guide_to_Publishing_Qt_Applications_to_the_Ovi_Store
[2] Qt content Q&A
[3] Ovi Publisher Guide (pdf)
[4] Packaging and signing

Update:
11/May/2011: Commands order changed in step 10 B (ii) – the .pkg files created by qmake shouldn’t get deleted.

I have been reading the largely negative comments in the blogs by Aron and Daniel about the future of Qt. Sigh, the anonymous internet has made it all too simple for people to post abusive comments and suggest conspiracy theories.

I empathize with the anger; my own business relies on the Qt ecosystem. However, this decision appears to be quite logical to me. On one hand, you have Symbian. Nobody in their right mind would want a Symbian future, let alone pitch it as the competitor for Android or iOS. If you think that line requires justification, you shouldn’t be reading this article, move along Second, MeeGo. I am going to speculate here since I have not seen the actual harmattan/MeeGo UI. So, let’s say we have something like the Intel MeeGo tablet shown at MWC. Shocking, no? They are “working” on Copy&Paste, zooming is slow, opening the app causes lots of flicker, the scrolling looks laggy and the presenter is defensive. Continuing my speculation, assuming Nokia’s UX is in a similar state, what would you as a CEO do? I mean, this is the state _today_, imagine 5 months back. I would just drop MeeGo and try shopping for the software elsewhere.

I believe that’s what has happened. Nokia had to make a tough call because MeeGo doesn’t appear to be shippable anytime soon. Some people are of the opinion that choosing an alternate OS is pointless because by the time one adapts to the new OS, one can clean up MeeGo. I think one reason here could be that Elop&Co simply lost confidence in their developers after seeing the state of MeeGo. Another reason probably is that Maemo has always been a research project inside Nokia. MeeGo was announced exactly a year back at MWC 2010. I don’t think Nokia internally ever took it out of the ‘research project’ mode. Yes, they had a little flirt with it trying to make it the main platform but they quickly seem to have discovered it’s not ready (Ballmer mentions that they started talks back in Nov 2010).

So, they now have to choose between Android and WP7. I simply don’t see how Nokia can compete with the existing vendors who have a big head start with Android phones. Back here in India, everybody and their great grandmother are _shipping_ Android phones. Motorola, LG, Samsung, HTC, Videocon (yes, that washing machine company), Dell, Sony, Acer, Micromax, Olive (yes, 10, that I know of!) are already shipping Android phones across all price ranges. With the upcoming cricket world cup and IPL, I expect lot more Android phones to be advertised. A Nokia android phone will be indistinguishable in this crowd. So, personally, I would go with WP7 too as there is some hope for differentiation. With WP7, Nokia can hopefully put pressure on MS (who wants this whole thing to succeed badly for their own future) to deliver on the software.

All this talk of conspiracy theories is quite baseless. Elop cannot make unilateral decisions, that’s not how companies work. He obviously requires the support of the directors. If the share holders think this is a bad decision, they can fire the board of directors but I don’t see this happening. If my speculation about MeeGo’s current state is correct, all it takes is to show the share holders the current MeeGo prototype and that would be the end of discussion.

It’s also a good decision for Nokia for not considering Qt port on WP7. Heck, Qt/Symbian local compilation support on Linux/Mac isn’t there (for what 2 years?) with Nokia having complete control over all the software layers – the toolkit, IDE, OS. Qt on WP7 is a massive massive investment. It is probably a worthwhile undertaking that project after Nokia/WP7 is successful.

FWIW, we all have to be happy that Nokia has been open about this even before it has done anything about it. I, for one, totally appreciate their Openness. So, before you pour out your hate for Nokia, please remember that this is just business as usual. At the end of the day, they have to pay salaries. They have had to disappoint us developers for their own survival. If you are going to argue that MeeGo was truly groundbreaking and what not – please put your money on MeeGo, start your own company and ship MeeGo devices instead of pointing fingers at Nokia.

Where does this leave Qt? Qt has taken a bit big hit because of this decision. This decision means that Qt has suddenly become lot less relevant. I expect the MeeGo phone to be only as successful as the n900. I don’t expect MeeGo or Qt to die inside Nokia until WP7 phones are wildly popular. One single phone, that has been delayed over and over again and that has been sidelined into research does not give me a lot of confidence. Personally, I was hoping for this uber-awesome device for which I can build and _sell_ great applications.

My view is that Qt’s future lies outside Nokia. The Qt fanboy I am, I will do everything I can to keep Qt going. Qt has a very good future in the embedded space (settop boxes, IVI etc). Many companies I met at CES this year were committed to using Qt. For them to continue using Qt, the open governance model simply has to happen. Now. Qt has to be seen as a toolkit that has constant progress with an active community. Ports of Android, iOS can then become part of main stream. If this does not happen soon, future companies are just going to switch over to Android for their devices.

If you are like me, you will sense an opportunity here to write a qlalr based parser So, here it is – qjsonparser. The grammar is from RFC4627. It behaves very much like QJson – it returns a QVariant for the parsed JSON, uses QVariantMap for objects and QVariantList for arrays. Unlike the others, code is meant to be compiled in place (instead of a library). So, there’s just 3 files overall that you need to drop into your code (README).

If you haven’t used qlalr before:
1. qlalr generates reentrant parsers out of the box. With flex/bison in C mode, one needs to do all sorts of stuff to create a reentrant parser. (QJson uses bison in c++ mode, so it’s reentrant)
2. Complete control over shift/reduce. AFAIK, parsing incrementally using bison isn’t possible easily. With qlalr, you have to write the equivalent of yyparse() yourself and this gives a great deal of control over the shift/reduce steps.
3. It’s completely undocumented. With bison, you don’t really need to understand how LALR parsers work. qlalr, on the other hand, will make you pull out your compiler book. Most of the yyparse() equivalent code that I mentioned can be copy/pasted. But if you are averse to copy/pasting seemingly obfuscated/random code, you absolutely have to understand how LALR parsers work before touching qlalr. Which is why you shouldn’t believe those posts which say that you will be at home with qlalr if you have used bison before. And oh, I don’t intend to spoil the fun of using it by documenting it .
4. qlalr is used by QXmlStreamReader and the old QtScript code. I think it’s the best parser generator for Qt projects.

PS: Currently, I use a hand made lexer, but you can use lexgen which is a Qt friendly scanner. I would have used it but it would add to the number of files. lexgen is used in the Qt’s CSS scanner. The CSS parser, however, is hand made since qlalr didn’t exist then.

UPDATE: This project has moved to http://gitorious.org/qjsonparser/qjsonparser