June 11, 2013

Getting into the Harmattan Share menu: The missing guide

Meego-Harmattan has this Share menu that’s a deceptively simple idea that lets apps talk to a remote service, or to another app. For example, you can ‘Share’ an image from the Gallery app to a remote service (say Flickr), or to another app in the phone (say Mail).

There is not much documentation about either ways of getting into the Share menu. The little official stuff that I could find are these:

  1. Share to a remote service using the web-upload plugins: http://gitorious.org/meego-sharing-framework/pages/Tutorials
  2. Share to anything else using method plugins: http://gitorious.org/meego-sharing-framework/share-ui/trees/master/method-plugins

The method plugins infrastructure seems to be the lower-level API of the two, offering only the integration with the share menu. The web-upload plugins infrastructure is more specialized for the specific case of uploading to remote services, and is most probably making use of the method plugins infrastructure internally.

This article is primarily concerned with only the method plugins infrastructure. It’s based on what I learnt looking at the reference code mentioned below under ‘References’, and based on my experience writing the Share-menu integration for Notekeeper.

What happens when you hit ‘Share’?

First, a quick look at how this sharing thing is architected.

When you hit ‘Share’, from say the Gallery, the Gallery app runs /usr/bin/share-ui as a separate process (probably done through the ShareUiInterface API) and tells it what needs to be shared (it could be an image, a PDF, a bunch of files, or even a web-URL).

This share-ui app first shows a sharing screen with a thumbnail of the image (or approriate icon if it’s not an image). Then it dynamically loads all Qt plugins installed at /usr/lib/share-ui/plugins. Each of these share-ui plugins can offer multiple items to be added to the Share menu – each item being a ‘sharing method’. The share-ui app would ask each sharing method of each plugin whether it wants to figure in the share menu for this particular image (or file/files/URL/whatever). Only those methods that say “Yes” at this point will be shown in the Share menu.

When the user actually selects something in the share menu (like Mail, or Flickr), the share-ui tells the appropriate sharing method that something has been shared to it. It’s up to the method to do whatever it wants at that point. Once the method signals that it’s done, the share-ui app quits.

<guesswork>

Interestingly, the web-upload plugins framework itself seems to be written using this infrastructure. There’s a libwebupload.so under /usr/lib/share-ui/plugins, which, I think, creates ‘sharing method’ objects based on the xml files under /usr/share/accounts/services, and when the user selects the method, defers the actual uploading to the appropriate plugin from /usr/lib/webupload/plugins.

</guesswork>

Writing a share-ui plugin

To create a share-ui plugin, we need to create a .lib file that can be placed at /usr/lib/share-ui/plugins, and can be loaded as a Qt plugin, and can offer the interface that the share-ui app expects to see.

We’re now going to try and write a simple share-ui plugin that copies the path to the image (or file or URL) shared to it to the system clipboard. It’s not going to be of much use by itself, but it would serve as a good example to illustrate how to write a share-ui plugin.

Let’s start by writing a .pro file:

# copy-share-ui-plugin.pro

TEMPLATE = lib
TARGET = copy-share-ui-plugin

CONFIG += share-ui-plugin

HEADERS = copyshareuiplugin.h   copyshareuimethod.h
SOURCES = copyshareuiplugin.cpp copyshareuimethod.cpp

We need the CONFIG += share-ui-plugin because we’re building this on top of the share-ui framework.

The CopyShareUiPlugin class should derive from ShareUI::PluginBase and have a public method called methods() which would return the ‘sharing methods’ that the plugin offers. Our plugin would offer just a single sharing method, the CopyShareUiMethod.

// copyshareuiplugin.h

#include <ShareUI/PluginBase>
#include <ShareUI/MethodBase>

class CopyShareUIPlugin : public ShareUI::PluginBase
{
    Q_OBJECT
public:
    explicit CopyShareUIPlugin(QObject *parent = 0)
        : ShareUI::PluginBase(parent) { }
    virtual ~CopyShareUIPlugin() { }

    QList<ShareUI::MethodBase*> methods(QObject *parent = 0);
};


// copyshareuiplugin.cpp

#include "copyshareuiplugin.h"
#include "copyshareuimethod.h"
#include <QtPlugin>

QList<ShareUI::MethodBase*> CopyShareUIPlugin::methods(
                                            QObject *parent)
{
    QList<ShareUI::MethodBase*> methods;
    methods << new CopyShareUIMethod(parent);
    return methods;
}

// Make this class available as a Qt Plugin
Q_EXPORT_PLUGIN2(copy-share-ui-plugin, CopyShareUIPlugin)

Now, onto the sharing method. The CopyShareUIMethod should derive from ShareUI::MethodBase and should implement four virtual methods, and two slots.

// copyshareuimethod.h

#include <ShareUI/MethodBase>

class CopyShareUIMethod : public ShareUI::MethodBase
{
    Q_OBJECT
public:
    explicit CopyShareUIMethod(QObject *parent = 0);
    virtual ~CopyShareUIMethod();

    virtual QString id();
    virtual QString title();
    virtual QString subtitle();
    virtual QString icon();

public slots:
    void currentItems(const ShareUI::ItemContainer *items);
    void selected(const ShareUI::ItemContainer *items);
};

The id() method should return a unique id identifying the sharing method. Ideally, this should be in the reverse domain name notation, like “com.meego.email” for the Email sharing method.

The return values for title(), subtitle() and icon() decide how the item appears in the Sharing menu.

// copyshareuimethod.cpp

#include "copyshareuimethod.h"

CopyShareUIMethod::CopyShareUIMethod(QObject *parent)
  : ShareUI::MethodBase(parent) {
}

CopyShareUIMethod::~CopyShareUIMethod()
{
}

QString CopyShareUIMethod::id()
{
    return "com.example.copy";
}

QString CopyShareUIMethod::title()
{
    return "Copy path";
}

QString CopyShareUIMethod::subtitle()
{
    return "Copy path to clipboard";
}

QString CopyShareUIMethod::icon()
{
    return "icon-m-content-document";
}

The share-ui app would call the currentItems() slot to check whether we want to figure in the Share menu. The argument to that method is a list of items being shared. If we want this sharing method to appear in the Share menu for these items, the currentItems() method should emit visible(true), else should emit visible(false). For now, we always emit true.

// copyshareuimethod.cpp

void CopyShareUIMethod::currentItems(
                          const ShareUI::ItemContainer * items)
{
    Q_UNUSED(items);
    emit visible(true); // declared in ShareUI::MethodBase
}

The share-ui app would call the selected() slot to tell us that the user tapped on our item in the share menu. We would be implementing the copy-to-clipboard part here. If we’re successful, we’re supposed to emit done(), if not, we can emit selectedFailed(“Error message”). The done() signal indicates that the sharing is complete, so on seeing that, the share-ui app would terminate.

// copyshareuimethod.cpp

void CopyShareUIMethod::selected(const ShareUI::ItemContainer * items)
{
    Q_UNUSED(items);
    // TODO: Copy path to clipboard
    emit done(); // declared in ShareUI::MethodBase
}

With just this, we should already be able to see our ‘Copy path’ item in the Share menu. To build this, open the .pro file in Qt Creator. Take care to include the Harmattan build target. It’s better you refuse when Qt Creator offers to add debian packaging files to this project – it sometimes messes things up.

If you build (‘Build’, not ‘Run’) the project for Harmattan in release mode, you should get a compiled libcopy-share-ui-plugin.so.1.0.0 file.

Copy that to /usr/lib/share-ui/plugins/libcopy-share-ui-plugin.so in your N9 or N950 (I’m assuming you have enabled developer mode and know how to get to a root prompt). If you so prefer, you can automate this copy-file-to-phone part by adding the following to your .pro file:

target.path = /usr/lib/share-ui/plugins
INSTALLS += target

With that, if you have your N9 / N950 configured for development with Qt Creator, you can say ‘Deploy project’ and the library file should get installed at the right path in the phone.

Once the .so file is installed in the phone, open the Share menu from any app in the phone (like Gallery or Documents). You should see ‘Copy path’ in the Share menu. Tapping it should make the Share menu disappear. So far, so good.

copy-share-menu-1-basic

Accessing what is being shared

Next, let’s do the actual write-to-clipboard part.

For that, we’ll need to access the list of items being shared. We should be able to get that from the ShareUI::ItemContainer pointer being passed to us. The best way to figure out how to access
the items is to refer to the header files under ShareUI/.

// copyshareuimethod.cpp

#include <ShareUI/ItemContainer>
#include <ShareUI/FileItem>
#include <ShareUI/DataUriItem>
#include <MDataUri>
#include <QClipboard>
#include <QApplication>

void CopyShareUIMethod::selected(const ShareUI::ItemContainer * items)
{
    QString path("");
    if (items->count() == 1) { // can handle only one shared item
        ShareUI::SharedItem item = items->getItem(0);
        ShareUI::FileItem *fileItem = 
                            ShareUI::FileItem::toFileItem(item);
        ShareUI::DataUriItem *dataUriItem = 
                            ShareUI::DataUriItem::toDataUriItem(item);
        if (fileItem) { // it's a file
            path = fileItem->filePath();
        } else if (dataUriItem) { // it's a URL
            path = dataUriItem->dataUri().textData();
        }
    }
    QApplication::clipboard()->setText(path);
    // Give some time for the setText() call to talk to the X Server
    // and then emit done() to quit the share-ui app
    QTimer::singleShot(500, this, SIGNAL(done()));
}

Now, the plugin actually does what we set out to do.

Hiding ourselves from the Share menu

Since the plugin works only if there’s just one item to share, it would be nice to not even show up when multiple items are being shared. We can do that by appropriately emitting visible() in the currentItems() method.

void CopyShareUIMethod::currentItems(
                          const ShareUI::ItemContainer * items)
{
    // This sharing method is valid only if
    // there's just one item to share
    emit visible(items->count() == 1);
}

Custom icon

So far, we’re using a standard Harmattan icon. What if we want a custom icon to be shown in the Share menu instead? We need a 64×64 png like the Harmattan content icons. Let’s pick a 64×64 Qt logo image and copy it to our project folder.

We should have it installed at some place in the phone. So, we should tell Qt Creator to include this icon in the package and deployed when we say ‘Deploy’.

# copy-share-ui-plugin.pro

icon.path = /usr/share/icons/hicolor/64x64/apps/
icon.files = copy-shareui-qtlogo64.png
INSTALLS += icon

And then, should return this as the icon in icon().

// copyshareuimethod.cpp

QString CopyShareUIMethod::icon()
{
    return "/usr/share/icons/hicolor/64x64/apps/copy-shareui-qtlogo64.png";
}

Surprise, surprise. It doesn’t work. Only a red square shows up as the icon for the ‘Copy path’ item in the share menu.

copy-share-menu-2-bad-icon

To understand why this happens, we need to know how the share-ui application loads icons. The share-ui app, like any Meego Touch app, loads graphics resources through the theme server. The theme server however doesn’t have our icon loaded, which is why it’s returning a red square.

We could place our image file in the theme directory (like /usr/share/themes/blanco/meegotouch/icons), but even with that, it would need the theme server to be restarted for it to be shown correctly.

Instead, a good way to solve this problem is to explicitly tell the theme server to index the folder where we placed our custom icon in the phone. A call to MTheme::addPixmapDirectory() will do that for us. After that, we only need to return the basename of the file.

// copyshareuimethod.cpp

#include <MTheme>

QString CopyShareUIMethod::icon()
{
    if (!m_isPixmapDirAdded) { // need to add it just once
        QString shareIconPath = "/usr/share/icons/hicolor/64x64/apps";
        MTheme::addPixmapDirectory(shareIconPath);
        m_isPixmapDirAdded = true;
    }
    return "copy-shareui-qtlogo64";
}

We use a member variable called m_isPixmapDirAdded to make sure we make the addPixmapDirectory() call only once per process.

To make it compile, we need to add meegotouch to the config:

# copy-share-ui-plugin.pro

CONFIG += meegotouch

That should do the trick. Build and Deploy, and you should be able to see the custom icon in the share menu.

copy-share-menu-3-custom-icon

Working code

The complete working code discussed in this post is available at: https://github.com/roop/copy-share-ui-plugin

Each step is committed separately, so you can browse the commits in line with the steps in this guide.

What’s next

Next week, I’ll talk specifically about integrating your application with the Share menu. Like, for example, how Notekeeper for N9 integrates with the Share menu.

References:

  1. Meego Sharing Framework examples: https://gitorious.org/meego-sharing-framework/share-ui/trees/master/method-plugins
  2. CmdShare by Tuomas Kulve https://projects.developer.nokia.com/cmdshare/browser

December 15, 2012

My XSRF exploit of build.phonegap.com

PhoneGap Build is a nice service from Adobe. You upload your PhoneGap application and it builds Android, iOS packages on the cloud. PhoneGap Build also provides an API for it’s service. I used the API to create pgbuild to automate the upload of the PhoneGap application source and download the built packages.

The exploit I found has to do with the JSONP API (scroll to the end of the api page). Using the exploit, one can read a logged in user’s registered apps, initiate a rebuild of the apps etc.

The exploit itself is fairly trivial, if you understand how browsers work. If you just want to know the exploit scroll to the very end. The goal of this post is to provide a high level overview of how browser security and xsrf-style exploits work. Let’s start with Same Origin Policy.

Same Origin Policy

Browsers are very anal (for good reasons) about what resources a web page can access. A web page served from foo.com can only access resources from foo.com. This principle is called Same-Origin Policy. The policy is applied to JavaScript, HTML, file:// protocol and each has it’s exceptions. For HTML, the img tag, for example, can have url’s that point to an arbitrary website. The script tag and style tag can have the src attribute set to content from arbitrary websites.

Cookies

Whenever the browser makes a request, it sends across the cookies that have been set for that domain. This is true even for the cross-domain img and script tags. For example, let’s assume that you were logged into your bank. Without logging out, if you navigate to evil.com which has img src=”http://bank.com” somewhere inside it, your browser will make a request to bank.com with the cookies of bank.com set. The bank might provide personal information about you (since cookies have been provided) and dutifully serve the web page. It’s as if you had typed bank.com in the url bar of the browser (smart browsers will set the HTTP accept header when requesting an image and smart servers will check that header, but that is another story). Thankfully, the response from bank.com is most likely some HTML and the browser will fail to load it as an image. Your bank content is thus not introspectable by evil.com. The same goes for the script tag. Since HTML cannot be executed as a script, the browser just ignores the response.

Same-Origin policy is restrictive

One of the main reasons tags like img, script are excused from same-origin policy is for ease of development of the web. It’s nice to be able to link to external images and to reuse scripts from another domain that one owns. However, the same-origin policy is restrictive for sharing “data” between domains. Data is usually offered by web services as XML or JSON. Unfortunately, the popular way for web pages to fetch data – XHR (aka AJAX), also follows the same origin policy.

JSONP – Sharing data across domains

JSONP (JSON with padding) takes advantage of the fact that script tags can point to different domains. A web page author issues a cross-domain API call that returns JSON data as part of the script tag. For example,

<script src="service.com/api/get_posts_as_json"> </script>

The result of the above call is JSON which not valid JavaScript. The browser will ignore the above just like it ignored HTML in the previous bank example. We need to somehow make the output of above a valid JavaScript. With JSONP, one writes:

function processPosts(result) { ... }

<script src="service.com/api/get_posts_as_json?callback=processPosts"></script>

service.com sees the callback parameter and responds with processPosts(json_result) which is valid JavaScript. All we have to do is to provide an implementation of processPosts function somewhere in our web page.

The exploit

First thing to notice about the build.phonegap.com’s JSONP API is that it uses build.phonegap.com as it’s domain. This is interesting because the main website is also hosted on build.phonegap.com. This means that if a user is logged into the build.phonegap.com, a evil web page can issue JSONP API calls with the user’s credentials.

Usually, “APIs” are hosted in a domain separate from the website. For example, api.service.com for API access and www.service.com for the website. This means that api.service.com can safely have JSONP support since the cookies from www.service.com and api.service.com won’t mix.

If APIs and the websites are hosted in the same domain distinguished only by path, then one needs to be more careful. For example, http://build.phonegap.com/api provides API access and http://build.phonegap.com/apps provides the website. Cookies do have a ‘path’ attribute which can be used to tell the browser that different paths in same domain don’t mix. However, http://build.phonegap.com/apps set the cookies in the ‘/’ path. This meant that cookies will be provided for http://build.phonegap.com/api too.

Do you see the exploit now? All I had to do was to use the JSONP api to access user information. In case you were wondering, exploits where evil websites steal credentials of a user or impersonate the unwitting user are called Cross-Site Request Forgery attacks (CSRF or XSRF).

Sample exploit code

Once you are logged onto build.phonegap.com, navigate to the links below:
Example 1 – List Your Apps
Example 2 – Rebuild Your Apps (WARNING: Don’t Click unless you know what you are doing)

Possible fixes

Since I don’t have access to build.phonegap.com code, I can only guess a few possible fixes

  • Strip out cookies when processing JSONP and rely on the HTTP auth header or the API token.
  • Make the website set cookies with path as /apps.
  • Remove JSONP support and add support for CORS.
  • Move api access to a separate domain.

    Current status

    I informed Adobe before this blog post and they quickly fixed the problem (I haven’t checked how).

  • May 23, 2012

    Blowfish for Qt

    Notekeeper uses Blowfish for encryption. Over the last few days, I separated out the encryption code, cleaned it up, and added some tests. And the result is QBlowfish – a Qt implementation of the Blowfish encryption algorithm.

    Blowfish requires the input size (in bytes) to be in multiples of 8. So QBlowfish optionally adds PKCS5 padding to the input data to make its size a multiple of 8. (For example, if the input is only 60 bytes long, 4 bytes will be padded to bring the bytecount to a multiple of 8.) When padding is enabled during decryption, QBlowfish will also remove the padded bytes from the output.

    QBlowfish is meant to be compiled with your code – you just drop in 3 files into your project. Then you can do stuff like:

    QBlowfish blowfishObj("Some secret key");
    blowfishObj.setPaddingEnabled(true);
    QByteArray clearText("Some stuff to encrypt");
    QByteArray encryptedBa = blowfishObj.encrypted(clearText);
    QByteArray decryptedBa = blowfishObj.decrypted(encryptedBa);

    You can get the code from github here.

    April 7, 2012

    Why did we make a Symbian app?

    With Notekeeper, we have our first paid app in the Nokia Store. So, why did we create an app for Symbian, despite it being one of the least popular smartphone platforms out there?

    Back in July 2011, I had just started using Evernote on my iPad, and I really liked the service. But I missed having access to those notes on my Nokia phone. The Evernote discussion forum had quite a few requests for a Symbian client, but Evernote had made it clear that they had no plans to make one.

    Given that background, it’s no surprise that I started thinking about writing an Evernote client for Symbian myself. Here’s my thought process on deciding whether to go ahead and do it or not:

    Genuine need: Like me, I could see that many others wanted an app like this, and they were expressing their need in blogs, forums and comments.

    Market size: There are significant number of phones running Symbian^3 today. Given that Notekeeper is Qt-based, I could have picked Symbian^3 or N9 as the first release platform. Symbian^3 won purely on market size.1 The number of people who buy Symbian^3 phones is on a steep decline, but Notekeeper targets those who already have these phones, not those who’re going to buy them.

    Niche problem: The addressable market might not be as big. The prevalent perception is that the subset of Nokia phone owners who would want to pay for an app is much smaller. So, an app that solves one niche problem well is a good fit for iOS, but might not be a good idea on Symbian. Can accessing Evernote (or, to generalize, accessing a cloud-based note-taking service) in Symbian be considered a niche problem? Maybe. Maybe not.

    Pricing: On a somewhat related note, even if the app solves a problem people have, and solves it well, there might be a reluctance to pay a premium price (like in the case of PaperMill for Android). Of course, there have also been instances where a quality app has sold well: Gravity.
    Given these, I thought the risks were well worth taking and went ahead with creating Notekeeper.

    Future: Symbian isn’t quite dead yet, but it’s days are numbered.2 Qt5 won’t support Symbian, as was known long back,3 but it has no business doing that – major Qt releases are for the long term (Qt4 was released in 2005, and will be in use for a few more years to come), and there’s no need for Symbian support when Nokia’s strategy is to move out of Symbian. That said, there’s a lot happening with the immediate future of Symbian in mind: Qt 4.8 already runs on Symbian, and a good amount of Symbian-specific work4 has gone into Qt 4.8.1, released last week.

    So, given that the future is not with Symbian, we won’t be writing Symbian apps a few years from now. But right now, it doesn’t look like it’s such a bad idea.

    1. 16.8 million Symbian^3 devices sold in Q3 2011 alone, given that the “Smart devices” mentioned here couldn’t have included the N9, which started shipping only in the last week of that quarter. The number of N9s shipped has not been made public, but my guess is that it’s far less than the number of Symbian^3 devices sold. []
    2. Updates and support for Symbian will continue “atleast upto” 2016. []
    3. “The initial thinking is that Qt 5 will focus on a small set of operating systems/platforms (i.e. platforms Wayland and X11 on Linux, Mac and Windows)” – Qt Blog, May 2011 []
    4. Changes in Qt 4.8.1 []

    March 23, 2012

    Coming soon: Evernote meets Symbian

    Evernote is a fantastic note-taking service, and one of its strengths is that it’s available on a lot of platforms. Nevertheless, Symbian has always lacked a proper Evernote client1. Till now.

    Notekeeper is a Qt app I have developed that acts as an Evernote client for Symbian Anna / Belle devices. Key features are:

    • Notes can be created / edited even without an internet connection; changes will be pushed to Evernote during the next sync
    • Supports notebooks, tags
    • Lets you search for a note
    • Can access rich text notes (but these notes are not freely editable)
    • Can see images in notes (but cannot add / remove images, for now2)
    • Supports favourite notes and offline notebooks

    Notekeeper preview

    Notekeeper: Start screen, Plain text note and Rich text note

    I’m currently testing and bugfixing the app. I hope to get it published in the Nokia Store in about a week or so as a paid app. I’m also considering making a trial version for people to try before buying.

    The app uses a Qt/C++ backend to talk to the Evernote Cloud API, and wraps it all up with a UI designed with QML and Qt Quick Components. Many thanks to Evernote for offering a complete, well-designed and free API to access their service, which makes it possible to create an app like this.

    Update Apr 7, 2012: Notekeeper has launched. Trial version is undergoing QA.

    1. Not counting WRT widgets like this and this []
    2. Adding / removing images is planned for the next release []

    December 19, 2011

    Qt Quick Best Practices

    I noticed that the video and slides for my Qt Dev Days 2011 talk in Munich are now online. Here’s the video and here are the slides.

    I couldn’t attend the event in San Francisco because I had to attend to some personal matters. Johannes and Donald covered for me there on very short notice (thanks guys!). For that matter, I have been mostly ‘offline’ for the last 2 months or so and will continue to be offline for atleast end of this month. So, if you sent me mail, I will get back at some point :)

    November 16, 2011

    Sharing wifi connection over ethernet

    I have a wireless router at home which is physically far from the place I actually work. I required an internet connection to update my Arch machine. I could use wpa_supplicant to connect to internet using wifi but I thought I would explore the option of connecting my laptop back to back with another that already had a wifi connection.

    Let’s call the computer with wifi connected to the router as I (as in connected to Internet). Let’s call the other computer C.

    1. Connect I and C using normal ethernet cable. You don’t really need a cross-over cable since most ethernet cards these days are smart enough to do automatic cross-over.

    2. Select a IP range for the ethernet connection between I and C. I chose 192.168.10.x.

    3. On computer I, set the ip : sudo ip addr add 192.168.10.1/24 dev eth0

    4. On computer I, Enable ip forwarding and setup iptables to masquerate (nat) the wifi connection.

    sysctl -w net.ipv4.ip_forward=1
    iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE

    5. On Computer I, install and configure dnsmasq. dnsmasq is a dhcp and dns server. The only configuration I needed after installing dnsmasq was adding a line “dhcp-range=192.168.10.100,192.168.10.150,12h” in /etc/dnsmasq.conf. That line specifes the ip range for dhcp leased addresses and the validity time. dns support is enabled by default, so nothing to configure there.

    6. Run dnsmasq. Just “sudo dnsmasq”. I actually used “dnsmasq –no-daemon” which lets me the see the debug output on the console.

    That’s it. Computer C should not get an IP address and be able to access the internet. You can use ‘dhclient eth0′ or ‘dhcpcd eth0′ to get an IP address through DHCP.

    October 21, 2011

    Qt DevDays 2011

    I am just about to catch my flight to Munich to attend Qt Developer Days 2011 where I will be giving a talk on Qt Quick Best Practices. I have been working with QML exclusively for over a year now and this talk is in essence a summary of all the things I have learnt about it. This will also be my 6th developer days as attendee and third time as a speaker.

    Am really looking forward to interesting conversations about Qt5, Qt Quick and the Qt project :-)

    August 20, 2011

    When sqlite queries fail for no reason

    If you have worked with QtSql, you might have hit the dreaded “Parameter count mismatch” for your perfectly valid SQL query. The problem is excruciatingly hard to debug because the query itself works perfectly fine with the sqlite3 tool.

    Here’s the solution: Compile Qt with -system-sqlite.

    The problem: Qt uses it’s own sqlite3 headers under src/3rdparty by default which are completely out of date. Qt 4.7 has sqlite3 header from 2009-10-14 version 3.6.18. Almost 2 years old and current sqlite version is 3.7.7! That’s like using Qt 4.5.3 in 2011 :-) FTS3/4 table queries fail consistently when using Qt’s own headers.

    I have opened QTBUG-21040.

    August 9, 2011

    On WebKit and WebKit2

    Ever heard of WebKit2 and wondering what it means from a Qt perspective? Here’s an attempt to explain QtWebKit and QtWebKit2 in simple terms. I make no attempt to be completely technically correct, it’s meant to be able to explain terminology to the WebKit uninitiated.

    In WebKit lingo, “WebCore” is the thing that takes of parsing/layouting/rendering of various css/svg/html documents, providing DOM bindings etc. “JavaScriptCore” implements JavaScript support and is also referred to as SFX (Squirrel fish Extreme). JavaScriptCore can be used as a stand alone JavaScript engine and has no dependencies on WebCore. WebCore uses JavaScriptCore to support JavaScript in web pages. WebCore also contains support for NPAPI plugins (like flash). “WebKit” uses the WebCore to build a platform/toolkit specific API. For example, the Qt “WebKit” port provides QWebElement which exposes the WebCore’s DOM. By definition, WebKit is platform/toolkit/port specific. The Qt port is simply called QtWebKit.

    The QtWebKit port is released periodically independent of Qt releases. These ports have the number QtWebKit 2.0, QtWebKit 2.1, QtWebKit 2.2 etc. QtWebKit 2.0 is identical to what was shipped with Qt 4.7.0. QtWebKit 2.1, intended to be mobile friendly, is not part of any shipping Qt release. QtWebKit 2.2, which will be shipped as part of the upcoming Qt 4.8.0, is yet to be released.

    Now for WebKit2. The first and most important thing you should know about WebKit2 (even before you know what WebKit2 is) is that WebKit2 is NEITHER AN UPGRADE NOR A NEWER RELEASE of WebKit. It is a parallel port that can happily co-exist with “WebKit”. Let me reiterate: Stop trying to think of WebKit2 as WebKit version 2 :-) Think of it as a completely different API from existing WebKit.

    WebKit is the traditional in-process renderer. If you create 100 web pages, they all reside in one process. If one page causes a crash, it brings everything down. WebKit2 provides a system and an API to make it possible to render a page in a separate process. The process management is taken care of by WebKit2. The actual rendering of the page happens using WebCore. WebKit2, therefore, spawns out processes, renders pages in these processes and makes the end result available to the application. It provides mechanisms deliver events from the application to the rendering process. The Qt port of WebKit2 is simply called QtWebKit2. QtWebKit2 is what is used in the N9 browser.

    White-space has never been more important. QtWebKit 2.x is a completely different beast from QtWebKit2. QtWebKit 2.x is plain old QtWebKit releases. QtWebKit2 is Qt’s port of WebKit2. This unfortunate naming is a result of Apple announcing WebKit2 shortly after the Qt guys deciding to call their releases QtWebKit 2.x.

    WebKit2 and Chromium are similar in their goal. Chromium does not use WebKit2 and probably never will. The Chromium code was intended for the chromium browser specifically. The WebKit2 code was designed upfront to be an API. This difference in motivation resulted in different implementations. See this page for more details.

    Because of the multi-process nature of QtWebKit2, many APIs that existed in QtWebKit simply don’t exist anymore. WebKit2 design lends itself to an asynchronous API compared to WebKit where most API was synchronous. For example, DOM introspection of web pages using QWebElement is not possible since the web page’s DOM resides in another process.

    QtWebKit2 has a hard dependency on Qt5 and is very much a moving target like Qt5. QtWebKit will probably not work well with Qt5, we have to wait and see.

    Current status: Nokia’s Qt WebKit team has decided to focus on QtWebKit2. They have decided to pass on maintainership of QtWebKit to someone else. At the time of writing, there is no publicly announced appointed maintainer to QtWebKit.

    Update: Mentioned about QtWebKit 2.x releases based on Jocelyn’s comment.

    Older Posts »