May 23, 2012

Blowfish for Qt

Notekeeper uses Blowfish for encryption. Over the last few days, I separated out the encryption code, cleaned it up, and added some tests. And the result is QBlowfish – a Qt implementation of the Blowfish encryption algorithm.

Blowfish requires the input size (in bytes) to be in multiples of 8. So QBlowfish optionally adds PKCS5 padding to the input data to make its size a multiple of 8. (For example, if the input is only 60 bytes long, 4 bytes will be padded to bring the bytecount to a multiple of 8.) When padding is enabled during decryption, QBlowfish will also remove the padded bytes from the output.

QBlowfish is meant to be compiled with your code – you just drop in 3 files into your project. Then you can do stuff like:

QBlowfish blowfishObj("Some secret key");
blowfishObj.setPaddingEnabled(true);
QByteArray clearText("Some stuff to encrypt");
QByteArray encryptedBa = blowfishObj.encrypted(clearText);
QByteArray decryptedBa = blowfishObj.decrypted(encryptedBa);

You can get the code from github here.

April 7, 2012

Why did we make a Symbian app?

With Notekeeper, we have our first paid app in the Nokia Store. So, why did we create an app for Symbian, despite it being one of the least popular smartphone platforms out there?

Back in July 2011, I had just started using Evernote on my iPad, and I really liked the service. But I missed having access to those notes on my Nokia phone. The Evernote discussion forum had quite a few requests for a Symbian client, but Evernote had made it clear that they had no plans to make one.

Given that background, it’s no surprise that I started thinking about writing an Evernote client for Symbian myself. Here’s my thought process on deciding whether to go ahead and do it or not:

Genuine need: Like me, I could see that many others wanted an app like this, and they were expressing their need in blogs, forums and comments.

Market size: There are significant number of phones running Symbian^3 today. Given that Notekeeper is Qt-based, I could have picked Symbian^3 or N9 as the first release platform. Symbian^3 won purely on market size.1 The number of people who buy Symbian^3 phones is on a steep decline, but Notekeeper targets those who already have these phones, not those who’re going to buy them.

Niche problem: The addressable market might not be as big. The prevalent perception is that the subset of Nokia phone owners who would want to pay for an app is much smaller. So, an app that solves one niche problem well is a good fit for iOS, but might not be a good idea on Symbian. Can accessing Evernote (or, to generalize, accessing a cloud-based note-taking service) in Symbian be considered a niche problem? Maybe. Maybe not.

Pricing: On a somewhat related note, even if the app solves a problem people have, and solves it well, there might be a reluctance to pay a premium price (like in the case of PaperMill for Android). Of course, there have also been instances where a quality app has sold well: Gravity.
Given these, I thought the risks were well worth taking and went ahead with creating Notekeeper.

Future: Symbian isn’t quite dead yet, but it’s days are numbered.2 Qt5 won’t support Symbian, as was known long back,3 but it has no business doing that – major Qt releases are for the long term (Qt4 was released in 2005, and will be in use for a few more years to come), and there’s no need for Symbian support when Nokia’s strategy is to move out of Symbian. That said, there’s a lot happening with the immediate future of Symbian in mind: Qt 4.8 already runs on Symbian, and a good amount of Symbian-specific work4 has gone into Qt 4.8.1, released last week.

So, given that the future is not with Symbian, we won’t be writing Symbian apps a few years from now. But right now, it doesn’t look like it’s such a bad idea.

  1. 16.8 million Symbian^3 devices sold in Q3 2011 alone, given that the “Smart devices” mentioned here couldn’t have included the N9, which started shipping only in the last week of that quarter. The number of N9s shipped has not been made public, but my guess is that it’s far less than the number of Symbian^3 devices sold. []
  2. Updates and support for Symbian will continue “atleast upto” 2016. []
  3. “The initial thinking is that Qt 5 will focus on a small set of operating systems/platforms (i.e. platforms Wayland and X11 on Linux, Mac and Windows)” – Qt Blog, May 2011 []
  4. Changes in Qt 4.8.1 []

March 23, 2012

Coming soon: Evernote meets Symbian

Evernote is a fantastic note-taking service, and one of its strengths is that it’s available on a lot of platforms. Nevertheless, Symbian has always lacked a proper Evernote client1. Till now.

Notekeeper is a Qt app I have developed that acts as an Evernote client for Symbian Anna / Belle devices. Key features are:

  • Notes can be created / edited even without an internet connection; changes will be pushed to Evernote during the next sync
  • Supports notebooks, tags
  • Lets you search for a note
  • Can access rich text notes (but these notes are not freely editable)
  • Can see images in notes (but cannot add / remove images, for now2)
  • Supports favourite notes and offline notebooks
Notekeeper preview

Notekeeper: Start screen, Plain text note and Rich text note

I’m currently testing and bugfixing the app. I hope to get it published in the Nokia Store in about a week or so as a paid app. I’m also considering making a trial version for people to try before buying.

The app uses a Qt/C++ backend to talk to the Evernote Cloud API, and wraps it all up with a UI designed with QML and Qt Quick Components. Many thanks to Evernote for offering a complete, well-designed and free API to access their service, which makes it possible to create an app like this.

Update Apr 7, 2012: Notekeeper has launched. Trial version is undergoing QA.

  1. Not counting WRT widgets like this and this []
  2. Adding / removing images is planned for the next release []

December 19, 2011

Qt Quick Best Practices

I noticed that the video and slides for my Qt Dev Days 2011 talk in Munich are now online. Here’s the video and here are the slides.

I couldn’t attend the event in San Francisco because I had to attend to some personal matters. Johannes and Donald covered for me there on very short notice (thanks guys!). For that matter, I have been mostly ‘offline’ for the last 2 months or so and will continue to be offline for atleast end of this month. So, if you sent me mail, I will get back at some point :)

October 21, 2011

Qt DevDays 2011

I am just about to catch my flight to Munich to attend Qt Developer Days 2011 where I will be giving a talk on Qt Quick Best Practices. I have been working with QML exclusively for over a year now and this talk is in essence a summary of all the things I have learnt about it. This will also be my 6th developer days as attendee and third time as a speaker.

Am really looking forward to interesting conversations about Qt5, Qt Quick and the Qt project :-)

August 20, 2011

When sqlite queries fail for no reason

If you have worked with QtSql, you might have hit the dreaded “Parameter count mismatch” for your perfectly valid SQL query. The problem is excruciatingly hard to debug because the query itself works perfectly fine with the sqlite3 tool.

Here’s the solution: Compile Qt with -system-sqlite.

The problem: Qt uses it’s own sqlite3 headers under src/3rdparty by default which are completely out of date. Qt 4.7 has sqlite3 header from 2009-10-14 version 3.6.18. Almost 2 years old and current sqlite version is 3.7.7! That’s like using Qt 4.5.3 in 2011 :-) FTS3/4 table queries fail consistently when using Qt’s own headers.

I have opened QTBUG-21040.

August 9, 2011

On WebKit and WebKit2

Ever heard of WebKit2 and wondering what it means from a Qt perspective? Here’s an attempt to explain QtWebKit and QtWebKit2 in simple terms. I make no attempt to be completely technically correct, it’s meant to be able to explain terminology to the WebKit uninitiated.

In WebKit lingo, “WebCore” is the thing that takes of parsing/layouting/rendering of various css/svg/html documents, providing DOM bindings etc. “JavaScriptCore” implements JavaScript support and is also referred to as SFX (Squirrel fish Extreme). JavaScriptCore can be used as a stand alone JavaScript engine and has no dependencies on WebCore. WebCore uses JavaScriptCore to support JavaScript in web pages. WebCore also contains support for NPAPI plugins (like flash). “WebKit” uses the WebCore to build a platform/toolkit specific API. For example, the Qt “WebKit” port provides QWebElement which exposes the WebCore’s DOM. By definition, WebKit is platform/toolkit/port specific. The Qt port is simply called QtWebKit.

The QtWebKit port is released periodically independent of Qt releases. These ports have the number QtWebKit 2.0, QtWebKit 2.1, QtWebKit 2.2 etc. QtWebKit 2.0 is identical to what was shipped with Qt 4.7.0. QtWebKit 2.1, intended to be mobile friendly, is not part of any shipping Qt release. QtWebKit 2.2, which will be shipped as part of the upcoming Qt 4.8.0, is yet to be released.

Now for WebKit2. The first and most important thing you should know about WebKit2 (even before you know what WebKit2 is) is that WebKit2 is NEITHER AN UPGRADE NOR A NEWER RELEASE of WebKit. It is a parallel port that can happily co-exist with “WebKit”. Let me reiterate: Stop trying to think of WebKit2 as WebKit version 2 :-) Think of it as a completely different API from existing WebKit.

WebKit is the traditional in-process renderer. If you create 100 web pages, they all reside in one process. If one page causes a crash, it brings everything down. WebKit2 provides a system and an API to make it possible to render a page in a separate process. The process management is taken care of by WebKit2. The actual rendering of the page happens using WebCore. WebKit2, therefore, spawns out processes, renders pages in these processes and makes the end result available to the application. It provides mechanisms deliver events from the application to the rendering process. The Qt port of WebKit2 is simply called QtWebKit2. QtWebKit2 is what is used in the N9 browser.

White-space has never been more important. QtWebKit 2.x is a completely different beast from QtWebKit2. QtWebKit 2.x is plain old QtWebKit releases. QtWebKit2 is Qt’s port of WebKit2. This unfortunate naming is a result of Apple announcing WebKit2 shortly after the Qt guys deciding to call their releases QtWebKit 2.x.

WebKit2 and Chromium are similar in their goal. Chromium does not use WebKit2 and probably never will. The Chromium code was intended for the chromium browser specifically. The WebKit2 code was designed upfront to be an API. This difference in motivation resulted in different implementations. See this page for more details.

Because of the multi-process nature of QtWebKit2, many APIs that existed in QtWebKit simply don’t exist anymore. WebKit2 design lends itself to an asynchronous API compared to WebKit where most API was synchronous. For example, DOM introspection of web pages using QWebElement is not possible since the web page’s DOM resides in another process.

QtWebKit2 has a hard dependency on Qt5 and is very much a moving target like Qt5. QtWebKit will probably not work well with Qt5, we have to wait and see.

Current status: Nokia’s Qt WebKit team has decided to focus on QtWebKit2. They have decided to pass on maintainership of QtWebKit to someone else. At the time of writing, there is no publicly announced appointed maintainer to QtWebKit.

Update: Mentioned about QtWebKit 2.x releases based on Jocelyn’s comment.

July 18, 2011

Qt/Caca Lighthouse Plugin

lighthouse_ascii

At the Qt Contributors Summit, Johannes‘ showed me his Qt/Caca Lighthouse plugin. Caca is a graphics library to output text instead of pixels. So this plugin lets you run Qt programs on the console :-)

His code needed some love, so I forked it and cleaned it up. Caca does not provide a event fd and so we have to keep polling caca for events. Since this wasn’t ideal, I moved the event handling to a separate thread and blocked for events. Unfortunately, I found that the caca library is not thread-safe and rendering and processing events in separate threads makes caca crash at randomly. So, I ended up moving the rendering to the event processing thread and having to resurrect the 20ms event timer again :-( The cool thing though is that now Qt renders to QImage in the main ui thread and hands it off to caca. Caca opens a X connection (or similar), converts the image into text, displays a window and handles events in another thread. With some refactoring and thanks to QMetaObject::invokeMethod, the threaded and non-threaded rendering are pretty much the same and can be switched using an environment variable (THREADED_CACA=1).

Animated tiles:

If you want to hack further, code is on gitorious. (Caca doesn’t seem to deliver gpm events with ncurses, so that would be a nice fix to have)

Update: Welcome slashdot readers

June 15, 2011

Qt Contributors’ Summit

Up until 3 hours back, I wasn’t intending to attend QCS since my visa was expected to arrive late. We have 15 working days waiting period here just to get to the visa interview phase. My tickets were booked for the 19th June to Berlin nevertheless for a Qt Media Hub hackfest. I attended the visa interview yesterday and mentioned that it would be great if they could give me visa for the 16th Jun. They wanted rebooked hotel and plane booking in under 20 mins and a german invite (The english invite left them wondering why a certain Alexandra from Oslo is inviting me to a conference in Germany)! The only internet parlor I found nearby was hopeless and I returned back home.

Incredibly, the German consulate was kind enough to not only send me my visa a couple of hours of back but they also gave me a visa starting from 16th June! With very many things working to my favor (like just a 100 EUR rescheduling fee for the ticket), I am very happy to say I will arrive on 16th afternoon!

Qt Contributors' summit

Qt Contributors' summit

April 11, 2011

Publishing Symbian apps to Ovi – a quick howto

I was trying to publish an app to Ovi last month using Nokia’s free signing service (the Nokia-recommended way), and was a little confused with instructions spread across different sites and with different approaches. I mailed Ovi Publish to understand the process better. (Thanks to Kevin from Ovi Publish for patiently answering all my questions.) From my experience, here’s a one-page howto on Publishing to Ovi.

You can use these instructions if you:

  • Want to publish a Qt app for a Nokia Symbian device to the Ovi Store
  • Do not use any of these in TARGET.CAPABILITY in your pro file: NetworkControl, MultimediaDD, CommDD, DiskAdmin, AllFiles, DRM and TCB
  • Have Nokia Qt SDK 1.0 (with Smart Installer) or Qt SDK 1.1 (beta/pre-release) in Windows platform to build the release executable
  • Have 1 euro to spare on your credit card

Pre-submission instructions:
(Start on this at least 4 days before your app’s coding/testing is expected to complete)

1. Go to publish.ovi.com and register yourself (it’ll cost you 1 euro)

2. Make a request for Nokia UIDs by sending an email to publishtoovi.support@nokia.com with the following information:

  • your Ovi Publish login id
  • the company name under which you registered for Ovi Publish (if any)
  • IMEI numbers of devices (upto 5) you intend to use for testing the app

3. You should receive a reply from Ovi Publish with a Terms and Conditions pdf (something like this (pdf)), asking you to print it, sign (or get signed by a company representative), scan and email that back to them. Sign on and scan all pages.

4. Once you email them the scan, you should receive the following:

  • Five 8-digit hexadecimal numbers: Nokia UIDs (you can publish exactly one app with each UID)
  • Three files (probably in a zip archive): <something>.cer, <something>.key and <somethingelse>.sis

5. Specify the version number of your app in the .pro file.

VERSION = 1.0.0

6. Create an icon for your app (256×256 pixels, PNG format) to be shown in the phone’s applications menu. Update your .pro file with a path to the icon.

symbian:ICON = icon.png

(Warning: Using PNGs as icon is undocumented afaik, but works. We’re supposed to use Tiny SVGs but the svgs I created out of Inkscape didn’t work – maybe they weren’t “tiny” enough.)

7. Identify the Symbian versions/devices you’re targeting the app for. If you support all Symbian versions, you can skip this step. If you support only Symbian^1 and ^3 (like me), include the following in your .pro file:

symbian {
  supported_platforms = \
    "[0x1028315F],0,0,0,{\"S60ProductID\"}" \ # only Symbian^1
    "[0x20022E6D],0,0,0,{\"S60ProductID\"}"   # and Symbian^3
  # remove default platforms
  default_deployment.pkg_prerules -= pkg_platform_dependencies
  # add our platforms
  platform_deploy.pkg_prerules += supported_platforms
  DEPLOYMENT += platform_deploy
}

For including support for other Symbian versions, pickup the appropriate hex codes from here. If a user tries to install the app in a version of Symbian not specified in the list, he will see a warning during installation (something like: “This software is not supported in this platform. Continue installation / Cancel?”).

8. Include the vendor name in the .pro file. (If you specified a company name in step 2, the vendor name should be the company name. Else, the vendor name can be the name of the author of the app.)

symbian {
  vendorinfo = "%{\"InsertVendorNameHere\"}" \
                      ":\"InsertVendorNameHere\""
  vendor_deploy.pkg_prerules += vendorinfo
  DEPLOYMENT += vendor_deploy
}

Submission instructions:
(Start on this after your app’s coding/testing is complete.)

9. Create a screenshot of the app. You can get it by PrintScreen-ing on Qt Simulator running at phone resolution. But in case your app doesn’t work well on the simulator (like mine), you can temporarily hack your QMainWindow to draw itself to a file on the device.

MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent)
{
    ...
    // grab a screenshot 5 secs after startup
    QTimer::singleShot(5000, this, SLOT(getScreenshot()));
}

void MainWindow::getScreenshot()
{
    QPixmap pixmap = QPixmap::grabWidget(this);
    pixmap.save("E:/app_screenshot.png", "PNG");
    qDebug() << "Saved screenshot";
}

One way to transfer this image from your phone to your PC is to choose Connectivity > USB > Mass storage in the phone and then connect the phone to the PC by USB cable.

Ovi accepts only square images as screenshots (why that is the case when my phone doesn’t have a square screen is one of those life’s little mysteries). So open up the screenshot in your favourite image editor (Gimp or Inkscape or MS Paint or whatever) and cut out a square portion or add a square background (256×256 or bigger, PNG or JPG format).

10. You now have two options to create the .sis file for submission to Ovi Publish. Option A is dead simple, but needs you to have a very recent Qt Creator. Option B is a tad complexer, but is also nicer as it lets us try out the .sis file before submission. Make your pick.

Option A:

  1. Check your Qt Creator’s version (in Windows, where you will be compiling the app for shipping). If it’s anything less than 2.2.81, install the latest bleeding-edge snapshot version. (If you compile Qt Creator yourself from source, make sure you have commit 8d781af).
  2. Open your project in this uber-recent version of Qt Creator, select the Symbian Device build target. Choose Build > Publish Project and follow instructions from the wizard that opens up.
  3. In the wizard, change the Application UID field to one of the Nokia UIDs sent to you in step (4) by Ovi Publish.
  4. Once you hit the Commit button in the wizard, you might have to wait a while until the Open Containing Folder button gets enabled. When it does, you can click that to pickup <yourapp>_installer.sis from that folder. Copy that file to someplace safe (maybe add it to your version control). This .sis is what you’ll be uploading to Ovi Publish. You cannot try out this .sis on the device, though.
  5. During step iv, Qt Creator changes the TARGET.UID3 specified in your pro file. To continue testing the app on your device (during further development), you might want to revert the UID to the earlier value.

Option B:

  1. Change the TARGET.UID3 specified your .pro file to one of the Nokia UIDs sent to you in step (4) by Ovi Publish.
  2. Open Qt for Symbian Command Prompt from the Windows start menu and run the following commands. Use the local path to the .cer and .key files you received in step (4).
    make clean
    rm *.pkg
    rm *.sis
    qmake
    make release-gcce
    make sis QT_SIS_CERTIFICATE=path/to/something.cer QT_SIS_KEY=path/to/something.key
  3. Open <yourapp>_installer.pkg in a text editor. You should find something like:
    #{"appname installer"},(0xA000D7CE),1,0,0

    where appname is the name of your app and 1,0,0 is the version number you specified in step (5). The UID has to be either 0x2002CCCF or 0xA000D7CE. If it’s anything else, change the UID to 0x2002CCCF.

  4. You should also find a line specifying where it should get installed on the device:
    "C:/path/in/dev/env/to/app/appname.sis" – "c:\private\2002CCCE\import\appname.sis"

    Change the “c:\private\2002CCCE\import\” in the right hand side to “!:\private\2002CCCE\import\”. The “!” tells the Symbian packager to allow the user of the app to choose a drive during installation.

  5. Go back to the open Qt for Symbian Command Prompt window and run:
    make installer_sis QT_SIS_CERTIFICATE=path/to/something.cer QT_SIS_KEY=path/to/something.key

    This creates a <yourapp>_installer.sis in the same folder. Copy that file to someplace safe (maybe add it to your version control). This .sis is what you’ll be uploading to Ovi Publish.

  6. Pick a test device (whose IMEI you sent to Ovi Publish in step (2)), and install on it the .sis file you received in step (4).
  7. You can now try out the <yourapp>_installer.sis on the test device. Apart from being able to make a test run of the actual binary you’re going to submit, you can get a feel for what the user experience would be during installation. (For eg., I noticed that the installation fails if there’s no internet, even if the device has the required version of Qt.) You can also make sure that your (or your company’s) name and the app’s version number are showing up correctly during installation.

11. Login to publish.ovi.com, and hit Create New Content. Choose Qt Based App, answer a few legal questions, and hit Continue. Fill the form that follows, making use of the icon you created in step (6) and the screenshot you created in step (9). Hit Create.

12. Click on the Content files tab. Click the button next to Qt file and upload the <yourapp>_installer.sis file you created in step (10).

13. Enter the Nokia UID that you used in step (10).

14. Select Requesting Nokia Signing as the Symbian Signed Type. Hit Upload & Continue

15. For the platforms/devices that you identified in step (7), specify the amount of testing done. Leave the rest of the platforms as is. Click Next

16. Select the countries to publish in, hit Next, select the languages that it works with, hit Save. Then click on the Submit to QA button under the Distribution tab. You might have to wait 6-8 working days for the QA to complete.

17. Be happy.

References:
[1] Guide_to_Publishing_Qt_Applications_to_the_Ovi_Store
[2] Qt content Q&A
[3] Ovi Publisher Guide (pdf)
[4] Packaging and signing


Update:
11/May/2011: Commands order changed in step 10 B (ii) – the .pkg files created by qmake shouldn’t get deleted.

Older Posts »