Symbian Blog: Articles by William Roberts

Integration Engineer in the Symbian Foundation

550 downloads, and rising…

Oct 30th by William Roberts Retweet Share on Facebook

The Kernel Taster Kit published on Wednesday 21st October has been downloaded by over 550 people in the last 9 days – 77 to the US, 66 to Russia, 42 to China and lots of other countries besides.

This is a great response – we were wondering about blogging that we’d reached 100 downloads, and 500+ was a real surprise. It has made me realise that we missed a few tricks with the kit and the documentation: we forgot to tell you how to get updates, and we forgot to tell you how you can contribute back (What were we thinking…?).

Time to fix that right now.

Get Connected…

The kit contained a snapshot of the source tree. There have been updates to the qemu package since then: Johnathan White at Accenture has added a sound driver and other good things which he demonstrated in the Hands-On Lab session at SEE – thanks, John!

To connect your copy of the source code to the Open Source Mercurial repositories on developer.symbian.org, you will need to install Mercurial, clone each of the three repositories directly into the source tree, and then you are all set to pull down future updates, save your changes, share changes with others and …

…Get Contributing!

As you explore with the code and make changes, you might find bugs: you might also find things which you think could be done better. Please help the Symbian community by raising these things in our Bugzilla database, which tracks both defects & enhancements. Better still, you could develop the necessary code changes to fix the problem, and offer them back as contributions by publishing them to the Mercurial repository.

If you haven’t previously seen the Symbian^3 product reference on the developer website, now’s a good time to visit to find guides to base porting and device drivers, among others, as well as API reference documentation. And don’t forget that the Symbian OS Internals book is available on our Wiki, if you want some introduction to the code, and our Forums are a good place to ask questions.

Thanks to everyone who has downloaded the kit so far – I’m thrilled that there is so much interest in this very technical “Under the Hood” part of the Symbian Platform.

William Roberts

PS. Our logs show that some of you click on the download links several times – please get in touch if you are having any difficulties downloading from our website.

Comments (10)

Green Shoots

Jun 24th by William Roberts Retweet Share on Facebook

No, not the economy. Instead, here are some recent signs that “the Open Source thing” is really getting started for the Symbian Platform.

Note: We are still in beta, so you will need to register to follow some of the links…

People are downloading and compiling the code

People are now raising issues in Bugzilla to report problems which they find when trying to use the code. Some of the reports come from  employees of member companies, but not all of them.

  • Ziteng Chen has reported several memory leak problems in the Webuis package, with suggested fixes. Great work, Ziteng, you are the first “member of the public” to help us in this way, and we thank you for it.
  • Johan Oreland from Sosco has raised bugs with suggested fixes about conflicting exports of the same source files

It’s good to see the reports and suggested fixes, but is that all…?

Package Owners are accepting contributions

We’ve seen the first contribution offered, accepted by the package owner and delivered back into Mercurial. A round of applause please for Alten Ltd with this contribution to the Comms Framework package, and to Remek Zajac for being the first package owner to run the contribution process for real.

And I’m getting to grips with my job

Well, making progress anyway. I’m happy to say that we have now the first PDK candidate which we’ve managed to build ourselves from the source code in Mercurial – I mentioned this at the end of my blog posting back in May, and it’s taken longer than I’d hoped (but not as long as I feared). Expect it to hit the web site this week.

Doesn’t sound like much? Maybe, but a journey of a thousand miles begins with getting up from the sofa, and these are positive signs that the journey is underway.

Comments (4)

A dilemma at the end of the line?

May 13th by William Roberts Retweet Share on Facebook

I have a technical choice to make, and I don’t know the right answer – perhaps you can help me?

I apologise in advance for a long and technical posting – it will doubtless be “dumbed down” in the opinion of those who already understand the problem, and “full of mind-numbing details” for those who don’t.

The Line Endings Problem

Symbian has millions of lines of source code, stored in text files. We want that code to be accessible to people who have PCs running Windows, and we’d also like it to be accessible to people running Linux, but those systems use different sequences of characters in a file to indicate the end of a line of text. It’s a standard problem faced by any programming project which expects to work with both Windows and Linux, and it’s usually solved by having your source code control system automatically convert text files into the appropriate format for each user.

Symbian is using Mercurial to store the files, but a fundamental Mercurial design decision is not to associate a “type” with each file. Another design decision is to leave the content of the files unaltered – there is no automatic translation in core Mercurial.

The standard Mercurial solution to the line endings problem is to ask all Windows users to enable a Mercurial extension which recognises text files, converts them from Windows format to Linux format before storing them in the repository, and converts them from Linux format back into Windows format when extracting them onto your local disk.

All of the Symbian Platform code has been added into Mercurial from Windows using the Mercurial extension, so they are stored internally in the Linux format.: that’s spread over 100+ repositories. The Tools code has not been handled in that way, so a further 30+ repositories store files in the Windows format.

So What’s Your Dilemma?

At present, anyone working on the Symbian platform will be using Windows. Enabling the Mercurial extension has to be done by each user changing either their global Mercurial settings, or by altering the settings in individual local copies of the repositories. If you have the extension enabled and read from a repository in the Linux format, you will get warnings about line endings for every file that you extract.  If you have the extension enabled and commit changes into a repository which doesn’t use the Linux format internally, then Mercurial will think that you are changing all of the files, even the ones you haven’t altered.

Clearly, I need to decide on a policy and apply it to all of the Symbian repositories. Equally clearly, the sooner the decision is made, the less disruption it will cause.

I can see two choices (which I will call Red and Blue because I don’t know which one is better than the other).

Red Choice – Linux format

Insist on using the Linux format for the internal storage in all repositories, requiring all Windows users to enable the extension.

This ought to make the files look right for all systems, regardless of how simple your compiler or text editor program is. A small snag is that there are some text files which are just data for use on the Symbian devices, and where the code currently has to have the Windows line-endings. The estart.txt file used to configure the Symbian file server is the one we’ve uncovered so far, and we would probably need to change the code to make it work with Linux line endings as well.

This solution forces all Windows users to enable the extension, and as there are 130+ repositories that’s probably something they would do in their global settings. Some other Open Source projects have taken this approach, e.g. NetBeans, but not very many, and could be very annoying for developers if they also work with code that has a different policy.

Blue Choice – No translation

Insist that Windows users do not use the extension, and instead require everyone to deal with the files in the format that they find them.

This is likely to mean that all text files are in the Windows format, and we just have to hope that Linux compilers and text editors will respect that and behave nicely.

It avoids making Windows users change anything, but if they have enabled the extension we could still get the whole repository changing “accidentally” unless they explicitly disable the extension for the Symbian Foundation repositories.

Help…!

What is the “common practice” for other projects which use Mercurial and support development on Windows?

Do you favour the Red choice over the Blue choice? Is there a Green choice which I haven’t considered?

Please let me know what you think I should do….

Thanks,

William

Comments (33)

Where is the development kit?

May 1st by William Roberts Retweet Share on Facebook

Those with access to the beta Symbian website will know that we currently have source code available, but no binaries. I thought I’d let you know what’s happening behind the scenes, and give you some idea of what’s coming.

On your marks …

At the beginning of April, Symbian received the initial contribution from Nokia: that’s the source code in our Mercurial repositories. It’s a subset of the live source code used within Nokia, the difference being the source code which Nokia has licensed from other companies.

We also received a set of binaries and tools built by Nokia from their live source, and then filtered to remove anything generated from the licenced source code. This isn’t a contribution and we won’t be distributing it, but it’s a fantastic help while we are establishing our build machines and build processes.

Get set …

Using the source code in Mercurial, we have been running builds to see which of those binaries we can generate for ourselves. The binaries that we can build will form the core of the Platform Development Kit (PDK) which will be freely downloadable by anyone once the Symbian website goes live.

We have also worked out which binaries we can’t generate from the source code in Mercurial. For each of those files, we are now asking

  • What is this file for? What happens if we leave it out?
  • If it’s needed, how can we deliver it alongside the PDK?

Some files are used when compiling the source code that we do have. Other files are executed when running the Symbian Platform emulator. Our generic solution for those files is to provide them under an “R&D licence”, which allows them to be used for development purposes but not sold in products. The only question then is who to licence each file from…?

Go ?

Not quite yet. We’ve reached the point of repackaging that original delivery from Nokia into PDK plus R&D licenced binaries that contain a working emulator, and we are close to being able to generate that structure from a full platform build which starts with just source code in Mercurial and the R&D licenced binaries.

We are also just about to receive an update from Nokia, reflecting an extra 4 weeks of work on the code including sorting out various known issues in the original delivery. This is intended to be the last delivery of binaries to Symbian, so we will use it to finalise the set of R&D licenced binaries. We then add the binaries we can build from the updated source, and publish!

Comments (6)