Hi, I’m Tyson Key, a college student currently residing somewhere in the north of England.
When I’m not studying, I happen to advocate and contribute to open source, and open platforms and enjoy spending time testing and documenting software, and exploring various aspects of low-level operating system and computer networking technology – all whilst trying to be a generally reasonable and agreeable person…
Of course, I also happen to be a heavy user, and advocate of the Symbian platform, and have been involved with the Symbian Foundation to varying extents, since its announcement.
Introduction aside, I’ve been invited to introduce the Symbian Virtual Platform. This is a project that I’ve been involved in beta testing, prior to the release of the Symbian Kernel Taster Kit.
What’s the Symbian Virtual Platform?
The Symbian Virtual Platform consists of a customised version of QEMU (an Open Source CPU instruction set and hardware emulator), containing several additional features, along with a corresponding baseport referred to as Syborg.
This combination potentially represents the future of inexpensive, accurate, user-level application testing, and low-level debugging and experimentation on the Symbian Platform, without the need for additional hardware.
As a bonus, some of the hallmark causes of frustration with the older Windows-based emulator (commonly referred to as WINS or WINSCW dependant upon toolchain used) are removed, such as:
- Low performance
- Simulation inaccuracy (e.g. differing memory models with various limitations, behavioural differences from real hardware, and disabled/restricted software functionality)
- Platform dependence – QEMU is well-supported under Windows, most Linux distributions and Mac OS X, and also runs under many other operating systems
What can it do for you?
The “official”/mainline version of QEMU already emulates an ARMv5 architecture CPU, and a number of hardware features including – but not limited to:
- Serial ports – complete with the ability to redirect traffic to a file, which is essential for Symbian Platform debugging and problem diagnosis
- Ethernet chipsets – a driver for which is included in Syborg, although additional work is necessary, before being suitable for intensive use
- Various forms of Flash memory devices (e.g. MultiMediaCard/SD Card, and SPI NAND Flash)
- An audio chipset – support for this in the baseport was recently contributed by Johnathan White of Accenture
However, the version supplied by Symbian also adds a number of useful features that enhance this functionality to potentially save development time and effort, and let users/enthusiasts test software quickly and easily without access to a physical handset, including:
- The ability to run real Symbian OS binaries “as-is” – meaning that you no longer also need to rebuild software to run within an x86 and Windows-only simulator
- A host file system-backed emulated drive (
S:\on the QEMU guest, which maps to\svphostfson the host) that acts as an extension of firmware when custom executables are added, and allows for quickly moving files between the host and guest whilst the Platform is running - A Python-based dynamic hardware modelling system that allows for varying degrees of control over the behaviour of the emulated system at runtime, without needing to modify QEMU source code and rebuild
The source code for these enhancements, and the baseport are available in the Symbian Foundation repositories.
Intrigued? Here’s what it looks like…
An EShell session, showing the output of various utilities, on a Windows XP host
What have I done with QEMU?
- Built several ROM images containing various Platform components, and documented the results on the Symbian forums so that others can assist in resolving problems, and enhancing the results
- Spent time debugging the execution of various binaries using serial port file redirection, and resolving dependency issues, as they arose
Which leads nicely on to a practical example, based upon a problem that I faced…
Getting the EUserHL Walkthrough test suite running
A forum thread that I started, with additional suggestions from the community, prior to writing this post is available here for reference. I’ll now attempt to explain how I answered a not-too-uncommon question using several features of the Symbian Virtual Platform, and a process of elimination -
How can I test a Platform component (in this case – EUserHL), when I don’t have access to the source code?
The initial steps involved boil down to:
- Install and configure the Symbian Kernel Taster Kit and RVCT 4.0 according to the instructions on the Symbian Wiki
- Create a directory on the host file system at \svphostfs and a set of subdirectories to hold executables and DLLs inside: sys\bin
- Copy \epoc32\release\armv5\urel\euserhl_walkthrough.exe and \epoc32\release\armv5\urel\euserhl.dll to \svphostfs\sys\bin
- Start a cmd.exe prompt session, and switch to \symbian-qemu-0.9.1\bin\
- Run QEMU using the prebuilt Syborg ROM image using: arm-none-symbianelf-qemu-system -M \sf\adaptation\qemu\baseport\syborg\syborg.dtb -kernel \epoc32\rom\syborg.img within the active cmd.exe session
- Once the ROM has booted to EShell, attempt to invoke euserhl_walkthrough.exe
- Notice that execution simply bails, before printing a rather vague, generic “Not found” error message
Obviously, this a rather sorry state of affairs – but there’s hope. So, let’s move on:
- Halt the previous QEMU session
- This time, run QEMU again using arm-none-symbianelf-qemu-system -M \sf\adaptation\qemu\baseport\syborg\syborg.dtb -kernel \epoc32\rom\syborg.img -serial file:”EUserHLDebug.txt” to redirect serial port output to a file
- Within the EShell session, invoke trace /L 4 to enable debugging of executable loading and DLL iteration
- Attempt to invoke euserhl_walkthrough.exe once again within EShell, to receive the familiar “Not found” error
- Open \symbian-qemu-0.9.1\bin\EUserHLDebug.txt on the host, using your favourite text editor
- Following the chain of execution, from euserhl_walkthrough.exe onwards, to euserhl.dll, we can see that various dependency DLLs are located successfully (demarcated by “New image found“)
- However, if we closely follow euserhl.dll, the fact that a critical dependency is missing at a late stage is alluded to by “No suitable image found” – although reattempts are constantly being made to locate the missing DLL
- Unfortunately, this debug output does not tell us exactly which DLL we require – so we turn to another useful tool (elftran)
- At this stage, it would be a good idea to halt the QEMU session, to avoid some rather confusing behaviour that arises from the handling of the S:\ drive, and executable caching when copying additional DLLs and executables to \sys\bin within it
Now, we should be on the home stretch:
- Within another cmd.exe session, run: elftran \svphostfs\sys\bin\euserhl.dll > EUserHLDeps.txt to create a file containing information related to the dependencies of the euserhl.dll binary, amongst other attributes
- Searching for “imports from” within the EUserHLDeps.txt file will reveal that euserhl.dll depends upon estor.dll – which doesn’t exist in the prebuilt ROM image, or in \svphostfs\sys\bin
- Copy \epoc32\release\armv5\urel\estor.dll to \svphostfs\sys\bin
- Run arm-none-symbianelf-qemu-system -M \sf\adaptation\qemu\baseport\syborg\syborg.dtb -kernel \epoc32\rom\syborg.img once again
- With a little luck, attempting to invoke euserhl_walkthrough.exe should result in:
- Have fun exploring the serial debugging and tracing tracing functionality, and the host-bound file system
Partial output from running the EUserHL Walkthrough test suite
If that’s not enough, you can even try to build these components into a ROM image:
- Copy \sf\os\kernelhwsrv\kernel\eka\rombuild\tshell.oby to \sf\os\kernelhwsrv\kernel\eka\rombuild\tshellwitheuserhl.oby
-
Add the following lines to the bottom of \sf\os\kernelhwsrv\kernel\eka\rombuild\tshellwitheuserhl.oby:file=\epoc32\release\armv5\udeb\estor.dll \sys\bin\estor.dllfile=\epoc32\release\armv5\udeb\euserhl_walkthrough.exe\sys\bin\euserhl_walkthrough.exefile=\epoc32\release\armv5\udeb\euserhl.dll \sys\bin\euserhl.dll
- Delete euserhl_walkthrough.exe, euserhl.dll and estor.dll from \svphostfs\sys\bin\ to ensure that the contents of the ROM are not being supplemented/overridden with the contents of S:\sys\bin on the guest at boot time
- Within a cmd.exe session, switch to \sf\os\kernelhwsrv\kernel\eka\rombuild
- Run rom –variant=syborg –inst=armv5 –build=udeb –type=tshellwitheuserhl –name=\epoc32\rom\tshellwitheuserhl.img to attempt to build a new ROM image containing the EUserHL DLL, walkthrough/demo application, and the EStor DLL
- If everything worked according to plan, you should be able to run your new ROM image using: arm-none-symbianelf-qemu-system -M \sf\adaptation\qemu\baseport\syborg\syborg.dtb -kernel \epoc32\rom\tshellwitheuserhl.img within the \symbian-qemu-0.9.1\bin\ directory
- Invoking euserhl_walkthrough.exe for the final time should produce the same result as the previous exercise!
Additional Resources and Information
- A series of forum threads related to PIPS and networking, and the legacy, development-specific TechView GUI, in addition to the aforementioned EUserHL thread that I started, and continue to contribute to
- A list of backlog items and planned features for the Symbian Virtual Platform are available, here
- Johnathan’s presentation about the Symbian Virtual Platform at SEE 2009:
Very informative, how far away is booting to a UI? It would be interesting to hear how long it takes to boot a ‘full fledged’ OS environment.
Hi tl,
At present, it’s capable of booting a full S60 5th Edition GUI, and Qt presumably works (if the presentation above is anything to go by), although not all of the necessary source code, components and infrastructure have been supplied to the general public by Symbian (yet).
Unfortunately, since I’m neither a member of a Symbian Foundation member company, or an employee of the Foundation, I also don’t have access to the aforementioned components.
However, a handful of prebuilt S60 components are supplied in recent PDKs, and I’ve contemplated attempting to test those, although I haven’t done anything with them, yet.
I would have been happy enough with just TechView, to be honest – although because certain core S60 components conflict with those supplied by TechView, it isn’t supplied in binary form at all to the public, to the best of my knowledge.
Of course, even if a working GUI was available, you’re still limited in what you can do with it, since the TCP/IP stack isn’t fully functional (or easy to configure in a user-friendly manner, from what I’ve discovered so far), and audio support isn’t quite ready.
Additional work would probably also be necessary, before a GSM/UMTS radio or some other device supporting Hayes/AT Commands can be used, although the ModemAdaptation source code repository has been opened, and QEMU itself has serial port support – as mentioned in the post itself.
If there’s anything else you want to know, just ask, and I’ll see what I can do.
Tyson.
Update: I’ve just rediscovered the video at http://developer.symbian.org/wiki/index.php/SEE_2009_Presentations/Demo_Labs/Videos#QEMU_Simulator_.26_Symbian_Foundation, after some digging around.
Hopefully, the very brief demo of S60 ROMs, with and without Qt should give an idea of how long it takes to boot with a full interactive S60 GUI, and how long it takes to launch Qt-based applications from within.
Naturally, it could do with some performance tuning, and better graphics acceleration support overall, before the experience is as fluid as a real handset, although it isn’t exactly a slouch from what I’ve seen.
Thanks, that video was just what I was looking for.
There’s no way to actually build a full graphics ROM for Qemu right now without full access to the source.
There are key components missing from the build.
Foremost, every adaptation needs a screendriver, a DLL which plugs in to the bottom of bitgdi.
Then, there’s the whole NGA issue.
Incidentally, using buildrom makes the whole determining and satisfying dependencies about a million times easier.
[...] rendering…oh dear) then you need to step outside the emulator’s sandbox. For the Symbian Virtual Platform, that means we need to tunnel the drawing operations through a virtual peripheral to a translation [...]
Hello. Please excuse me because i’m not a developer, but a unhappy user, that spent a lot of money in a supossed top-of-the-notch e90 communicator.
With all this thing of the symbian kernel released, how far is the day that someone could build a rom with a updated OS for any current symbian based phone?
I’ve seen that in linux world there are an active community in creating the perfect open OS for mobile devices.
Do you think this will happen with symbian?
Thanks. Your talk is purely ‘developer / hacker stuff’, but maybe with your knowledge you can have a greater vision of the things to come than me, just a simple user.
Hi Claudio,
Whilst I can’t speak for the Foundation officially, I believe that efforts are being made to allow for upgrading individual Platform components in a piecemeal manner, via in-place upgrades and firmware delta patches. There’s also a general consensus that a low-cost “development phone” in some form would be nice to have. Naturally, I agree that it would be nice to see some flexibility in letting users modify and upgrade their device firmware though, in the spirit of openness and freedom to tinker.
Technically, it’s possible for anyone to create new ROM images, although deploying them on current “off-the-shelf” devices is difficult; since “traditional” device manufacturers tend to keep hardware schematics, baseport code, and user-level applications and enhancements proprietary. OEMs also tend to sign and/or encrypt delta patches and full images with private keys that are never externally shared, in order to control their usage, and provide a predictable, “trusted” (subjective, dependent upon party) boot sequence, too.
Of course, because said device manufacturers have access to cryptographic keys, and device-specific materials; they’re capable of building new firmware images or firmware update deltas. However, preparing them is expensive, and time-consuming, since it involves locating device-specific components and documentation, and people familiar with the architecture of a device. Not to mention that it’s then necessary to re-integrate and test both new, and old components. In the case of operator-branded devices, approval from the operator is usually necessary before they can be deployed.
There’s also the not-insignificant “problem” from the OEM perspective of keeping customers on the hardware replacement treadmill as far as features are concerned, given that firmware upgrades have their limits. (Not that I agree with the planned obsolescence philosophy).
Historically, It didn’t help that Symbian Software Ltd. (related to the Foundation by name, staff members – in a few cases, and codebase only) used to charge OEMs for the privilege of licensing new major versions of Symbian OS, as far as I know; which reduced the incentive for providing firmware upgrades, and indirectly created the current culture of “If you want a new version of the Platform, buy an entirely new device”. Likewise, there’s also a culture around licensing proprietary, third-party components and technologies that has persisted in both the Symbian, and general device development communities for a long time – which has only recently started to cede to the current trends towards open hardware and software platforms.
As they say – you can’t easily change a habit of a lifetime…
Sorry if that was excessively long, and difficult to follow; although there’s a lot involved technically, and politically in this stuff. I don’t know, or even pretend to know everything about what’s involved (and I may even be incorrect in some cases), although I hope that’s provided something to ponder.
Tyson.