Augmented Reality on the N900

finally I reached a stage where I could upload my small augmented reality app to extras-devel, so all those who asked for it can now play with it. But be aware that it is in extras-devel for a reason. In case you are wondering what I am writing about, here is a video of the demo:

in order to make it work, you will have to print the artoolkitplus markers. Furthermore there are these controls:

  • scale the objects using the volume buttons
  • select one of the objects for scaling by tapping on it
  • tapping on the palette symbol triggers annotating by drawing on the screen
  • tapping on the sun symbol fixes the sun to the current device position
  • once fixed the shadows can be rotated using the arrow keys

Gallium3D is taking over

after there was a major performance regression with the introduction of kernel mode setting(KMS) for the radeon driver, it looks like the change starts to pay off now. You can already find on the fly display reconfiguration and dynamic power management with the upcoming kernel 2.6.34.

But if you enable the Gallium3D based r300g driver 3D performance takes off too – and makes you forget fglrx. While the Gallium3D based driver only had OpenGL2 to offer but had a horrible performance, it now easily overtakes the performance the classical r300 driver had with user mode setting.(UMS)

The last major boost came along with ColorTiling support which greatly improves the memory access, here a fast glxgears benchmark:

  • 10208 frames in 5s (no kms + classic mesa)
  • 9200 frames in 5s (kms + classic mesa)
  • 13400 frames in 5s (kms + gallium driver)

although this gives you a clue, the performance gain is much bigger in something that actually uses textures – like openarena.

Handheld based interaction using AR

it is time for the next demo of my project, as I reached the beta status. (feature complete) I think you can see quite clearly now where this is going and which kind of interactions will be possible using this technique. The concrete features are described in the annotations.

If you use more advanced tracking methods and add some physics to this, you could easily port numpty physics to this kind of interaction or create an easy to use level editor. In case you missed my last video, here is the link.

It will still take some weeks until this hits maemo-extras as there are still some bugs left and I still want to get rid of keyboard use for interaction.

Thoughts about MeeGo

In a country with freedom of speech, one has to say something to every happening, right? So here is my try:

Basically the merge of Maemo and Moblin is logical and consequent as Nokia and Intel already collaborated with ofono and merging helps joining the efforts. This is quite necessary as neither Maemo nor Moblin could survive on their own in a world where everybody else uses Android and soon will start using Chrome OS. There was also a need to make Moblin more like Meamo to be able to compete with the iPad.

That sounds greet so far, right? Joint efforts, bigger community, open to everybody… But the problem is the way the merge is going to happen. Moblin is more or less a huge techdemo so far – everybody who I know uses Ubuntu Netbook Remix on their Netbook, as it is more production ready and end user oriented. The same also applies to Maemo.

It is a bit sad that the next Maemo/ MeeGo Harmattan will be Qt based though, at it means that all the currently working applications have to be rewritten without gaining an immediate benefit. But considering that Qt is technically more advanced than GTK and that it allows deploying your application on the different OS Nokia uses this is understandable.

What is less understandable is that MeeGo will be based on RPM/ Moblin/ Fedora. And at least for Fedora the official motto is merging new features as fast possible, which is nice for developers but less nice for end users, as the distribution is less stable. So while it is logical to base a tech-demo like Moblin on Fedora, I would not base anything that is supposed to be stable on it.

But this is exactly what shall happen with MeeGo. This means Maemo has to abandon its Debian roots and rebase everything to RPM. By everything I mean the huge amount of packaging experience gained during the last 5 years, the build infrastructure and of course the core package management applications. This has also an impact on the community infrastructure, downloads, karma are coupled to the DEB format too.

So what do we gain by rebasing to RPM? Maybe the Moblin interface which is indeed nice? Actually no, as it is Clutter/GTK based while MeeGo will use Qt – besides the Moblin interface was packaged by Ubuntu as deb too. Ok the Moblin community will not need to change its infrastructure, but is the Moblin community actually that big?

As I really wonder why we switch to RPM I started a wiki to collect the arguments, and as it does not look too good for RPM also a brainstorm vote.

Power management with radeon on Ubuntu

As you might have already heard, lucid will introduce kernel modesetting(KMS) for most chipsets including intel, nvidia and ati. But at least for ati there is are drawbacks compared with the latest user mode setting(UMS) code. With UMS you could downclock your graphics which was quite handy when using a laptop. But the code shipped with lucid allows no power management at all although power management for KMS is already available in drm-next.

To continue the good news, the KMS power management is already more advanced than the UMS code used to be as it can change the engine clock, the memory clock and the number of PCIE lanes on the fly – depending on the current load.

If you want to try this out for yourself, you have to get an updated kernel from here.(this build tracks drm-radeon-testing) And then load the radeon module with dynpm=1. To just try this for one boot you can append radeon.dynpm=1 to your grub boot line. To make it permanent put it in a new file in /etc/modprobe.d/.

If it succeeds you should see something like this in your dmesg output: (engine, memory, pcie lanes)

[ 7827.808061] [drm] Requested: e: 43200 m: 33000 p: 16
[ 7828.012066] [drm] Setting: e: 43200 m: 33000 p: 16
[ 7828.508058] [drm] Requested: e: 32400 m: 13500 p: 1
[ 7828.708094] [drm] Setting: e: 32400 m: 13500 p: 1

Of course this things are not stable yet, as they are not even contained in Linux 2.6.33, but I think they are supposed to go in for 2.6.34 and it works here quite well 🙂

AR shadowmapping demo

while my last video already contained augmented reality and shadow mapping, it did not really show the potential of shadowmapping, therefore I created another video

other news are the tracking of multiple individual markers(quite obvious) and the camera relative light source. The latter is necessary to cast the shadow of all objects in the same direction.

As some of you wondered where the sense behind all of this is; I write the program as a project work at the university. The aim is to create an easy way to create and modify 3D scenes.

Tablet PCs – a chance for Maemo?

I just have watched the Apple iPad announcement and I have to say that I am quite impressed by the Apple marketing team. Before the film I could not think of a good use case for a oversized iPod, but after the film I have to say that Apple greatly refined the use case of the netbooks as a second PC.

Instead of putting an ordinary OS into a differently shaped device, like Microsoft is seemingly doing with the Slate, Apple adjusted the OS to the new use case.

If you have a much smaller screen and a much smaller keyboard, like you have on netbooks, you don’t want to write long articles or aim for the tiny buttons of ordinary user interfaces. Instead one should of a netbook like a playback device, which only requires rudimentary interaction.

As apple is great at streamlining stuff, they simply left out the keyboard and used a modified version of the iPhone OS, which is optimized for easy usage and – voilla here comes the computer you actually want to use in your living room, to quickly peek on facebook or your mail inbox.

But there are two big disadvantages that come with using the iPhone OS. First it is stripped down to much; there is no multitasking and no system clipboard which takes a lot of the convenience you have when using a real OS.

And second you are again locked-in by apple. If you use the iPad, you are also more or less forced to use iTunes for your music, iBook Store for your eBook and the AppStore if you want new Software.

Of course you might be able to Jailbreak the device and use third-party software but this will be nowhere as convenient as using the defaults. This is Apples Achilles heel and where Maemo can triumph.

With Maemo you basically have a full-fledged Linux with a easy to use UI. You have multitasking, you have a system clipboard and most importantly you have an open software repository – and all of this very well integrated in the UI.

You can freely choose your email provide, your music player and even the format you save your music in. And even though Nokia does not support OGG by default, the open nature of the OS allows it to be just as integrated as everything else.

Actually Nokia only has to build a Internet Tablet with the size of the iPad…

ArtoolkitPlus for Maemo

I just uploaded the latest version of ArtoolkitPlus to Frementle devel, so you can start playing around with it soon. Artoolkit is a marker tracking library which allows Augmented Reality applications similar to ARhrrr possible. I am currently working on a AR application for the N900 as part of a project work for the university. Of course it is not quite as advanced as ARhrrr, but it certainly runs better than this demo on the iPhone 🙂

PowerVR SGX 530 does NOT support Depth Textures

If you download the PowerVR SDK for the TI OMAP3430 which is used in the N900, you will find a nice shadow mapping example which uses the OES_depth_texture extension. You can even run this example using the included emulator using the SGX 530 profile.

However if you try to run your code on the actual device you will soon find out that it does NOT support OES_depth_texture. This is quite surprising as the device supports OES_texture_float and OES_depth24, so depth_texture should be not too hard to implement.

But the situation is the same on the iPhone 3GS, which uses the same hardware. This leads me to believe that this is not just a driver limitation.

However here is a workaround using a ordinary RGBA texture and byte packing:

const highp vec4 packFactors = vec4(256.0 * 256.0 * 256.0, 256.0 * 256.0, 256.0, 1.0);
const highp vec4 cutoffMask  = vec4(0.0, 1.0/256.0, 1.0/256.0, 1.0/256.0);

void main() {
    highp vec4 packedVal = vec4(fract(packFactors*gl_FragCoord.z));
    gl_FragColor = packedVal - packedVal.xxyz*cutoffMask;
}

the packFactors vector basically contains the 3 necessary byte wise shifts to the left. The call to fract() cuts off anythig which is to the left of the byte we want to save and subtracting packedVal.xxyz*cutoffMask cuts off anything to the right of the byte.

The cutting is necessary as we are dealing with floating-point here and we dont know how the hardware selects the value that should go in.