update 27.10.2018 – use TWRP instead of CWM (discontinued)
update 14.10.2017 – new instructions to set-up udev rules
update 26.02.2016 – instructions for Android 6 Marshmallow

The Big Picture

Android consists of three parts relevant to rooting

1. the bootloader
2. recovery system
3. main system

typically only the main system is running, that is the Linux Kernel, the launcher, the phone app etc.. If we talk about rooting, that means we want to add an additional app to the main system which has access to secured parts of the system and acts as a gatekeeper for other apps that also want to get access.

The problem is the secured parts of the system are locked down – otherwise they would not be secure. This means that we can not simply install that app (e.g. an apk) from within the main system.

Therefore we have to go one level down. This is where the recovery system is. Typically you do not see it, as it is only active when the main system can not run – either because a system update is installed or because you do a factory reset.
As the recovery system can do a full system update, it means that it has also access to the secured parts of the main system – exactly what we need.
The stock recovery system obviously does not allow altering the main system – otherwise everybody could get your private data if you lose your phone.
So we need to replace it as well. But before that we have to talk about the bootloader.

The bootloader is a tiny piece of software which decides whether to start the recovery or the main system (or another main system, like Ubuntu Phone).
In the default configuration in only starts systems that it knows and trusts. In this configuration the bootloader is called locked.
Although this prevents malicious software to change the phone and spy on us, it also prevents us from replacing the recovery system. By the way, this concept is also coming to the PC where it is called UEFI secure-boot.

Here is a graphical overview of the Android components:

So what we need to do in order to get root access is

1. unlock the bootloader
2. replace the recovery system
3. install a superuser app

Note that unlocking the bootloader also allows attackers to circumvent any of the android security features (PIN etc). It becomes possible to access all the files on the device using a different recovery system. (unless userdata is encrypted)
Therefore android will wipe all userdata when the bootloader state is changed from locked to unlocked.

So if you lose your unlocked device or it gets stolen, you better hope the thief is not tech savvy.

Preparations

First you need to install the fastboot binary to be able to perform low-level communication with the device

apt-get install android-tools-fastboot android-tools-adb android-sdk-platform-tools-common

The android-sdk-platform-tools-common package most importantly contains a whitelist (/lib/udev/rules.d/51-android.rules) with devices to which users can send commands over USB, so you do not have to run fastboot as root.

Now you have to reboot into fastboot mode. Usually there is a key combination you have to press on startup.

• Key combinations for Google Nexus Devices
• Key combinations for Sony Xperia Devices

Remember this key combination as you will need some more times.

Samsung Devices however, like the Galaxy S3, do not support the fastboot mode – instead they have a download mode, which uses a proprietary Samsung protocol. To flash those you have to use the Heimdall tool. While this article does not cover the heimdall CLI calls, the general discussion still applies.

Unlocking the Bootloader

last warning: this will wipe all user data on the device

for google devices, like a Nexus 4 or Nexus 7 it is just do

fastboot oem unlock

if you have a Sony Xperia device, like a Xperia Z, you additionally have to request a unlock key and then do

fastboot oem unlock 0x<KEY>

where <KEY> is the key you obtained.

Using AutoRoot to install SuperSU

There are several superuser apps to choose from for Android 4 and below. However the only superuser app working on Android 5/ Lollipop and above is SuperSU by Chainfire.

As there are devices like the Nexus 5X shipping with Android 6/ Marshmallow, I will describe this method first.

Chainfire created an “installer” called AutoRoot that includes the fastboot utility and will perform the unlocking step described above. However if you have read this far, you probably also want to understand the rest of the process.

First you have to download the appropriate package for your device. There you will find a recovery image which we have start with

fastboot boot image/CF-Auto-Root-hammerhead-hammerhead-nexus5.img

the command above will not flash anything on your device, but just upload the image and immediately start it. The image contains a script to modify the main system (change startup to get around SELinux) and install the superuser app.

If everything goes well, you can now just reboot your phone and you are done.

You could lock your bootloader again now to make your device more secure. However the next Android update will remove root again and repeating the rooting procedure will wipe userdata – so you have to balance security update vs. the risk of your device being stolen. For the latter case you still have the option to enable encryption of userdata though.

Installing OTA updates

Android over the air (OTA) updates contain only the changes to the current system. In order to verify that the update succeeded Android computes a checksum of the patched system and reverts to the old state otherwise.

As SuperSU has changed the boot image to start itself, the updates obviously will fail. So to install an OTA update you will have to grab a factory image and restore the boot partition using the included boot.img

fastboot flash boot boot.img

after this you will have to patch the boot partition again using the procedure described above.

Also note that if you use apps that change the system partition (like AdAway that changes the hosts file), you will have to revert those changes as well in order for the OTA update to succeed.

Optional: Replacing the Recovery System

If you want some advanced features, like backing up all your installed apks, you can permanently replace the recovery image on your device. However this will most likely prevent you from installing OTA updates.
There are two prominent alternative recovery systems with the ability to install apps

• Team Win Recovery Project
• Clock Work Mod

Clock Work Mod has been discontinued, so we will use TWRP. From the Website linked above download the recovery image which fits your phone.

fastboot flash recovery <RECOVERY>.img

where <RECOVERY> is the name of the file you downloaded. For instance for a Nexus 9 and TWRP 3.2.3 it would be

fastboot flash recovery twrp-3.2.3-0-flounder.img

restoring stock recovery

If you have a Google Device, you can grab the factory images here.  There you will find a image of the stock recovery. You can restore it by

fastboot flash recovery recovery.img

Alternative superuser apps

If you run a device with Android older than 5/ Lollipop you have some alternatives to SuperSU:

• Superuser by ChainsDD
• Superuser by CWM

I would recommend getting Superuser by CWM, as it is open source and also nag-free as there is no “pro” version of it. There is even a pull-request which might make it also work with Android 5 in the future.

To install the app we need to get this zip archive and copy it to the device. Then we need to reboot into fastboot mode and then select “Recovery Mode” to get to the recovery system. Once in Recovery mode select

install zip -> choose zip from /sdcard

then browse and select the “superuser.zip” you just copied.

Once installed select

Go Back -> reboot system now

Once the system has started you should have a “Superuser” App on your device. Congratulations, you are done.

I recently ran into this problem and could not find any good solution on the Internet. So next comes a small summary of the problem with hopefully enough buzzwords, so Google can lead you here.

If you want to do C++ development on Android, you need the NDK for cross compilation. It comes by default with its own build system called ndk-build, which basically is a bunch of custom makefiles. But if you are sharing code between the Android Platform and lets say plain Linux, you have likely already a build system installed. For C/C++ CMake is quite popular as it supports different platforms and compilers. Fortunately there is already a project which adds Android support to CMake. I will not cover that – instead I assume you are using it already.

Unfortunately you cant use the ndk-gdb script supplied with the NDK to debug your application as it relies on the behaviour of ndk-build. But as said earlier, ndk-build is no wizardy, but just a bunch of scripts. So it is possible to emulate the behaviour using CMake, as following:

Add the following macro to your CMakeLists.txt file

macro(ndk_gdb_debuggable TARGET_NAME)
    get_property(TARGET_LOCATION TARGET ${TARGET_NAME} PROPERTY LOCATION)
    
    # create custom target that depends on the real target so it gets executed afterwards
    add_custom_target(NDK_GDB ALL) 
    add_dependencies(NDK_GDB ${TARGET_NAME})
    
    set(GDB_SOLIB_PATH ${PROJECT_SOURCE_DIR}/obj/local/${ANDROID_NDK_ABI_NAME}/)
    
    # 1. generate essential Android Makefiles
    file(WRITE ${PROJECT_SOURCE_DIR}/jni/Android.mk "APP_ABI := ${ANDROID_NDK_ABI_NAME}\n")
    file(WRITE ${PROJECT_SOURCE_DIR}/jni/Application.mk "APP_ABI := ${ANDROID_NDK_ABI_NAME}\n")

    # 2. generate gdb.setup
    get_directory_property(PROJECT_INCLUDES DIRECTORY ${PROJECT_SOURCE_DIR} INCLUDE_DIRECTORIES)
    string(REGEX REPLACE ";" " " PROJECT_INCLUDES "${PROJECT_INCLUDES}")
    file(WRITE ${PROJECT_SOURCE_DIR}/libs/${ANDROID_NDK_ABI_NAME}/gdb.setup "set solib-search-path ${GDB_SOLIB_PATH}\n")
    file(APPEND ${PROJECT_SOURCE_DIR}/libs/${ANDROID_NDK_ABI_NAME}/gdb.setup "directory ${PROJECT_INCLUDES}\n")

    # 3. copy gdbserver executable
    file(COPY ${ANDROID_NDK}/prebuilt/android-arm/gdbserver/gdbserver DESTINATION ${PROJECT_SOURCE_DIR}/libs/${ANDROID_NDK_ABI_NAME}/)

    # 4. copy lib to obj
    add_custom_command(TARGET NDK_GDB POST_BUILD COMMAND mkdir -p ${GDB_SOLIB_PATH})
    add_custom_command(TARGET NDK_GDB POST_BUILD COMMAND cp ${TARGET_LOCATION} ${GDB_SOLIB_PATH})

    # 5. strip symbols
    add_custom_command(TARGET NDK_GDB POST_BUILD COMMAND ${CMAKE_STRIP} ${TARGET_LOCATION})
endmacro()

Then use it like

add_library(YourTarget ...)
ndk_gdb_debuggable(YourTarget)

You should now be able to use ndk-gdb with CMake, just as if you would have used ndk-build.

Note that steps 4 and 5 are optional for debugging. They just reduce the size of the library that has to be transferred to the device. If you dont care, you can just leave them out. But then the solib search path from step 2 must be set to:

file(WRITE ./libs/${ANDROID_NDK_ABI_NAME}/gdb.setup "set solib-search-path ./libs/${ANDROID_NDK_ABI_NAME}\n")

Ideally someone should integrate that in the Android toolchain linked above.

Update Merged Upstream

When I first read about GNOME dropping support for BSD and Solaris, my impression was that this is a good idea to aiming to unify limit resources and get the work done. I was also excited about the idea of the GNOME OS. I think it is necessary to keep the big picture in mind when developing the different components. Previously Ubuntu was the only project that did this and it was also the reason why I started using Ubuntu. Because it made the different parts of Linux work together to achieve the big goal of a great overall system.

But then things started to go wrong. Instead of picking existing components and giving them the final polish like Ubuntu did before, the GNOME project started developing things from scratch without any apparent reason to do so. And even worse: incompatible to existing solutions. It started with the rejection of the appindicator specification implemented by Ubuntu and KDE. At that point it was not clear to me whether the specification was broken or whether the responsible people at GNOME were just ignorant.

Then came systemd. And it started to be apparent that unfortunately it was the latter. To my knowledge Ubuntu is the biggest deployment of GNOME and it is based around the Linux ecosystem. So dropping support for Ubuntu has nothing to do with unifying limited resources. Ubuntu is your target audience, so if you should try to collaborate with a project you should collaborate with Ubuntu. My opinion on that is that some Fedora developers were pissed that the Unity interface was exclusive for Ubuntu and instead of packaging it for Fedora they started making GNOME Shell exclusive for Fedora.

Next I read about the overlay scrollbars re-developed for GNOME. While the first reaction might be the developers simply do not want to use Ubuntu technology, I think the reason is different. The developer does not seem to have any antipathy towards Ubuntu and if we look at the project he developed the scrollbars for another explanation becomes visible.

But first lets take a step back. Lets take a look at the core of GNOME. By this I mean the programming language it is written in. It is C/GObject; plain C extended with naming conventions and libraries to allow modern paradigms such as object oriented programming and events/ observer pattern. From today’s perspective one might wonder why one should choose this over C++, which integrates most of the features at the language level. But back when the GNOME project started C++ was not mature yet which meant that your program might break with the next compiler update or even the next STL update.

Therefore basing your project on plain C was a good idea. But a few years back it became obvious that programming in C/GObject seriosly lacked behind more modern programming languages like C++, Java and C# for application development.

Unfortunately instead of moving the straightforward route from C to C++, which most of C developers took when C++ matured(that was about 10 years ago), Vala was born.

So instead of using a proven and mature foundation, a new layer of indirection was created to essentially provide the same feature set. Commonly this is referred to as the “not invented here” symptom. A more derogative phrase would be reinventing the wheel..

What is sad here is that being an open source project, GNOME disregards the biggest advantage of open source software, namely standing on the shoulders of giants. With open source software you can use take an existing solution and improve upon it. This way you get the base functionality as well as the bug fixes that went in it for free. If you would develop it from scratch, you most likely would have to fix the same bugs again yourself.

To sum up here is what GNOME is losing right now

• 30 years of language and library experience by using Vala instead of C++
• 5 years of deployment and bug fixing by using systemd instead of extending upstart
• 1 year of development testing and design if they reimplement overlay-scrollbars
• 8 years of foundation development that went into Eclipse, by developing Gnome Builder from scratch
• but most importantly: the synergy effects by collaborating with others

Do not get me wrong, I am not saying that the GNOME solutions could be replaced by existing solutions – I am saying that by extending existing solutions the GNOME project and the free software landscape would be better off as a whole.

I just have watched the Apple iPad announcement and I have to say that I am quite impressed by the Apple marketing team. Before the film I could not think of a good use case for a oversized iPod, but after the film I have to say that Apple greatly refined the use case of the netbooks as a second PC.

Instead of putting an ordinary OS into a differently shaped device, like Microsoft is seemingly doing with the Slate, Apple adjusted the OS to the new use case.

If you have a much smaller screen and a much smaller keyboard, like you have on netbooks, you don’t want to write long articles or aim for the tiny buttons of ordinary user interfaces. Instead one should of a netbook like a playback device, which only requires rudimentary interaction.

As apple is great at streamlining stuff, they simply left out the keyboard and used a modified version of the iPhone OS, which is optimized for easy usage and – voilla here comes the computer you actually want to use in your living room, to quickly peek on facebook or your mail inbox.

But there are two big disadvantages that come with using the iPhone OS. First it is stripped down to much; there is no multitasking and no system clipboard which takes a lot of the convenience you have when using a real OS.

And second you are again locked-in by apple. If you use the iPad, you are also more or less forced to use iTunes for your music, iBook Store for your eBook and the AppStore if you want new Software.

Of course you might be able to Jailbreak the device and use third-party software but this will be nowhere as convenient as using the defaults. This is Apples Achilles heel and where Maemo can triumph.

With Maemo you basically have a full-fledged Linux with a easy to use UI. You have multitasking, you have a system clipboard and most importantly you have an open software repository – and all of this very well integrated in the UI.

You can freely choose your email provide, your music player and even the format you save your music in. And even though Nokia does not support OGG by default, the open nature of the OS allows it to be just as integrated as everything else.

Actually Nokia only has to build a Internet Tablet with the size of the iPad…