Monday, September 05, 2011

Android ARM Assembly: A trivial program (Part 2)

This is part two of a series on learning ARM assembly on Android. This part covers a walkthrough of a ARM assembly program.

Part 1: Motivation and device set up
=> Part 2: A walk-through of a simple ARM assembly program
Part 3: Registers, memory, and addressing modes
Part 4: Gnu tools for assembly; GCC and GDB
Part 5: Stack and Functions
Part 6: Arithmetic and Logical Expressions
Part 7: Conditional Execution
Part 8: Assembly in Android code

The articles follow in series, each article builds on the previous.

Hello world assembly program

The easiest program that most languages introduce is the Hello World program. If you had followed with Part one, you typed the Hello World program in C. Now generate the assembly language for that example using the command given at the end of part one.

Here is the assembly language program in full:
 1 	.cpu arm9tdmi
 2 	.fpu softvfp
 3 	.eabi_attribute 20, 1
 4 	.eabi_attribute 21, 1
 5 	.eabi_attribute 23, 3
 6 	.eabi_attribute 24, 1
 7 	.eabi_attribute 25, 1
 8 	.eabi_attribute 26, 2
 9 	.eabi_attribute 30, 6
10 	.eabi_attribute 18, 4
11 	.file	"hello.c"
12 	.section	.rodata
13 	.align	2
14 .LC0:
15 	.ascii	"Hello World\000"
16 	.text
17 	.align	2
18 	.global	main
19 	.type	main, %function
20 main:
21 	@ Function supports interworking.
22 	@ args = 0, pretend = 0, frame = 8
23 	@ frame_needed = 1, uses_anonymous_args = 0
24 	mov	ip, sp
25 	stmfd	sp!, {fp, ip, lr, pc}
26 	sub	fp, ip, #4
27 	sub	sp, sp, #8
28 	str	r0, [fp, #-16]
29 	str	r1, [fp, #-20]
30 	ldr	r0, .L3
31 	bl	puts
32 	mov	r3, #0
33 	mov	r0, r3
34 	sub	sp, fp, #12
35 	ldmfd	sp, {fp, sp, lr}
36 	bx	lr
37 .L4:
38 	.align	2
39 .L3:
40 	.word	.LC0
41 	.size	main, .-main
42 	.ident	"GCC: (Debian 4.3.2-1.1) 4.3.2"
43 	.section	.note.GNU-stack,"",%progbits
As you can see, it is quite a lot of code for a simple program. Luckily, most of it is boilerplate.  Let's break it down piece by piece.

Declaration and options
 1 	.cpu arm9tdmi
 2 	.fpu softvfp
 3 	.eabi_attribute 20, 1
 4 	.eabi_attribute 21, 1
 5 	.eabi_attribute 23, 3
 6 	.eabi_attribute 24, 1
 7 	.eabi_attribute 25, 1
 8 	.eabi_attribute 26, 2
 9 	.eabi_attribute 30, 6
10 	.eabi_attribute 18, 4
11 	.file	"hello.c"
The first eleven lines are declarations of various options in the ARM cpu. You can ignore them for now. In case you are curious, we specify the CPU type, the way we want the Floating Point Unit (FPU) to operate, and then specify options for the ARM Embedded Application Binary Interface (EABI). The filename is specified on line 11.

Declaring constants
12 	.section	.rodata
13 	.align	2
14 .LC0:
15 	.ascii	"Hello World\000"
The string "Hello World" is specified as a constant in assembly on lines 12-15. It is in the Read Only DATA section (.section .rodata), it needs to be aligned on two-byte boundaries, and the string is specified as an ASCII string. Byte alignment is very important in assembly language programming.  You must pay careful attention to data that needs to be word aligned (2-byte), quad aligned (4-byte) or byte aligned (1-byte). In general, you can't go wrong with word alignment, so if you are uncertain, add a .align 2 at the top of data and functions.
The data is specified as a string in assembly, though the assembler writes it out as bytes behind the scenes.

Declaring functions
16 	.text
17 	.align	2
18 	.global	main
19 	.type	main, %function
20 main:
The program consists of a single function called main. Program code is always in the text section, thus the declaration on line 16. It is word aligned (line 17). main is a global variable, and line 18 allows it to be visible elsewhere in the program. Finally, it is listed as a function, and line 20 is a label containing the name 'main'.

Registers and data
Before we look at the function contents, it might be useful to know what the ARM architecture is like. ARM processors are an example of Reduced Instruction Set Computing (RISC). This means that there are few instructions, and most instructions operate on registers. For user programs, there are 16 registers, called r0, r1, r2, .., r15. Each register is 32 bits long. Registers correspond roughly to variables, though they don't have a data type. Most arithmetical and logical operations are performed on registers. The processor can also access the entire memory using load and store instructions.

The registers r12-r15 are special:
r12: IP, or Intra-Procedure call stack register. This register is used by the linker as a scratch register between procedure calls. A procedure must not modify its value on return. This register isn't used by Linux gcc or glibc, but another system might.
r13: SP, or Stack Pointer. This register points to the top of the stack. The stack is area of memory used for local function-specific storage. This storage is reclaimed when the function returns. To allocate space on the stack, we subtract from the stack register. To allocate one 32-bit value, we subtract 4 from the stack pointer.
r14: LR, or Link Register. This register holds the return value of a subroutine. When a subroutine is called, the LR is filled with the program counter.
r15: PC, or Program Counter. This register holds the address of memory that is currently being executed.

Here are some data move instructions:
24 	mov	ip, sp
This moves the value from sp (r13) to ip (r12). This achieves ip = sp.
32 	mov	r3, #0
This moves the value 0 into r3. This achieves r3 = 0
 In addition to moving values within registers, you can load values from memory into registers, and store registers into memory. Here are instructions that achieve this:
28 	str	r0, [fp, #-16]
This stores the contents of r0 into the memory location pointed to by (fp -16). Since memory is addressed by bytes, and registers are 4 bytes each, memory offsets are often multiples of 4. This is the same as the C statement *(fp - 4) = r0
30 	ldr	r0, [fp]
This loads the data from memory pointed to by register fp into register r0. This is the same as the C statement r0 = *(fp)
25 	stmfd	sp!, {fp, ip, lr, pc}
This is a multi-register move operation. This moves the registers FP,IP,LR,PC into the area specified by the register SP. Since SP is the stack pointer, this is the same as pushing registers FP,IP,LR,PC to the top of the stack in a single operation. Once this is done, the stack pointer is updated since it has an exclamation mark.
35 	ldmfd	sp, {fp, sp, lr}
This is another multi-register move that undoes the action on line 25. This reads back the values that were written earlier, popping them from the stack. The combined effect of lines 25 and 35 is to store the important register values on the stack and then restore them. This allows the function to modify them in the main body. We don't restore IP

Manipulating data
Assembly instructions to manipulate data are very basic: you can do basic arithmetic operations: ADD, SUB, and basic logical operations: AND, OR. ARM Assembly language operations are of the type: OPERATION ARG1, ARG2, ARG3 This performs the operation ARG1 = ARG2 OPERATION ARG3. In the above program, you see some basic arithmetic.

26 	sub	fp, ip, #4
This performs the action fp = ip - 4.

Function calls and returns
Assembly language is written as a flat set of instructions: there is very little structure once the instructions are written to memory in the computer. In order to make code modular, functions can be written. Without the protection of the C compiler, assembly programs must manage their own function calling.
The basic function call involves the following structure:
 	.text
 	.align	2
 	.global	functionName
 	.type	functionName, %function
 functionName:
 	mov	ip, sp
 	stmfd	sp!, {fp, ip, lr, pc}
 	sub	fp, ip, #4  @ Space for local variables
 	sub	sp, sp, #8
 
 	sub	sp, fp, #12
 	ldmfd	sp, {fp, sp, lr}
 	bx	lr
Subroutines are required to preserve every register except for r0-r3. So if you need to use the other registers (r4 onwards), you should save them on the stack before over-writing them. The exact function call convention is listed in the ARM procedure call standard.
In order to call a function, you Branch and Link using the BL instruction. The return address is placed in the LR register. To return from a function, you call a branch on the LR register. This is a BX rather than a B to correctly move between ARM and Thumb instructions. (Ignore ARM and Thumb differences for now, they will be made clear later).

Hello World functionality
Put together, the lines 30-33 lines print out the message Hello ARM world. Let's break this down instruction-by-instruction to see how this is achieved.
30 	ldr	r0, .L3
31 	bl	puts
32 	mov	r3, #0
33 	mov	r0, r3
Line 30 loads the address of label .L3 into register r0. The function calling convention is that the first four arguments are stored in r0-r3, and subsequent arguments are stored on the stack. The function to put a value on the screen is called puts, and it accepts just one argument: the string to be printed. The address of this string is stored in the first register: r0.
Line 31 calls the puts function, which consumes r0 and prints the value on the screen. After calling the function, we can expect the registers r0-r3 to be trashed. The return value of the puts is in r0, but we don't care for it.

Line 32 and 33 put together achieve r0 = 0. This is the return value that the main method returns.

Final word
You are now capable of reading ARM assembly and understanding the main elements in the program. As an exercise, you could try reducing the size of the code while keeping the functionality intact.

 In the next article, we can examine each piece in some detail.


Android ARM assembly: Device set up (Part 1)

This is part one of a multipart series about learning ARM assembly programming with Android. This covers motivation and device set up.

=> Part 1: Motivation and device set up
Part 2: A walk-through of a simple ARM assembly program
Part 3: Registers, memory, and addressing modes
Part 4: Gnu tools for assembly; GCC and GDB
Part 5: Stack and Functions
Part 6: Arithmetic and Logical Expressions
Part 7: Conditional Execution
Part 8: Assembly in Android code

The articles follow in series, each article builds on the previous.

Motivation

You might not realise it, but ARM processors are ubiquitous. When I look around my house, I can count eight ARM processors in routers, phones, eBook readers and web servers. There are more ARM processors in my house than Intel processors.

ARM assembly language is perhaps the easiest assembly language in widespread use. The Intel instruction set was developed over years of CPU revisions, which made the Complex Instruction Set (CISC) even more complex. In case you want to learn assembly, the ARM instruction set is simple to learn. Once you know how to write ARM assembly, it is easier to learn Intel assembly.

Finally, ARM processors are being used everywhere. Whether you like the iPhone or Android, they both use ARM processors. ARM processors excel at low-power computing, which makes them valuable for mobile computing, and consumer electronic devices like routers, NAS storage, eBook readers, game consoles and cell phones. Unlike desktop computers, resources are very limited on mobile devices. On such devices optimisation makes the difference between a slow and unusable application and a fast and responsive one.

Thanks to the profusion of Android devices, you can start programming ARM assembly within minutes and at low cost. You don't need to sign an NDA or pay for development tools. If you have an Android phone and a computer, you already have everything you need. Assembly language programming will not require the buttons or the touch-screen. Any device with a working USB port and with root access will do.

Setting up an ARM device with Linux was a monumental effort: you had to buy a dev-kit which easily ran in the hundreds of dollars. Then you struggled to shoe-horn Linux onto the tiny device. With Android, you already get an ARM device running Linux. All the device drivers are in place. All you need is a Linux distribution that allows you to get the native development tools. This is considerably easier. It should take around an hour of effort.

At this point, you have a choice. You can either go with setting up your own device, which lets you develop on a real ARM computer. Or you can download a prebuilt QEMU image.

Choice A: Emulation
This is the easy choice. For this, you need a computer: Windows, Mac, Linux, all are good.

QEMU is a full CPU emulator that can emulate an ARM computer. It is capable of running the full Android stack, and is shipped with the Android SDK. QEMU is much easier than setting up Linux on Android. To start, download and install QEMU on your system and a utility to extract RAR files. I have prepared QEMU images with the full software development environment.

Download all three files [part 1], [part 2], [part 3], and then run this command:
$  unrar x arm-qemu.part1.rar
This will create a directory called ARM. You can enter that directory and run the command runme.sh. It will start up the virtual machine. The virtual machine is slow, so be patient. The administrator has username "root" and password "root".

Choice B: Using a real device
In this case you install Linux on your Android phone.

You need:
  1. Android phone with 2GB free space on the SD card
  2. Any computer with USB
Obtain an Android phone on which you can get root access. If you have a working Android phone, you can use that. Otherwise you can buy a used Android phone. Install Debian on it using Jay Freeman's instructions on getting Debian on an Android phone.

Another option is to buy an ARM device like this Zipit.  This is a harder option. Many devices have poor support, and installing Linux can be a herculean task. I would recommend sticking to an Android phone unless you have easy access to an ARM device and you know that you can install Linux on it easily.

Once you have Debian working, you need to create a user and install ssh to allow easy access to the device from your laptop or desktop:
phone$ sudo adduser # Create a normal user for dev purposes.
phone$ sudo apt-get install openssh-server
phone$ sudo /etc/init.d/ssh start
 
You now have a working Debian installation that you can access using SSH. To ssh into your device, you can use port forwarding with the Android Debug Bridge (adb) to forward the desktop's port 2222 to port 22 on the phone as follows:
desktop$ adb forward tcp:2222 tcp:22
Finally, you can login to your phone from your desktop as follows:
desktop$ ssh user@localhost -p 2222

Once the port forwarding is working correctly, you can also login from any computer on your network as follows:
desktop$ ssh user@mydesktop -p 2222
This allows you to connect to your ARM device from any computer in your house. You can punch holes in your firewall by forwarding ports from the firewall to port 2222 on your desktop's IP address, and be able to access your ARM device from anywhere in the world. Even if the screen has turned off, you will be able to ssh into it, edit, compile and debug programs.

Congratulations. You now have a completely silent ARM device which you can connect to from anywhere.

Once you have Debian installed on your device, you can get the development system using apt-get:
phone$ sudo apt-get install gcc gdb make vi emacs
Install the editor you prefer: vi or emacs. Now, your phone is a fully capable ARM dev environment.


Test the development tools
You can try a simple C program to verify the setup.
/* hello.c: Hello World program */
#include <stdio.h>

int main(int argc, char* argv[]){
  printf("Hello ARM World\n");
  return 0;
}

You can compile this and run this as follows:
phone$ gcc hello.c -o hello && ./hello 
If you have got this far, you can get a sneak peak at ARM assembly with the following command:
phone$ gcc hello.c -S && cat ./hello.s 
This shows you the intermediate Assembly language from the Hello world program.

Congratulations, you have turned your Android phone into a full-featured ARM dev kit. The next article will cover programming your device in Assembly language.


Monday, August 29, 2011

The Linux Programming Interface: a beautifully written technical book

I have been reading, "The Linux Programming Interface" recently. It covers everything about the Linux programming environment: the layout of process space in memory, threads, signals, sockets. Nearly everything you can think of while programming Linux is covered here. Go get it. Now.

It is a beautifully written book. The author, Michael Kerrisk, combines technical knowledge about Linux internals with a clear, precise writing style. This book is fun to read. I have used Linux for many years now, and I found the chapters both illuminating and delightful. At first, I picked it up to learn about memory layout in Linux. I found the writing style so good that I read on, finishing not just that chapter, but many others. I have been reading a chapter at a time at random since. Each chapter deals with a single topic (e.g. Threads) and consists of roughly 25 pages. This is an excellent organisation: it lets you completely understand one area at a time in manageable chunks. Its 64 chapters are spread across 1500 pages, giving it unprecedented breadth and depth.

Technical books are difficult to write. The subject matter can be dry and the terminology makes sentences verbose and difficult to parse. In addition, there is a problem of target audience. You can assume the reader knows too much, making the book inaccessible. Or you could assume too little, and require the reader to go through trivial material that they already know. In addition, technical books are often read with a purpose in mind. They need to answer the question, "How do I do X in Linux" in minimum time. Remarkably, this book does well along all these dimensions. It is a case study in pleasant, lucid writing.


If you do anything with Linux, and have ever programmed in Linux, get a copy now. Keep it on your shelf, and leaf through it when you are bored. You will enjoy the book, and learn more about Linux while doing so.

Tuesday, August 16, 2011

1 Watt webserver

I recently bought a Zipit 2 from an online store. It is a tiny device with an ARM CPU (XScale-PXA270) with 32MB of RAM, an SD card slot, and 802.11 networking. The device cost me $30 including shipping, which is a cheap price for the computing potential. My goal was to run Linux on it, and learn the ARM instruction set. There are many Linux distributions for the Z2, and I found Debian on the Z2 to be perfect. The device is small, light and fits in the pocket of my jeans. Its battery lasts four hours when the screen is on.

Installing Linux on it was trivial: you flash a custom bootloader using FlashStock. This loads a new kernel on the small flash partition on the device, and it gives you the ability to boot from an SD card. Then, you download a Linux distribution for the z2 (I chose z2sid), copy the image to the SD card (using a raw copy tool like dd) and boot from it. I have an 8GB SD card in the Zipit. This Z2 is better than my first Pentium computer along every single dimension except screen size.


Once Debian is installed on the Zipit, you have access to the 'apt-get' package management tool. After installing vim, gcc, make, it turned into a fully functioning ARM development system. I did not want to type using the tiny keyboard, which is uncomfortable and lacks important keys. So I installed an ssh server called dropbear. Now I can remotely log in to the device, copy files back and forth, and use it for development.

Once it was set up, I wondered if I can serve web pages from it. After installing a webserver called nginx, the zipit holds up very well. I'm serving three virtual hosts externally from it, and two hosts internally. For a small amount of traffic, this is the perfect device.

Power consumption

With the screen turned on, the Zipit consumes less than 2 watts of power. When the lid is shut, the screen turns off and power consumption drops to 1 watt. By comparison, a typical laptop consumes 30 Watts and a typical desktop consumes 100 watts. The device is dead silent and cold to the touch. Due to the low power consumption, it generates no perceptible heat. You can leave it in the corner of a cramped closet, and it will chug away. Silently.

There is a lot of potential in these low-power devices called plug computers. Plug computers and mobile devices have caused a resurgence of ARM processors due to their low power consumption. With improvements in processor technology, an entry-level ARM CPU can handle tasks reserved for a "real" computer. Web serving, file serving and ssh-login don't require a beefy processor.  Rather than maintain a full computer, you can purchase a small, silent, and low-footprint computer to handle these tasks for you. Once it is set up, you can store it in a closet somewhere, completely out of sight. Low power consumption allows the device to run exclusively on solar power.

You could use it as an entry point to a home network. Once the SD card is copied to a desktop, you can recover from a malicious attack by copying a known-good image back onto the card.

Low-power computing holds a lot of promise. In the developed world, it can be used instead of power-hungry computers for routine tasks. In the developing world, it can be the first computer for a family.

Sunday, August 07, 2011

Beautiful, professional resumes with LaTeX

I love LaTeX, but when it comes to resume writing, my skills at LaTeX fall short. This means that I have to load up OpenOffice, and spend hours tuning every single spacing and pagination problem.

Enter Moderncv, a class file for professional looking resumes. If you use LaTeX, it is the perfect document class for writing a resume.

This is what my resume looks like:


You can download the resume in PDF. This resume was generated from this LaTeX source file.

The moderncv class is straightforward: you fill in the details and it generates a beautiful, professional resume for you. Once you fill in the critical details, the page layout is handled for you. So if one section is too big it is automatically shifted to the next page. You fill in the contents, LaTeX handles the rest.

Itching to give it a try? To get started, download the moderncv class from CTAN. Either use my resume as an example, or look through the examples directory for a file called 'template.tex'. On Ubuntu, you need the texlive and texlive-latex-extra packages.

On Ubuntu or Debian machines, you can download all the packages with:
$ sudo apt-get install texlive texlive-latex-extra 
 

Lego NXT Mindstorm with Linux

There is a lot of documentation on the NXT and Linux on the Internet. A lot of it is very good, like this page showing Linux setup with NXT. Unfortunately it is scattered all over the place, making it difficult for a complete newbie to make headway with the Lego Mindstorms on Linux. This page links to the major steps and will enable you to set up Linux-Mindstorms communication and the programming environment.

Why Linux?

Linux is better suited for Lego programming. It is essential if you have a Linux machine, and don't want to install Windows (or MacOS) just for using the Mindstorm. But the advantages of Linux go beyond that. Linux is a great programming environment, and interfacing it with Mindstorms allows you to connect it to other devices. Linux machines have a good programming interface for Bluetooth, GPS, networking through the Internet, sound, display, and large-scale processing.  Connecting the NXT to a Linux device opens up a lot of possibilities.

Also, many netbook computers are ideal robot building blocks: they are small, light and lack fragile spinning disks. Linux is space efficient, and runs well on these devices.


Journey

These are the steps you need to be completely up and running with Linux and the NXT. These pages are listed in order, and each page builds upon the previous work.
  1. Setting up the Linux-NXT Bluetooth connection
  2. Setting up the Linux-NXT NXC programming environment
  3. Writing a program that demonstrates NXT Robot-Linux communication over Bluetooth 
With a little bit of setup, you can combine the flexibility and design of the Mindstorms with the power of Linux.