Android & CM build basics

[CM source code folders]

bootable/
Among other things, the source for ClockworkMod recovery is in here.

device/
device/ contains part (if not all) of the board support package for a device, and is organized as device/<vendor>/<codename>. So the files for Nexus 7 are under device/asus/grouper/, the files for Nexus 4 are under device/lge/mako/. If you want to do a port of Android for a particular device that currently has no source, you should probably start by finding a similar device (with a similar underlying platform) and use it as a "base" for modifying for that device. Do take a peek into device/asus/grouper to see the various files that are used to build for the Nexus 7 and compare with other devices.
Note:
Many of the xml files that control Android core functionality and verbiage (strings) can be overridden by new xml files that you place in the device-specific device/[vendor]/[codename]/overlay folder.

hardware/
Platform/hardware-specific libraries and such go in this folder. For example, the code in hardware/ti is used to support devices using a chipset by Texas Instruments.

kernel/
Like device/, kernel/ directories are organized like kernel/<vendor>/<codename> and inside there you'll find the source for the kernel. This source may be downloaded "on the fly" by the CyanogenMod build system.

out/
The output of your build will go in here by default. When you type cd $OUT after building, you're actually getting a shortcut to out/target/product/<devicename>. This is the only folder that should have stuff added to it (or changed) when you're doing a build. stuff that will become the system folder on your device goes in system. Stuff that will become your ramdisk goes in root. The various stages and temporary scripts used to create your recovery.img and boot.img and your kernel go here as well. The object files get built into the obj directory. And there are generic compiler tools and other build (such as from the ndk) and such that get built in out/ as well. When you do a make clobber, you're just getting rid of out/.

system/
Lower-level linuxy system stuff such as netd, fastboot support, the shell (which you can use to type commands into your device), etc. are here.

vendor/
The /vendor folder contains stuff particular to a vendor build such as CyanogenMod. It's also the place where proprietary blobs are extracted from the device, again using the same vendor/device codename convention as under device/.

-------------------

[Generic Android build description]

To set up your build environment, you need to load the variables and functions in build/envsetup.sh. Do this by 'source-ing' the file into your shell environment, like this:

$ source build/envsetup.sh
or
$ . build/envsetup.sh

You can type 'help' at this point to see some utility functions that are available to make it easier to work with the source.

To select the set of things you want to build, and what items to build for, you use either the 'choosecombo' function or the 'lunch' function. 'choosecombo' will walk you through the different items you have to select, one-by-one, while 'lunch' allows you select some pre-set combinations.

Run "make" to start building, also -j can be used, like:

$ make -j4 showcommands

Make targets
make sdk - build the tools that are part of an SDK (adb, fastboot, etc.)
make snod - build the system image from the current software binaries
make services
make runtime
make droid - make droid is the normal build.
make all - make everything, whether it is included in the product definition or not
make clean - remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/
make modules - shows a list of submodules that can be built (List of all LOCAL_MODULE definitions)
make <local_module> - make a specific module (note that this is not the same as directory name. It is the LOCAL_MODULE definition in the Android.mk file)
make clean-<local_module> - clean a specific module

You can also specify to use the 'ccache' compiler cache, which will speed up things once you have built things a first time. To do this, specify 'export USE_CCACHE=1' at your shell command line. (Note that ccache is included in the prebuilt section of the repository, and does not have to be installed on your host separately.)

<Building only an individual program or module>

If you use build/envsetup.sh, you can use some of the defined functions to build only a part of the tree. Use the 'mm' or 'mmm' commands to do this.
The 'mm' command makes stuff in the current directory (and sub-directories, I believe). With the 'mmm' command, you specify a directory or list of directories, and it builds those.
To install your changes, do 'make snod' from the top of tree. 'make snod' builds a new system image from current binaries.

<Setting module-specific build parameters>

Some code in Android system can be customized in the way they are built (separate from the build variant and release vs. debug options). You can set variables that control individual build options, either by setting them in the environment or by passing them directly to 'make' (or the 'm...' functions which call 'make'.)

For example, the 'init' program can be built with support for bootchart logging by setting the INIT_BOOTCHART variable. (See Using Bootchart on Android for why you might want to do this.)
You can accomplish either with:

$ touch system/init/init.c
$ export INIT_BOOTCHART=1
$ make

or

$ touch system/init/init.c
$ m INIT_BOOTCHART=1

<Makefile tricks>

These are some tips for things you can use in your own Android.mk files.

build helper functions
A whole bunch of build helper functions are defined in the file build/core/definitions.mk
Try grep define build/core/definitions.mk for an exhaustive list.

Here are some possibly interesting functions:

print-vars - shall all Makefile variables, for debugging
emit-line - output a line during building, to a file
dump-words-to-file - output a list of words to a file
copy-one-file - copy a file from one place to another (dest on target?)

<Add a file directly to the output area: used in your Android.mk>

You can copy a file directly to the output area, without building anything, using the add-prebuilt-files function.
The following line, extracted from prebuilt/android-arm/gdbserver/Android.mk copies a list of files to the EXECUTABLES directory in the output area:

$(call add-prebuilt-files, EXECUTABLES, $(prebuilt_files))

<Adding a new program to build>

Steps for adding a new program to the Android source tree:

- make a directory under 'external', e.g. ANDROID/external/myprogram
- create your C/cpp files.
- create Android.mk as clone of external/ping/Android.mk
- Change the names ping.c and ping to match your C/cpp files and program name
- add the directory name in ANDROID/build/core/main.mk after external/zlib as external/myprogram
- make from the root of the source tree

your files will show up in the build output area, and in system images.
You can copy your file from the build output area, under out/target/product/..., if you want to copy it individually to the target (not do a whole install)
See http://www.aton.com/android-native-development-using-the-android-open-source-project/ for a lot more detail.

-------------------

[Android build & Android.mk description]
From: http://ericdev.blogspot.com/2010/06/cc-closer-look-at-android-project-build.html

envsetup.sh
There is one important file in build directory itself. That is the envsetup.sh file. This shell script contains a lot of functionality that is nice to have when working with Android. To invoke it type
". build/envsetup.sh" in the android root directory. This will allow you to use a number of new commands including:

m - make from the top of the tree
mm - builds all modules in the current directory
cgrep - grep in all c-files
jgrep - grep in all java-files
mgrep - grep in all makefiles
lunch - choose build target

These shell commands are very useful when working with the platform. Whether you are looking for something specific in a source file or you just want to build the current module for a test.

build/target
The target directory contains make configurations for different boards and products. The target directory is where new products can be added as needed by a hardware manufacturer. In the current release there is a generic product description, and sdk product and some other files. The available boards are the emulator, a generic board and the simulator.

The board makefiles describe the hardware platform level and can be shared among a number of products. For example you can build both the generic product and the generic_with_google product to run on in the emulator.

The makefiles for different products are listed in the file AndroidProducts.h. To add a new product put the makefile in the build/target/product folder and add the product to AndroidProduct.h. It is possible for product makefiles to inherit from each other to reduce the need for copying. The generic_with_google products inherits the generic product that inherits the core product to make up a three level product specification.

build/core
The core folder is the location of the build system files. These files are used to set up the build system and to build all the different modules as configured in the individual Android.mk files. The main.mk file is the entry point to the build system and it is this files that is included in the top level makefile in the project root directory. The build system can also be invoked by the shell command mm with at subset of the makefiles

Most of the build configuration is set in the config.mk file and the definitions.mk file contains the functions to be invoked for the different source types. The Android makefile uses the BUILD_SHARED_LIBRARY variable set in the config.mk file. This will set the build chain for building a shared library in a number of steps using several other files.

<Android.mk description>

The full Android.mk explanation(syntax) is here:

http://www.kandroid.org/ndk/docs/ANDROID-MK.html

This is the second post on the build system where we will take a closer look at the Android.mk file and what options are available. An Android.mk file describes the build for any native module that should go in the platform. We will start by looking at the makefile for the external ping facility found in external/ping.

---
ifneq ($(TARGET_SIMULATOR),true)

LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)
LOCAL_SRC_FILES:= ping.c
LOCAL_MODULE := ping
LOCAL_STATIC_LIBRARIES := libcutils libc
include $(BUILD_EXECUTABLE)

endif # TARGET_SIMULATOR != true
---

The first line is just a check if we are building for the (obsolete?) simulator. The next line sets the local path to the current directory (the directory of this Android.mk file). This is handled by a function called mydir that can be found in the definitions.mk file in the build system.

The next step is to include a makefile that will clear all local variables with the line include $(CLEAR_VARS). CLEAR_VARS is set in the build configuration in config.mk and will include theclear_vars.mk file at this point. The main purpose of this include is to make sure that we do not use any local variables set by another module.

After the two initial lines that basically prepares the build system to start with this module we set the build variables. These are:

LOCAL_SRC_FILES - the source files that make up this module
LOCAL_MODULE - the name of the module
LOCAL_STATIC_LIBRARIES - libraries to statically link to this module

Since ping will be an executable command placed in /system/bin in the Android file system the make file for building an executable should be used. This done with the line:
include $(BUILD_EXECUTABLE)

There are makefiles available in the build system that can be used to build a number of different types of module. To include them you may use one of the variables set in the config.mk file. A few important ones are:

BUILD_EXECUTABLE - Build an executable module
BUILD_SHARED_LIBRARY - Build a shared library
BUILD_STATIC_LIBRARY - Build a static library
BUILD_PREBUILT - Add prebuilt components

There is also a number of local variables that are recognized by the build system and that needs to be set in order to build your module.

LOCAL_C_INCLUDES - path to include files needed by your module, e.g. ($KERNEL_HEADERS)
LOCAL_CFLAGS - Any additional flags to pass to the compiler
LOCAL_LDFLAGS - Any additional flags to pass to the linker
LOCAL_SHARED_LIBRARIES - Shared libraries that the module directly links against
LOCAL_SRC_FILES - The source files to compile
LOCAL_STATIC_LIBRARIES - Static libraries to include in the module

You may use one Android.mk to build several items. It is possible to build both a library and an executable using the same makefile. To illustrate how to write a makefile for some native functionality we will look at another example. This makefile will build a shared library and an executable. It links to a couple of shared libraries in the system.

---
LOCAL_PATH:= $(call my-dir)

include $(CLEAR_VARS)
#Name of target to build
LOCAL_MODULE:= libmylibrary
#Source files to compile
LOCAL_SRC_FILES:= mysrcfile.c mysothersrcfile.c
#The shared libraries to link against
LOCAL_SHARED_LIBRARIES := libcutils
#No special headers needed
LOCAL_C_INCLUDES +=
#Prelink this library, also need to add it to the prelink map
LOCAL_PRELINK_MODULE := true
include $(BUILD_SHARED_LIBRARY)

#Clear variables and build the executable
include $(CLEAR_VARS)
LOCAL_MODULE:= myinfocmd
LOCAL_SRC_FILES:= mycmdsrcfile.c
include $(BUILD_EXECUTABLE)
---

The output of the build system is put in the out folder in the Android project. Build output is directed to a subfolder based on the product. A default build is for the generic product and will output everything target related to that folder. A shared library ends up in/out/target/product/generic/system/lib. In the same way an executable will end up in/out/target/product/generic/system/bin. The object files and intermediates are kept in the obj folder. The root, system, and data folders will be used to generate the file system images needed for Android in the last build step.

-------------------

[CM envsetup.sh and CM build commands]

The envsetup.sh in CM adds some additional commands, some useful cmds are:

<mka> -- Basically this means "make -j"
This little gem of a command is basically equivalent to a super-charged version of make. make is the program that gets called to build our source, choosing the correct compiler for each part of the Android OS that we are making. Problem is, make is SLOW in its default configuration. It can take hours longer depending on your hardware. So what did they do? They mated make with a cheetah, and took their child mka. mka improves upon make by using the program sched_tool to make full use of all the threads available on your machine (For AMD, this is equivalent to the number of cores your processor has; For Intel, this is usually equivalent to twice the number of cores your processor has, due to HyperThreading). What this means is that ALL of your processor is working, not just one small part of it.

<breakfast>
You may not ever use this command, but in order to explain brunch, we have to explain breakfast first. Breakfast is a function used to configure your build. It keeps track of a list of officially-supported devices to build for, and allows you to choose one. You can do this two ways,

$ breakfast [device name]

or

$ breakfast (no device name)

The first method is like a shortcut to select your device. If you do not put a device, the script will output a list of available devices to build for (limited to those CM builds nightlies for, which isn't necessarily complete), and you can then choose yours. breakfast then goes on to configure your build environment with the correct variables to create your device-specific rom.

<brunch>
Defined simply, brunch is equivalent to

$ breakfast [device name] && mka bacon

This means that it sets up your build environment to be configured for your device, and then commences the build process. mka bacon is just CyanogenMods’s way of saying build the code for your device. It’s generally only used for officially supported devices (those for which a public CyanogenMod-maintained device repository exists).

<lunch>
This one's also pretty simple to explain. It’s used EXACTLY like breakfast, the only difference being the choices you have to build with it, and how it configures the device. Using lunch, you can choose non-official or non-standard builds for your device. This includes special debug versions and also allows you to build CyanogenMod for use on the Android Emulator. Unlike breakfast, lunch makes no assumptions with regard to device configuration locations and formats, so it will scan the entire tree for matching options (depending on the number of locally available devices, this may take a significant amount of time). To build after running lunch, simply issue the command mka. No bacon here… sorry.

The fulllist can be checked here: http://wiki.cyanogenmod.org/w/Envsetup_help

<Commands used when building & debugging>

croot -- this command will take you to the root of the source code.
mm and mm -B -- this is the "make module" command and it is very useful if you are working on a particular module (that is, a section of the code to be built) and don't want to have to rebuild everything just to test it. To use it, cd into the directory that you want to test, then just type mm to build just the module in the working directory. If you want to start the build from scratch, add the -B. This is a good companion with adb sync system above, which you can use to push the newly built files directly to your device for testing without having to reflash everything.

adb sync system -- This is useful when debugging. assuming you have adb installed, this little-known, but powerful command will sync the contents of $OUT/system with the /system directory on your device. It does require that you have write access to /system. This command is very useful for making and testing quick changes to your device. Note though that adb sync system sometimes has problems with permissions and symlinks.