Accelerating Android AOSP and Embedded Linux Builds, Part 1
https://electric-cloud.com/blog/accelerating-android-aosp-and-embedded-linux-builds-part-1/
Accelerating Android AOSP and Embedded Linux Builds: Introduction
Embedded Linux is everywhere today. Look around and you can probably see a half dozen or more devices running some Linux-based operating system. Everybody knows about Android on mobile devices, of course. But have you ever thought about the software inside your LG TV? How about your Samsung washing machine? Or your Garmin smartwatch? What about the infotainment system in your GM, Ford, Audi, Volvo or Mazda vehicle? All of these feature Android or another variant of Embedded Linux at the heart of the system.
If you’re in the business of making software for these or any of the thousands of other devices running embedded software you probably already know the role Linux plays, and you may be familiar with the pain that it brings in the form of long build / test / release cycles. What you may not know is that there’s a solution: ElectricAccelerator. And even if you are a long-time user of ElectricAccelerator, you may not be aware of recent improvements that specifically target the Android, Yocto Project and Buildroot ecosystems to deliver world-class build performance.
In this first of a series of posts, we will discuss accelerating system builds for any of the three most common platforms used for embedded device development – Android/AOSP, Yocto Project and Buildroot.
Using the latest 11.0 release of ElectricAccelerator which ships with support for all three platforms, we will provide a high-level review of the different build architectures while providing some reference acceleration benchmark results. In the coming weeks, we will publish more in-depth articles describing how to use ElectricAccelerator to accelerate these builds beyond what’s possible using other tools, while outlining strategies to consider for getting the most insights, performance and output from Android and Embedded Linux build environments.
A survey by VDC Researchin January 2018 showed that nearly 45% of users base their embedded development on open source operating systems like Linux. This is expected to grow by ~53% by 2021.
VDC Research, January 2018
The Android and Embedded Linux (Yocto Project / Buildroot) builds
Android AOSP
Android has been around for years, maturing into a viable alternative for a vast range of embedded device types. At Electric Cloud, we serve customers using Android to power their mobile phones, TVs, washing machines, watches, infotainment systems, RFID readers, point-of-sale systems, and more.
In 2018 alone, nearly 3100Google Play certified Android devices were launched, adding to the 16,000or so certified devices that existed in the market at the start of the year. As staggering as these numbers may be, it’s noteworthy to highlight this is likely a significant underrepresentation of actual devices in existence, given the data only account for those launched by OEMs signed up with Google, thereby paying a fee to get certified.
ElectricAccelerator 11.0 for Android AOSP
Our customers have relied on ElectricAccelerator to accelerate their Android builds for years. Ever since the release of ElectricAccelerator 7.0 in 2013, we have used the Android build as a reference environment to drive innovation in build acceleration and push the technological boundaries of ElectricAccelerator. Noteworthy capabilities that we have introduced in recent years include:
- Parse Avoidance for caching of makefile parse results,
- Dependency Optimization for aggressive-but-safe parallelization, and,
- Job Caching for reuse of output from compilers, documentation generators and other tools.
Among performance and reliability improvements, ElectricAccelerator 11.0 introduces support for the Android LineageOS distribution.
Using an Android 9 Pie build in our ElectricAccelerator 11.0 reference environment, the below screenshot from the ElectricInsight build analysis tool for ElectricAccelerator shows a simulation report where the y-axis represents the number of ElectricAccelerator agents (~cores), and the x-axis represents time. The blue bars simulate ElectricAccelerator build speed without enabling any of the Android-optimized capabilities, the green bars are with everything turned on.
Simulating Android Pie build times with ElectricInsight
For the optimized (green) setup, the report shows a significant acceleration already at low densities of cores/agents. From the report, we can also derive that we will need 132+ build agents with the unoptimized (blue) setup to get anywhere close to optimal acceleration. In future articles, we will go into much more depth on Android build acceleration.
Embedded Linux
As the term implies, Embedded Linux defines and builds a specialized configuration of Linux, tailor-made for embedded devices. That said, there is a lot of resemblance of Embedded Linux development environment to standard or desktop Linux distributions in how one reference and configure packages to define the system. These packages are then either directly compiled into the kernel or leveraged as 3rd party libraries or components in the system build.
We have chosen to focus on the Yocto Project and Buildroot platforms, two popular frameworks for defining and building an Embedded Linux environment. The Yocto Project is an open source collaboration project that helps developers create custom Linux-based systems regardless of the hardware architecture.It has ~75 participating organizations, strategically funded and supported by industry household names like Intel, AMD, Arm, and Facebook. Buildroot is a vendor-neutral open source project with a growing and active developer community, featuring 2300+ supported packages and components. Other Embedded Linux build platforms exists such as e.g. OpenWRT, which we may look at in future articles.
The below image describes the high-level architecture of the two Embedded Linux build systems, Yocto Project and Buildroot. While the goal of both these system is to produce a Linux kernel, bootloader and root file system, the tools and processes by these two platforms are quite different. Yocto Project uses bitbake as the meta build tool (originating from the OpenEmbeddedecosystem) and Buildroot leverages standard Linux kernel build tools, kconfig and make. To build each package included in the Linux system, Yocto and Buildroot are using a similar workflow of tasks, e.g. for retrieving sources, patching, configuring and compiling.
ElectricAccelerator 11.0 for Embedded Linux
Our recent 11.0 release of ElectricAccelerator improves support for Yocto Project and Buildroot. Specifically for Yocto Project with its bitbake tool, ElectricAccelerator 11.0 adds support for caching of configure tasks, and includes the bb2anno utility to enable end-to-end bitbake build visualization using ElectricInsight. Below image presents benchmark results from a reference build environment, along with the high-level scope of the ElectricAccelerator 11.0 support, accelerating configure and compile tasks of bitbake builds and all tasks of Buildroot builds.
For our Yocto Project reference build, ElectricAccelerator is reducing the overall runtime of the configure jobs by 14x due to the configure caching introduced in 11.0, while accelerating the overall build time by 46% or 1.9x. For our Buildroot reference build, ElectricAccelerator is accelerating the build by 3x, bringing down build duration from ~24m to ~8m. In future articles of this series, we will dive deeper into the specifics of these reference benchmark results and the ElectricAccelerator features making it all possible.
The top 2 Mobile Android device makers in the world are using ElectricAccelerator to power their private build clouds, accelerating software builds while optimizing infrastructure utilization. Automotive companies such as Visteon and Hyundai Mobis are leveraging ElectricAccelerator to speed up their long Android and Yocto Project builds, helping meet time-to-market pressures. IoT device makers such as Zebra Technologies depend on ElectricAccelerator for reducing their development cycle time.
