Evaluation of a Benchmark Suite Exposing Android System Complexities Using Region-Based Caching

The computer architecture community relies on standard benchmark suites like MiBench, NAS, PARSEC, SPEC CPU2006 (SPEC)®, and SPLASH to study different hardware designs, but such suites are insufficient for evaluating mobile platforms like Android. Even suites that were developed for embedded systems cannot be used to gain an understanding of Android device/system interaction because they do not exercise key components of the software stack. Although based on a conventional Linux ® kernel, Android includes native libraries, a virtual machine runtime, and an application framework with multiple components for managing resources. All these interact in complex ways to support Android applications. C programs running on Linux have a relatively simple virtual memory organization, and most memory references come from the application code. In contrast, Android has a much more complex virtual memory organization (due to its multiple APIs and numerous shared libraries), and most memory references come from the Android software stack. The complexity of Android's execution environment provides opportunities for computer architects to better support the execution characteristics, structures, and resource requirements of the Android software stack and opportunities for software developers to optimize their applications for this rich environment. To help the community to exploit these opportunities, we introduce Agave, an open-source benchmark suite designed to expose the complex interactions between components of the Android software stack.