Parallel Compilation on Virtual Machines in a Development Cloud Environment

September, 2013
IDA document: D-4996
FFRDC: Systems and Analyses Center
Type: Documents
Division: Information Technology and Systems Division
David A. Wheeler See more authors
Government employees and contractors cannot, in some cases, install software development tools on their local systems. For example, they may be using a mobile device that cannot adequately support software development tools, or security policy restrictions may inhibit installing these tools. A possible solution is to create “development clouds” that support software development without requiring any tools specific to software development to be installed in the local system. An additional argument for development clouds is that they could use parallel processing to greatly reduce compilation time. However, it would be wise to test whether this hypothesis is actually true for virtual machine (VM) environments, since they are often used to implement clouds. This document describes an IDA innovation lab (iLab) experiment that verifies that VM environments exist that support parallel processing for performing compilations, and that using them can (at least in some circumstances) significantly speed compilations compared to a single-CPU case. Using the default settings, it took 166.85 minutes to compile the Linux kernel version 3.10.5 with 1 CPU, compared to 28.50 minutes with 16 CPUs, resulting in a reduction of 138.35 minutes (over 2 hours) on average for each full compilation. Thus, 16 CPUs compiled the software 5.85 times faster compared to one CPU. Additional optimizations (involving the “-j” flag) slightly decreased the average 16-CPU compilation time even further. This shows that a system that can cost-effectively provide many CPUs to execute a VM, such as a development cloud environment, can significantly speed up compilations. Therefore, development cloud environments may be a useful approach for supporting software creation and maintenance.