FoMR: Speculative Super-optimization: Boosting Performance via Speculation-Driven Dynamic Binary Optimization

University Of Virginia Main Campus, Charlottesville VA

Investigators

Abstract

Modern processors are characterized by increasing 'core' counts on a single multi-core processor, where a 'core' is a computing unit, allowing each core to operate in parallel to, and independently of, other cores on the processor. Yet, a substantial chunk of software applications is inherently sequential. Although modern compilers feature sophisticated optimizations, significant waste of computational resources across these multiple-cores still occurs due to computational patterns that are unpredictable at compile-time. This project explores techniques to deploy aggressive speculative dynamic optimizations within the micro-processor, enabling continuous optimization of inherently sequential code. This work addresses a pressing need for systems that can aggressively and yet seamlessly super-optimize machine code, adapting to the dynamic execution environment, and thereby speed up execution of applications on computers. This research will also foster existing efforts and initiatives of the investigators to build a pipeline of students from diverse backgrounds. This project explores dynamic binary optimization techniques at the processor level to speculatively generate and execute a super-optimized instruction stream, by leveraging established speculative processor features such as branch prediction, value prediction, and loop stream detection. This project introduces two distinct flavors of speculative super-optimization: (a) a hardware implementation that leverages dynamically predicted program state to perform simple, yet powerful peephole optimizations on short instruction sequences, and (b) a firmware implementation that leverages a microcode-based dynamic re-compiler running as a helper thread to perform more sophisticated optimizations. Owing to the plethora of speculative processor features and compiler/runtime/hardware optimizations to choose from, this project explores a rich sample space of speculative super-optimization strategies, along with extensive profitability analysis to dynamically identify appropriate targets for super-optimization. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.