SaTC: CORE: Small: Practical Whole Kernel Memory Safety Enforcement

University Of California-Riverside, Riverside CA

Investigators

Abstract

The operating system (OS) kernel is the security-critical foundation of a computer system. Unfortunately, errors in the kernel software of commodity operating systems like Windows and Linux can allow a malicious attacker to take over the whole system. This research project is developing new techniques to eliminate certain types of critical errors from commodity OS kernels in a way that is both mathematically provable and efficient. The researchers are developing a unique combination of static and dynamic techniques to provide practical spatial memory safety for commodity OS kernels. These techniques include a new type system which guarantees that for every kernel module all its memory accesses are within bounds and the corresponding access type (read, write, execute) is allowed by the module's capabilities. This is achieved by automatically inserting necessary runtime checks during compilation. Since these new checks introduce performance overhead, the project includes additional techniques to reduce the overhead. The first optimization approach is utilizing software compartmentalization and model checking to eliminate redundant checks. The second optimization is adopting Intel MPX (memory protection extension), a new and commercially available hardware feature to reduce the cost of runtime checks.