Doctoral Dissertations

Date of Award

8-2021

Degree Type

Dissertation

Degree Name

Doctor of Philosophy

Major

Computer Science

Major Professor

Austin Z. Henley

Committee Members

Max Schuchard, Austin Henley, Michael Jantz, Scott Ruoti, Matthew Van Gundy

Abstract

Intra-process memory isolation can improve security by enforcing least-privilege at a finer granularity than traditional operating system controls without the context-switch overhead associated with inter-process communication. Because the process has traditionally been a fundamental security boundary, assigning different levels of trust to components within a process is a fundamental change in secure systems design. However, so far there has been little research on the challenges of securely implementing intra-process isolation on top of existing operating system abstractions. We find that frequently-used assumptions in secure system design do not precisely hold under realistic conditions, and that these discrepancies lead to exploitable vulnerabilities. We evaluate two recently-proposed memory isolation systems and show that both are vulnerable to the same generic attacks that break their security model. We then extend a subset of these attacks by applying them to a fully-precise model of control-flow integrity, demonstrating a data-only attack that bypasses both static and dynamic control-flow integrity enforcement by overwriting executable code in-memory even under typical w^x assumptions. From these two results, we propose a set of kernel modifications called Xlock that systemically addresses weaknesses in memory permissions enforcement on Linux, bringing them into line with w^x assumptions. Finally, we present modifications to intra-process isolation systems that preserve efficient userspace component transitions while drastically reducing risk of accidental kernel mismanagement by modeling intra-process components as separate processes from the kernel's perspective. Taken together, these mitigations represent a more robust architecture for efficient and secure intra-process isolation.

Download
Files over 3MB may be slow to open. For best results, right-click and select "save as..."

Share

COinS