Repository logo
Log In(current)
  1. Home
  2. Colleges & Schools
  3. Graduate School
  4. Masters Theses
  5. Provenance-Aware Fine-grained Containerized Workflows and Orchestration
Details

Provenance-Aware Fine-grained Containerized Workflows and Orchestration

Date Issued
May 1, 2026
Author(s)
Anderson, Ty Julian
The University of Tennessee
Advisor(s)
Michela Taufer
Additional Advisor(s)
Kin Ng Lugo
Jack Marquez
Permanent URI
https://trace.tennessee.edu/handle/20.500.14382/57790
Abstract
Reproducibility in computational science requires precise control over execution environments, workflow structure, and data lineage. Prior work introduced a provenance-aware fine-grained containerization framework built on Apptainer that decomposes scientific workflows into individual data and application containers and captures runtime provenance. However, the system was limited to single-node execution, static SIF-based data containers, and manually orchestrated workflows. This thesis extends fine-grained containerized environments to support arbitrary multi-layer Directed Acyclic Graph (DAG) workflows and distributed execution via the Pegasus Workflow Management System. We augment the Apptainer plugin to construct and execute DAG-based workflows locally using Kahn’s topological sorting algorithm and develop a Jupyter-based orchestration layer that translates containerized DAG specifications into Pegasus workflows for distributed execution. To address static storage limitations, we introduce IMG-based data containers that support dynamic resizing during runtime. Provenance capture is decoupled into a reusable library, enabling consistent metadata generation under both local and distributed execution. We evaluate the extended system by measuring zero-copy throughput for SIF and IMG containers, comparing Pegasus job wall time for a bioinformatics variant-calling workflow under containerized and non-containerized execution, and analyzing the data-transfer-to-computation ratio. Results demonstrate that fine-grained containerized workflows can scale to distributed infrastructures while preserving provenance guarantees with manageable performance overhead.
Subjects

High-Performance Comp...

Containers

High-Performance Comp...

Reproducibility

Degree
Master of Science
Major
Computer Science
File(s)
Thumbnail Image
Name

ANDERSON-PRIMARY-2026.pdf

Size

1.87 MB

Format

Adobe PDF

Checksum (MD5)

208ac6d0bf6b652839fadec5233bdbc7


University Libraries

1015 Volunteer Boulevard
Knoxville, TN 37996
865-974-4351

Map & Directions
Donate to the Libraries
  • About
  • John C. Hodges Society
  • Speaking Volumes magazine
  • Outreach
  • Directory
  • Employment
  • Policies
  • Library Intranet
University of Tennessee power T logo

The University of Tennessee, Knoxville
Knoxville, Tennessee 37996
865-974-1000

Events
A-Z
Apply
Privacy
Map
Directory
Give to UT
Accessibility

Built with DSpace-CRIS software - Extension maintained and optimized by 4Science