Workflow Provenance and Its Application to Responsible and Trustworthy Artificial Intelligence

Artificial intelligence (AI) has raised concerns in governments, research communities, industry and the public opinion regarding its ethical and practical implications. Consequently, the ideal of Responsible and Trustworthy AI (RTAI) garnered attention worldwide [LWM24, Oak24, IBM24, Mic24, SLD+22]. The path towards RTAI involves developing practices to prioritise not only model performance (e.g., accuracy and loss), but also principles that play crucial societal roles, such as explainability, or making AI’s decision-making process understandable; transparency, ensuring clarity in AI’s design, data and operation; and environmental impact of the machine learning life-cycle (e.g., large-scale model training) [GMRRG19]. Recent works suggest that workflow provenance (i.e., the documentation and tracking of all processes within AI development) might be the key to support RTAI [SAL+22, KNHJ+23]. Workflow provenance refers to capturing detailed information about all activities, processes, and transformations applied to data and code during AI development and operations. It includes information about data sources, data preprocessing, model selection, hyperparameter tuning, and evaluation metrics, among others. Capturing this provenance could provide a holistic view of the AI workflow, making it transparent and reproducible, and allowing further analysis of features such as the carbon footprint and computational power required for training a given model [VW21]. However, today there are no comprehensive formalisms able to capture the complex relationships in workflow and model provenance data [BBFM23]. Furthermore, multiple technical challenges arise when attempting to capture, store and manage the full provenance of AI workflows [MCSAGBS21, SS23], and it is still not well understood what information is valuable and how it can be exploited [JRO+20].

This project aims to investigate methods to leverage workflow provenance metadata in support of RTAI. It will investigate mechanisms to formalize, capture, store, retrieve, inspect, analyse and visualise metadata in AI-powered workflows, and will explore the relationship between model provenance, metadata, and model and system behavior, aiming to decipher how architectural and algorithmic characteristics impact in the model's outcome and efficiency. First, we will explore the formalisation of ontologies and taxonomies for AI workflow provenance data from multiple angles: system (e.g., hardware, computing infrastructure, storage), platform (e.g., workflow manager, machine learning framework), model (e.g., hyperparameters, performance, architecture), and application (e.g., input and intermediate data, feedback). The resulting theoretical framework will expand our previous work on taxonomies for neural network metadata [RCLJT22] to other machine learning methods. At the technical level, new data structures, algorithms, system architectures and interfaces will be explored to efficiently produce and query a detailed record of data sources, processing steps, and model configurations. We will build upon our previous work on reproducible workflow execution [RCAV21] to rarchitect a proof-of-concept for a large-scale provenance data management system suitable for the analysis AI workflow applications. Finally, we will work towards a methodological framework leveraging provenance matadata taxonomies, the causal model between the studied models and their behavior, and associated statistical findings that support RTAI in practice.