Data Interfaces for Hybrid Quantum-Classical Scientific Computational Workflows

Modern scientific discovery depends heavily on computational workflows that can orchestrate large and complex in-silico experiments. The evolution of workflow application requirements in contemporary computing is pushing the need to integrate a diverse portfolio of services for high-performance computing (HPC), artificial intelligence (AI), cloud computing, and other large-scale computing environments and their associated data [BDF+20, MDM+21]. Looking ahead, key players in cutting-edge technologies are making substantial investments and long-term strategic decisions about the future landscape of distributed computing and supercomputing infrastructures [Des22]. Quantum Computing (QC) systems are being explored as the next high-impact extension to the computing spectrum, particularly with integration into supercomputers and cloud environments. The successful interoperability between classical and quantum systems will depend on middleware interacting with heterogeneous hardware technologies (e.g., CPU, GPU, TPU, FPGA, QPU) and their associated software stacks and data management methods. The most realistically feasible approach towards leveraging QC in the near term involves loosely-coupled integration of the classical and quantum devices through classical computing networks [BBB+24]. Although these solutions can offload computation to quantum systems, approaches toward high-level hybrid programming are still lacking. Current work on integrating QC into classical computing ecosystems focuses on the algorithms' interoperability and performance without considering workflow-specific challenges like task-resource mapping, orchestration, workload balancing, and integration with existing workflow management environments. In this context, we find complex open challenges in combining multiple programming models in a single application with workflow steps that combine quantum and classical processing in a domain-agnostic manner. Specifically, data-oriented optimizations in the areas of data encoding, arrangement, locality, or mapping to high-level data abstractions are rarely explored in hybrid scenarios [WBLV22].

This project has the overarching goal of researching novel data interfaces for hybrid quantum-classical workflows able to reconcile quantum and classical data representations. We will focus on the most realistically feasible approach to near-term hybrid systems: loosely-coupled integration of the classical and quantum devices through classical computing networks, for example through cloud services or integration with supercomputers. While solutions to offload computation to quantum systems exist, they do not have data models suitable for high-level hybrid programming. First steps include studying the low-level data structures and patterns in existing approaches for interacting with quantum devices; and characterising the data access and transfer patterns in small scenarios of variational quantum algorithms [CAB+21]. Further exploration of the formalisation of a data-oriented benchmark for hybrid quantum-classical workflows covering aspects of data encoding, arrangement, locality, and mapping to high-level data abstractions is expected, with the goal of facilitating the assessment of the diverse data representations, volumes and processing rates in existing hybrid workflows.