T3: Validity Constraints for DAWs

Data Analysis Workflows (DAW) for scientific problems are often developed only for a specific problem at a specific institution for a specific infrastructure. This leads to the situation that DAWs contain multiple hard-coded decisions, realizing implicit expectations regarding their input data and their execution environment. However, DAWs also are frequently reused for different problems (the research question evolves), for different data (the data to be analyzed changes), or on different infrastructures (the execution environment changes). Such reuse is painful when hard-coded, undocumented decisions strongly influence the DAW execution [1]. Running a DAW with such properties under different circumstances may lead to incorrect results, sudden aborts, meaningless log entries, non-terminating executions, overflows of buffers (memory, disk, log-space) etc. Very often, the execution simply stops with an arbitrary, undocumented low-level error (e.g., “core dump”).

One way to prevent or at least reduce the number of such cases is the enhancement of DAW execution engines with the ability to manage and control validity constraints. A validity constraint (VC) is a constraint on some property of the input data or the execution environment. We will differentiate between hard VCs, which must be fulfilled to ensure correct execution of a DAW (e.g., result file size must be larger than 0 bytes), and soft VCs, which indicate deviation from expected behavior that may or may not be fatal (e.g., result file size is typically between 120 MB and 180 MB). VCs of both types may be detected automatically from log file analysis or can be specified by a DAW developer as part of the DAW specification (like integrity constraints for databases or assertions in programming languages). They can target any level of abstraction of a DAW, i.e., the abstract, logical, or physical level. Once uncovered, VCs can be controlled automatically by a DAW engine; typically, violation of hard VCs lead to abortion of DAW execution with a meaningful error message; violation of soft constraints will result in warnings to the developer. Depending on the specific VC, they may also be checkable prior to executing the DAW, which may lead to substantial reductions of computations and wait times.

Accordingly, the goal of team T3 is to develop jointly a framework for managing VCs during the DAW life cycle of creation, execution/control, and adaptation. We will create a software library that can be integrated into DAW specification languages, into DAW execution engines, and into resource managers. The framework will model the most important aspects of DAWs in terms of VCs (i.e. the properties and attributes that are to be constrained), allow for the registration and efficient search of VCs before and during execution of a DAW, and provide a driver to check VCs during DAW execution.

Collaborations

  • A1 What is a valid trace of log information?
  • A3 What is a valid analysis results?
  • A5 What is a valid classification model?
  • B3 What is a valid execution?
  • B4 What are valid file placements?
  • B5 What are valid file properties?

Related Publications

2023

Florian Schintke; Ninon De Mecquenem; David Frantz; Vanessa Emanuela Guarino; Marcus Hilbrich; Fabian Lehmann; Rebecca Sattler; Jan Arne Sparka; Daniel T. Speckhard; Hermann Stolte; Anh Duc Vu; Ulf Leser

Validity Constraints for Data Analysis Workflows Miscellaneous

2023.

BibTeX

References

  1. Sehrish Kanwal and Farah Zaib Khan and Andrew Lonie and Richard O. Sinnott (2017): Investigating reproducibility and tracking provenance – A genomic workflow case study. In: BMC Bioinformatics, vol. 18, 2017.