A3: Deriving Trust Levels for Multi-Choice Data Analysis Workflows



DAWs often contain series of analysis tasks where for each task a multitude of possible programs exist (multi-choice DAW). A comprehensive characterization of the input data often requires executing several of these programs to combine their respective strengths and avoid their specific weaknesses. However, running all configurations often is not feasible, which raises the problem of finding the best possible combination to increase dependability and to maximize accuracy. A3 will approach this problem by adapting proven quality assurance and software testing techniques to the specific issues in multi-choice DAWs. Important cooperations exist with A6, which also analyzes variants of DAWs, and with B3 in the field of software test techniques for DAWs. A3 is an interdisciplinary project and will develop its methods for the problem of computing physical properties of simulated new materials at extremely large-scale. The subproject will be carried out by Prof. Draxl, one of the main initiators of the NOMAD project, a worldwide unique large-scale repository of comprehensively characterized materials, and Prof. Grunske, an expert in search-based derivation of test cases and robustness estimates for complex software systems.




Martin Kuban; Šimon Gabaj; Wahib Aggoune; Cecilia Vona; Santiago Rigamonti; Claudia Draxl

Similarity of materials and data-quality assessment by fingerprinting Journal Article

In: MRS Bulletin, 2022, ISSN: 1938-1425.

Abstract | Links | BibTeX

H J Kulik; T Hammerschmidt; J Schmidt; S Botti; M A L Marques; M Boley; M Scheffler; M Todorović; P Rinke; C Oses; A Smolyanyuk; S Curtarolo; A Tkatchenko; A P Bartók; S Manzhos; M Ihara; T Carrington; J Behler; O Isayev; M Veit; A Grisafi; J Nigam; M Ceriotti; K T Schütt; J Westermayr; M Gastegger; R J Maurer; B Kalita; K Burke; R Nagai; R Akashi; O Sugino; J Hermann; F Noé; S Pilati; C Draxl; M Kuban; S Rigamonti; M Scheidgen; M Esters; D Hicks; C Toher; P V Balachandran; I Tamblyn; S Whitelam; C Bellinger; L M Ghiringhelli

Roadmap on Machine learning in electronic structure Journal Article

In: Electronic Structure, vol. 4, no. 2, pp. 023004, 2022.

Abstract | Links | BibTeX

Daniel Trübenbach; Sebastian Müller; Lars Grunske

A Comparative Evaluation on the Quality of Manual and Automatic Test Case Generation Techniques for Scientific Software-a Case Study of a Python Project for Material Science Workflows Inproceedings

In: 2022 IEEE/ACM 15th International Workshop on Search-Based Software Testing (SBST), pp. 6–13, IEEE 2022.

Links | BibTeX

Marcus Hilbrich; Sebastian Müller; Svetlana Kulagina; Christopher Lazik; Ninon De Mecquenem; Lars Grunske

A Consolidated View on Specification Languages for Data Analysis Workflows Inproceedings

In: Margaria, Tiziana; Steffen, Bernhard (Ed.): Leveraging Applications of Formal Methods, Verification and Validation. Software Engineering, pp. 201–215, Springer Nature Switzerland, Cham, 2022, ISBN: 978-3-031-19756-7.

Abstract | BibTeX

Sebastian Müller; Valentin Gogoll; Anh Duc Vu; Timo Kehrer; Lars Grunske

Automatically finding Metamorphic Relations in Computational Material Science Parsers Inproceedings

In: 2022 IEEE 18th International Conference on e-Science (e-Science), pp. 521-528, 2022.

Links | BibTeX

Martin Kuban; Santiago Rigamonti; Markus Scheidgen; Claudia Draxl

Density-of-states similarity descriptor for unsupervised learning from materials data Journal Article

In: Scientific Data, vol. 9, no. 1, pp. 646, 2022, ISSN: 2052-4463.

Abstract | Links | BibTeX