The main goal of team T5 is to publish a comprehensive benchmark for DAW processing systems and apply it to FONDA’s subprojects. As far as our research shows that the scientific community is currently missing a benchmark specification including a representative set of real-life DAWs. T5 will fill this gap such that approaches that target DAWs as a use-case can be compared in an objective way. The foundation of the benchmark will be DAWs collected and developed by the subprojects of FONDA. As a final verification step all subprojects apply the benchmark to compare there approaches with the state-of-the-art.
Collaborations
- S1 provides the infrastructures and links existing publicly available sets of DAWs.
- A2 provides DAWs for genome data analyzes.
- A3 provides DAWs from the computational material science domain and empirical evaluation results with focus on verification and validation.
- A5 provides DAWs for biomedical image analyzes.
- A6 provides DAWs that analyse large sets of genomes.
- B1 uses the developed micro benchmarks.
- B3 contributes experimental use-cases for the benchmark and develops a benchmark set mainly for error detection.
- B4 provides a monitoring API for the visualization of DAW executions and the utilized infrastructure.
- B5 provides large-scale, self-adapting analyzes of satellite data and contributes to the benchmark design.
- B6 initially supposed to only apply the benchmark for there experiments. Nowadays B6 took over the coordination of the benchmark development.
- T1 applies the benchmark for DAW language models.
- T2 provides provenance traces for the benchmark.
- T3 tests there models and provides soft and hard validity constraints.
Related Publications
2022
PerMA-Bench: Benchmarking Persistent Memory Access Proceedings Article
In: Proceedings of the VLDB Endowment, pp. 2463-2476, 2022.
A Survey of Big Data, High Performance Computing, and Machine Learning Benchmarks Proceedings Article
In: Nambiar, Raghunath; Poess, Meikel (Ed.): Performance Evaluation and Benchmarking, pp. 98–118, Springer International Publishing, Cham, 2022, ISBN: 978-3-030-94437-7.