B5: Adaptive, Distributed and Scalable Analysis of Massive Satellite Data

Description

b5_image

Subproject B5 investigates means to improve adaptability and portability of DAWs for large-scale analysis of satellite data. DAWs for such problems are often long and involved and include tasks with very heterogeneous resource requirements (in terms of memory, runtime, and bandwidth). As the manner in which the concrete requirements of tools depend on the input data is a-priori unknown, these DAWs are very difficult to schedule. Today, this problem typically is solved by hardwiring data- and infrastructure dependent scheduling decisions into the DAW. B5 will research new methods for adaptive scheduling of DAWs that adapt to the concrete remote sensing scenario. B5 is an interdisciplinary subproject, led by Prof. Hostert, expert in large-scale satellite data analysis, and Prof. Leser, an expert in workflow management systems for large-scale scientific data analysis.

PIs

Publications

2022

Jonathan Bader; Joel Witzke; Soeren Becker; Ansgar Lößer; Fabian Lehmann; Leon Doehler; Anh Duc Vu; Odej Kao

Towards Advanced Monitoring for Scientific Workflows Inproceedings

In: 2022 IEEE International Conference on Big Data (IEEE BigData 2022), IEEE, 2022.

Links | BibTeX

Jonathan Bader; Fabian Lehmann; Alexander Groth; Lauritz Thamsen; Dominik Scheinert; Jonathan Will; Ulf Leser; Odej Kao

Reshi: Recommending Resources for Scientific Workflow Tasks on Heterogeneous Infrastructures Inproceedings

In: 41th International Performance Computing and Communications Conference 2022, IEEE, 2022.

Links | BibTeX

Jonathan Bader; Fabian Lehmann; Lauritz Thamsen; Jonathan Will; Ulf Leser; Odej Kao

Lotaru: Locally Estimating Runtimes of Scientific Workflow Tasks in Heterogeneous Clusters Inproceedings

In: 34th International Conference on Scientific and Statistical Database Management (SSDBM 2022), pp. 1–12, ACM, 2022.

Links | BibTeX

Marcus Hilbrich; Fabian Lehmann

ßMACH — A Software Management Guidance Inproceedings

In: Reichelt, David Georg; Müller, Richard; Becker, Steffen; Hasselbring, Wilhelm; Hoorn, André; Kounev, Samuel; Koziolek, Anne; Reussner, Ralf (Ed.): Symposium on Software Performance 2021, CEUR-WS, Leipzig, Germany, 2022.

Abstract | Links | BibTeX

David Frantz; Patrick Hostert; Philippe Rufin; Stefan Ernst; Achim Röder; Sebastian Linden

Revisiting the Past: Replicability of a Historic Long-Term Vegetation Dynamics Assessment in the Era of Big Data Analytics Journal Article

In: Remote Sensing, vol. 14, no. 3, 2022, ISSN: 2072-4292.

Abstract | Links | BibTeX

Marcus Hilbrich; Fabian Lehmann

Discussing Microservices: Definitions, Pitfalls, and their Relations Inproceedings

In: 2022 IEEE International Conference on Services Computing (SCC), pp. 39-44, 2022.

Links | BibTeX

2021

Fabian Lehmann; David Frantz; Soeren Becker; Ulf Leser; Patrick Hostert

FORCE on Nextflow: Scalable Analysis of Earth Observation Data on Commodity Clusters Inproceedings

In: CIKM Workshops, 2021.

BibTeX