FONDA members, along with over 100 co-authors from 54 organizations in 17 countries, are proud to present Workflow Systems for Large-Scale Scientific Data Analysis!
The book’s 25 chapters are divided into four main areas (1) Introduction, (2) Systems, (3) Applications, and (4) Technologies, and cover the current state of the art in workflow research for scientific data analysis. The book is open access, and digital copies can be downloaded for free. Hard copies can also be ordered (at the same link) for € 42 each.
Links and author lists for each chapter can be found here:
Chapter 1: The anatomy of scientific workflow management systems
Ulf Leser
Chapter 2: An extended, consolidated view on specification languages for data analysis workflows
Sebastian Müller, Ninon De Mecquenem, Christopher Lazik, Svetlana Kulagina, Jan-Arne Sparka, Fabian Lehmann, Ben Sherman, Marcus Hilbrich, Lars Grunske
Chapter 3: Towards next generation data engineering pipelines
Kevin Kramer, Valerie Restat, Sebastian Strasser, Uta Störl, Meike Klettke
Chapter 4: An ecosystem of services for FAIR computational workflows
Carole Goble, Stian Soiland-Reyes, Finn Bacall, Daniel Garijo, Sean Wilkinson, Johan Gustafsson
Chapter 5: Tackling analytical variability with Workflomics
Vedran Kasalica, Peter Kok, Rob Marissen, Magnus Palmblad, Anna-Lena Lamprecht
Chapter 6: Designing benchmarks for data analysis workflow systems
Rafael Moczalla, Ilin Tolovski, Tilmann Rabl
Chapter 7: Reproducible multi-cloud data analysis with Nextflow
Paolo Di Tommaso, Ben Sherman
Chapter 8: Managing distributed scientific workflows with Globus
Kyle Chard, Ryan Chard, Ian Foster
Chapter 9: Programming task-based workflows with COMPSs
Rosa M Badia, Javier Conejero, Jorge Ejarque, Daniele Lezzi, Francesc Lordan, Raül Sirvent
Chapter 10: Serverless workflow execution models and engines
Maciej Malawski, Bartosz Balis, Tomasz Szydlo, Aleksander Slominski
Chapter 11: Benchmarking and simulating scientific workflow systems: a review
Taina Coleman, Henri Casanova, Federic Suter, Sean Wilkinson, Ketan Maheshwari, Rafael Ferreira da Silva
Chapter 12: Differences in workflow systems: a use-case driven comparison
Vasilis Bountris, Fabian Lehmann, Felix Kummer, Luis Neuhaus, Ulf Leser
Chapter 13: Portable and scalable workflows for earth observation data analysis with Nextflow
Fabian Lehmann, Katarzyna Ewa Lewińska, David Frantz, Florian Katerndahl, Felix Kummer, Dirk Pflugmacher, Patrick Hostert, Ulf Leser
Chapter 14: Reuse and reproduce bioinformatic pipelines using scientific workflow systems
Sarah Cohen-Boulakia, Frederic Lemoine, George Marchment, Marine Djaffardjy, Alban Gaignard, Clémence Sebe, Khalid Belhajjame
Chapter 15: Workflows in materials science
Claudia Draxl, Martin Kuban, Daniel Speckhard, Fabian Peschel, Marcel Schloz, Pablo Robledo, Christoph Koch
Chapter 16: pyiron – developing and managing materials science workflows
Tilmann Hickel, Jan Janßen, Sarath Menon, Osamu Waseda, Liam Huber, Jörg Neugebauer
Chapter 17: Predicting the performance of scientific workflow tasks for cluster resource management: an overview of the state of the art
Jonathan Bader, Sören Becker, Svetlana Kulagina, Fabian Lehmann, Odej Kao, Henning Meyerhenke, Lauritz Thamsen
Chapter 18: Optimizing workflow execution by cost-effective I/O monitoring, bottleneck analysis, and proactive resource assignment
Joel Witzke, Ansgar Lößer, Jonathan Bader, Fabian Lehmann, Björn Scheuermann, Florian Schintke
Chapter 19: From suspicious results to insights: a study on debugging practices in scientific data analysis workflows
Anh Duc Vu, Timo Kehrer, Christos Tsigkanos, Caroline Jay
Chapter 20: Resource allocation of DAWs using mathematical programming Somayeh Mohammadi, Latif Pourkarimi, Somayeh Abdi, Ninon De Mecquenem, Ulf Leser, Knut Reinert
Chapter 21: Reprohackathons: training efforts to increase bioinformatics reproducibility using scientific workflow systems
Sarah Cohen-Boulakia, George Marchment, Thomas Cokelaer, Frederic Lemoine
Chapter 22: Interactivity in scientific workflows: a survey
Nour Elfaramawy, Kedi Cao, Matthias Weidlich
Chapter 23: Provenance in support of workflows for science
Paolo Missier, Débora Pina, Adriane Chapman, Bertram Ludäscher
Chapter 24: Energy-aware workflow execution: an overview of techniques for saving energy and emissions in scientific compute clusters
Lauritz Thamsen, Yehia Elkhatib, Paul Harvey, Syed Waqar Nabi, Jeremy Singer, Wim Vanderbauwhede
Chapter 25: Privacy concerns in workflows and their provenance: where are we?
Ahmad Qadeib Alban, Khalid Belhajjame, Daniela Grigori