Continued from B6: Distributed Run-Time Monitoring and Control of Data Analysis Workflows
Description
Data centers have a large and growing contribution to global energy consumption. Factors like manufacturing, construction, disassembly, green energy emissions are frequently neglected in current estimates. The total climate footprint can vary dramatically, based on energy sources, hardware, application, utilization, life-cycle management. B6 aims at a holistic management of end-to-end energy profiles and climate footprints of ML-based data analysis workflows (DAW), as well as system-internal configuration and tuning knobs.
Scientists
- Philipp Ortner
- Ilin Tolovski
Publications
2025
Bodner, Thomas; Rabl, Tilmann
An Interactive Analysis of Serverless Cloud Infrastructure Proceedings Article
In: EDBT, pp. 1146–1149, OpenProceedings.org, 2025.
@inproceedings{bodner2025clouddemo,
title = {An Interactive Analysis of Serverless Cloud Infrastructure},
author = {Thomas Bodner and Tilmann Rabl},
url = {https://www.openproceedings.org/2025/conf/edbt/paper-341.pdf},
doi = {10.48786/edbt.2025.110},
year = {2025},
date = {2025-03-25},
urldate = {2025-01-01},
booktitle = {EDBT},
pages = {1146–1149},
publisher = {OpenProceedings.org},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Bodner, Thomas; Ritter, Daniel; Boissier, Martin; Rabl, Tilmann
Skyrise: Exploiting Serverless Cloud Infrastructure for Elastic Data Processing Journal Article
In: Datenbank-Spektrum, vol. 25, no. 1, pp. 29–38, 2025.
@article{bodner2025skyrise,
title = {Skyrise: Exploiting Serverless Cloud Infrastructure for Elastic Data Processing},
author = {Thomas Bodner and Daniel Ritter and Martin Boissier and Tilmann Rabl},
url = {https://link.springer.com/article/10.1007/s13222-025-00496-7},
doi = {10.1007/s13222-025-00496-7},
year = {2025},
date = {2025-03-13},
urldate = {2025-03-13},
journal = {Datenbank-Spektrum},
volume = {25},
number = {1},
pages = {29–38},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Bodner, Thomas; Boissier, Martin; Rabl, Tilmann; Salazar-Díaz, Ricardo; Schmeller, Florian; Strassenburg, Nils; Tolovski, Ilin; Weisgut, Marcel; Yue, Wang
A Case for Ecological Efficiency in Database Server Lifecycles Proceedings Article
In: Conference on Innovative Data Systems Research (CIDR), www.cidrdb.org, 2025.
@inproceedings{des2025ecological,
title = {A Case for Ecological Efficiency in Database Server Lifecycles},
author = { Thomas Bodner and Martin Boissier and Tilmann Rabl and Ricardo Salazar-Díaz and Florian Schmeller and Nils Strassenburg and Ilin Tolovski and Marcel Weisgut and Wang Yue},
url = {https://www.vldb.org/cidrdb/2025/a-case-for-ecological-efficiency-in-database-server-lifecycles.html},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
booktitle = {Conference on Innovative Data Systems Research (CIDR)},
publisher = {www.cidrdb.org},
abstract = {Like other software systems, database systems benefit from hardware performance improvements. For the longest time, acquiring new hardware resulted in significant software efficiency gains due to exponential improvements of hardware capabilities. Physical limits in hardware manufacturing have brought former niche designs into standard components, such as multiple cores and specialized circuits. Even with these new designs, hardware improvements are decreasing, while software and applications are still becoming increasingly complex and resource demanding. Given the resource consumption of hardware manufacturing, the ideal lifecycle of hardware naturally has to extend from an efficiency aspect. In this paper, we try to estimate efficiency of lifecycle duration of database hardware. We calculate the reduction in performance improvements of hardware using publicly available performance numbers, as well as our own benchmarks, and relate them to the specified thermal design power to get the power efficiency. Incorporating estimations on hardware and power production carbon intensity, we challenge current wisdom on hardware replacement frequencies and try to establish new rules of thumb on the ideal hardware lifecycles for database deployments. We present opportunities for future research trends.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Like other software systems, database systems benefit from hardware performance improvements. For the longest time, acquiring new hardware resulted in significant software efficiency gains due to exponential improvements of hardware capabilities. Physical limits in hardware manufacturing have brought former niche designs into standard components, such as multiple cores and specialized circuits. Even with these new designs, hardware improvements are decreasing, while software and applications are still becoming increasingly complex and resource demanding. Given the resource consumption of hardware manufacturing, the ideal lifecycle of hardware naturally has to extend from an efficiency aspect. In this paper, we try to estimate efficiency of lifecycle duration of database hardware. We calculate the reduction in performance improvements of hardware using publicly available performance numbers, as well as our own benchmarks, and relate them to the specified thermal design power to get the power efficiency. Incorporating estimations on hardware and power production carbon intensity, we challenge current wisdom on hardware replacement frequencies and try to establish new rules of thumb on the ideal hardware lifecycles for database deployments. We present opportunities for future research trends.
Boissier, Martin; Weisgut, Marcel; Rabl, Tilmann
Compression in Main Memory Database Systems: Cost and Performance Trade-Offs of Workload-Driven Data Encoding Proceedings Article
In: BTW, pp. 779–786, Gesellschaft für Informatik e.V., 2025.
@inproceedings{boissier2025compression,
title = {Compression in Main Memory Database Systems: Cost and Performance Trade-Offs of Workload-Driven Data Encoding},
author = {Martin Boissier and Marcel Weisgut and Tilmann Rabl},
url = {https://dl.gi.de/server/api/core/bitstreams/6cde6d62-62fb-4364-949e-08bfc0cdb933/content},
doi = {10.18420/BTW2025-42},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
booktitle = {BTW},
volume = {P-361},
pages = {779–786},
publisher = {Gesellschaft für Informatik e.V.},
series = {LNI},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Schmeller, Florian; Rabl, Tilmann; Graefe, Goetz
Poster: Offset-Value Coding using SIMD Intrinsics Book Section
In: Datenbanksysteme für Business, Technologie und Web - Workshopband (BTW 2025), pp. 323–328, Gesellschaft für Informatik, Bonn, 2025.
@incollection{schmeller2025offset,
title = {Poster: Offset-Value Coding using SIMD Intrinsics},
author = {Florian Schmeller and Tilmann Rabl and Goetz Graefe},
url = {https://dl.gi.de/server/api/core/bitstreams/e7e53cc6-8d7b-4cf6-b0cf-8d21e47a18f7/content},
doi = {10.18420/BTW2025-136},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
booktitle = {Datenbanksysteme für Business, Technologie und Web - Workshopband (BTW 2025)},
pages = {323–328},
publisher = {Gesellschaft für Informatik, Bonn},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Bodner, Thomas; Rabl, Tilmann
A Demonstration of Skyrise: A Serverless Query Processor Proceedings Article
In: BTW, pp. 811–818, Gesellschaft für Informatik e.V., 2025.
@inproceedings{bodner2025skyrisedemo,
title = {A Demonstration of Skyrise: A Serverless Query Processor},
author = {Thomas Bodner and Tilmann Rabl},
url = {https://dl.gi.de/server/api/core/bitstreams/85fcfd70-4a99-4094-b9ea-78d024b6b057/content},
doi = {10.18420/BTW2025-46},
year = {2025},
date = {2025-01-01},
urldate = {2025-01-01},
booktitle = {BTW},
volume = {P-361},
pages = {811–818},
publisher = {Gesellschaft für Informatik e.V.},
series = {LNI},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
2024
Schmeller, Florian; Nugroho, Dwi P. A.; Zeuch, Steffen; Rabl, Tilmann
Towards A GPU-Accelerated Stream Processing Engine Through Query Compilation Proceedings Article
In: Lernen, Wissen, Daten, Analysen. (LWDA '24), 2024.
@inproceedings{SchmellerNZR2024,
title = {Towards A GPU-Accelerated Stream Processing Engine Through Query Compilation},
author = { Florian Schmeller and Dwi P. A. Nugroho and Steffen Zeuch and Tilmann Rabl},
url = {https://www.informatik.uni-wuerzburg.de/fileadmin/1003-lwda24/LWDA_Paper/DB_LWDA_CRC_152.pdf},
year = {2024},
date = {2024-09-01},
urldate = {2024-09-01},
booktitle = {Lernen, Wissen, Daten, Analysen. (LWDA '24)},
abstract = {Over the last decade, data stream processing has emerged to provide real-time insights into large, unbounded volumes of data. At the same time, graphics processing units (GPU) have become an important accelerator for improving the performance of compute-bound applications. Nevertheless, state-of-the-art data streaming systems opt to scale-out and typically do not make efficient use of the underlying hardware. Recent work has shown that query compilation is a viable technique to support hardware advancements in query processing engines. However, it often comes with high development and maintenance costs. In particular, when the process involves hardware accelerators such as GPUs. In this paper, we propose a framework for compiling data stream queries to efficient GPU code in a developer-friendly manner. We demonstrate the feasibility of our framework by integrating it into the data management system NebulaStream. Our experiments show that frequent memory transfers between CPU and GPU impact the query processing throughput.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Over the last decade, data stream processing has emerged to provide real-time insights into large, unbounded volumes of data. At the same time, graphics processing units (GPU) have become an important accelerator for improving the performance of compute-bound applications. Nevertheless, state-of-the-art data streaming systems opt to scale-out and typically do not make efficient use of the underlying hardware. Recent work has shown that query compilation is a viable technique to support hardware advancements in query processing engines. However, it often comes with high development and maintenance costs. In particular, when the process involves hardware accelerators such as GPUs. In this paper, we propose a framework for compiling data stream queries to efficient GPU code in a developer-friendly manner. We demonstrate the feasibility of our framework by integrating it into the data management system NebulaStream. Our experiments show that frequent memory transfers between CPU and GPU impact the query processing throughput.
Salazar-Díaz, Ricardo; Glavic, Boris; Rabl, Tilmann
InferDB: In-Database Machine Learning Inference Using Indexes Journal Article
In: Proceedings of the VLDB Endowment, vol. 17, no. 8, pp. 1830-1842, 2024.
@article{salazar2024inferdb,
title = {InferDB: In-Database Machine Learning Inference Using Indexes},
author = { Ricardo Salazar-Díaz and Boris Glavic and Tilmann Rabl},
url = {https://dl.acm.org/doi/pdf/10.14778/3659437.3659441},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
journal = {Proceedings of the VLDB Endowment},
volume = {17},
number = {8},
pages = {1830-1842},
abstract = {The performance of inference with machine learning (ML) models and its integration with analytical query processing have become critical bottlenecks for data analysis in many organizations. An ML inference pipeline typically consists of a preprocessing workflow followed by prediction with an ML model. Current approaches for in-database inference implement preprocessing operators and ML algorithms in the database either natively, by transpiling code to SQL, or by executing user-defined functions in guest languages such as Python. In this work, we present a radically different approach that approximates an end-to-end inference pipeline (preprocessing plus prediction) using a light-weight embedding that discretizes a carefully selected subset of the input features and an index that maps data points in the embedding space to aggregated predictions of an ML model. We replace a complex preprocessing workflow and model-based inference with a simple feature transformation and an index lookup. Our framework improves inference latency by several orders of magnitude while maintaining similar prediction accuracy compared to the pipeline it approximates.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The performance of inference with machine learning (ML) models and its integration with analytical query processing have become critical bottlenecks for data analysis in many organizations. An ML inference pipeline typically consists of a preprocessing workflow followed by prediction with an ML model. Current approaches for in-database inference implement preprocessing operators and ML algorithms in the database either natively, by transpiling code to SQL, or by executing user-defined functions in guest languages such as Python. In this work, we present a radically different approach that approximates an end-to-end inference pipeline (preprocessing plus prediction) using a light-weight embedding that discretizes a carefully selected subset of the input features and an index that maps data points in the embedding space to aggregated predictions of an ML model. We replace a complex preprocessing workflow and model-based inference with a simple feature transformation and an index lookup. Our framework improves inference latency by several orders of magnitude while maintaining similar prediction accuracy compared to the pipeline it approximates.
Hendrik, Makait.; Monte, Bonaventura Del; Rabl, Tilmann
Ghostwriter: a Distributed Message Broker on RDMA and NVM Proceedings Article
In: 15th International Workshop on Accelerating Analytics and Data Management Systems Using Modern Processor and Storage Architectures, 2024.
@inproceedings{hendrik2024ghostwriter,
title = {Ghostwriter: a Distributed Message Broker on RDMA and NVM},
author = { Makait. Hendrik and Bonaventura Del Monte and Tilmann Rabl},
url = {https://vldb.org/workshops/2024/proceedings/ADMS/ADMS24_04.pdf},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {15th International Workshop on Accelerating Analytics and Data Management Systems Using Modern Processor and Storage Architectures},
volume = {15},
abstract = {Modern stream processing setups heavily rely on message bro- kers such as Apache Kafka or Apache Pulsar. These systems act as buffers and re-readable sources for downstream systems or applica- tions. They are typically deployed on separate servers, requiring extra resources, and achieve persistence through disk-based storage, limiting achievable throughput. In this paper, we present Ghost- writer, a message broker that utilizes remote direct memory access (RDMA) and non-volatile memory (NVM) for highly efficient mes- sage transfer and storage. Utilizing the hardware characteristics of RDMA and NVM, we achieve data throughput that is only limited by the underlying hardware, while reducing computation and dis- aggregating storage and data transfer coordination. Ghostwriter achieves performance improvements of up to an order of magnitude in throughput and latency over state-of-the-art solutions.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Modern stream processing setups heavily rely on message bro- kers such as Apache Kafka or Apache Pulsar. These systems act as buffers and re-readable sources for downstream systems or applica- tions. They are typically deployed on separate servers, requiring extra resources, and achieve persistence through disk-based storage, limiting achievable throughput. In this paper, we present Ghost- writer, a message broker that utilizes remote direct memory access (RDMA) and non-volatile memory (NVM) for highly efficient mes- sage transfer and storage. Utilizing the hardware characteristics of RDMA and NVM, we achieve data throughput that is only limited by the underlying hardware, while reducing computation and dis- aggregating storage and data transfer coordination. Ghostwriter achieves performance improvements of up to an order of magnitude in throughput and latency over state-of-the-art solutions.
Wang, Yue
Efficient Stream Processing in Decentralized Networks Proceedings Article
In: PhD@ VLDB. 2024, 2024.
@inproceedings{wang2024Phd,
title = {Efficient Stream Processing in Decentralized Networks},
author = { Yue Wang},
url = {https://vldb.org/2024/files/phd-workshop-papers/vldb_phd_workshop_paper_id_4.pdf},
year = {2024},
date = {2024-01-01},
urldate = {2024-01-01},
booktitle = {PhD@ VLDB. 2024},
abstract = {Internet-of-things (IoT) devices are widely used in industry as well as in research and are deployed in many applications. These massive amounts of devices are connected in large decentralized networks and produce unbounded data streams with continuous data. To process these data streams timely, current stream processing engines (SPEs) collect all data in a centralized data center. This approach leads to high network utilization and can create a bottleneck in the data center, as all data is transmitted via the network and results are computed centrally. State-of-the-art solutions push down partial window aggregations to machines that are near data streams. However, these solutions are limited to a single simple query. In this paper, we present our work on three solutions for different decentralized aggregations: Desis, Deco, and Dema, which significantly improve the performance of stream processing in decentralized networks. Our solutions reduce network traffic by up to 99.9%.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Internet-of-things (IoT) devices are widely used in industry as well as in research and are deployed in many applications. These massive amounts of devices are connected in large decentralized networks and produce unbounded data streams with continuous data. To process these data streams timely, current stream processing engines (SPEs) collect all data in a centralized data center. This approach leads to high network utilization and can create a bottleneck in the data center, as all data is transmitted via the network and results are computed centrally. State-of-the-art solutions push down partial window aggregations to machines that are near data streams. However, these solutions are limited to a single simple query. In this paper, we present our work on three solutions for different decentralized aggregations: Desis, Deco, and Dema, which significantly improve the performance of stream processing in decentralized networks. Our solutions reduce network traffic by up to 99.9%.