The group Scientific Computing conducts research and development on high performance storage systems. We develop HPC concepts and apply them to simulation software with a focus on earth system models.

More information about specific projects and our publications can be found on their respective pages.

In high performance computing it is important to consider I/O capacity and bandwidth. A multitude of cluster file systems exist, each with different requirements, interfaces and behaviors. Benchmarks are used to evaluate their performance characteristics for specific use cases. However, due to the fact that the performance of file systems usually depends on the used access patterns it is difficult to compare them with each other. While storing large amounts of data is usually unproblematic, storing a large number of files poses another challenge, because of the associated management overhead. Some applications produce billions of files, pushing file systems to their limits. One important factor are file system semantics which can affect the overall performance heavily. The group's focus lies on evaluating their effects and proposing new strategies with regards to these semantics.

Universität Hamburg has become one of five Intel Parallel Computing Centers for Lustre worldwide. The project Enhanced Adaptive Compression in Lustre aims to enable compression within the Lustre filesystem. Since computational power continues to improve at a faster pace than storage capacity and throughput, reducing the amount of data is an important feature. At first, the infrastructure will be prepared to pass through the compressed data and make the backend (ZFS) handle them correctly. This already involves client- as well as server-side changes. Each stripe will be chunked, compressed and sent over the network. Preliminary user space analysis has shown that read-ahead can become a big problem when the chunks are read with logical gaps. The next technical challenge is to integrate the changes into ZFS. Once the infrastructure is done, the actual topic of adaptivity and dynamic decision making will be investigated.

BigStorage was a European Training Network (ETN) whose main goal is to train future data scientists in order to enable them to apply holistic and interdisciplinary approaches for taking advantage of a data-overwhelmed world. This requires HPC and cloud infrastructures with a redefinition of storage architectures underpinning them, while focusing on meeting highly ambitious performance and energy usage objectives. According to the main objectives of BigStorage, power-saving and energy-efficient data reduction solutions and approaches for measuring and modeling power consumption were examined. Work on a framework for energy-efficient compression of scientific data is still ongoing even after the end of the project. It makes use of machine learning to find optimal data reduction strategies.

Contact: Michael Kuhn

For the use of HPC environmental modelling plays an important role. Climate models are well known as typical users of HPC infrastructure. Nevertheless, a number of other environmental modelling aspects are also reliable on the access to both, high computational power and large storage facilities for the simulation results. At our group models representing the ecosystem of the North Sea are in the focus of environmental modelling activities. For example, based on the hydrodynamical model HAMSOM (Hamburg Shelf Ocean Model) the effects of offshore wind farm installations are looked at. The changes in the marine environment is analyzed in relation to the wake effect, which results from the rotation of the propellers. The calculation of the nearly one million wet grid points from the North Sea topography needs up to date computational power and the possibility to store large volumes of simulation results.

Another example of our work is the EU-Project Coastal Biomass Observatory Services (CoBiOS): CoBiOS aims towards the integration of satellite products and ecological models into a user-relevant information service to predict the development of high biomass algal blooms in North Europe’s coastal waters. These blooms can be potentially harmful because, when they decay, they can consume most of the oxygen present in the water causing dead zones. The project needs ready at hand HPC resources and a fit for purpose scheduled interaction between satellite derived data products and ecosystem models including large data storage capacities.

Contact: Dr. Hermann Lenhart