BigStorage is an European Training Network (ETN) whose main goal is to train future data scientists in order to enable them and us to apply holistic and interdisciplinary approaches for taking advantage of a data-overwhelmed world, which requires HPC and Cloud infrastructures with a redefinition of storage architectures underpinning them – focusing on meeting highly ambitious performance and energy usage objectives.

Nowadays there is a lack of professionals who know how to deal with storage, management and analysis of Big Data. Indeed, there is a gap between infrastructures for dealing with Big Data and applications using these volumes of data. In 2011, the McKinsey Global Institute published a study that found that, by 2018, there could be a shortage of up to 190,000 data scientists in the United States, representing a 50 percent to 60 percent gap between supply and demand. Similarly, European officials estimate that 300,000 data scientists will be needed in Europe in the forthcoming years. Other reports, such as those from PRACE and ETP4HPC, have also emphasized the need of skills in HPC, Cloud, Storage, Energy, or Big Data to maintain Europe’s economy. In this context, a major goal of this project is to bring a substantial contribution to the training process of these future experts.

• Improving Energy Efficiency of Scientific Data Compression with Decision Trees (Michael Kuhn, Julius Plehn, Yevhen Alforov, Thomas Ludwig), In ENERGY 2020: The Tenth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, pp. 17–23, IARIA XPS Press, ENERGY 2020, Lisbon, Portugal, ISBN: 978-1-61208-788-7, ISSN: 2308-412X, 2020-09-27
  Publication details – URL

• Towards Green Scientific Data Compression Through High-Level I/O Interfaces (Yevhen Alforov, Anastasiia Novikova, Michael Kuhn, Julian Kunkel, Thomas Ludwig), In 30th International Symposium on Computer Architecture and High Performance Computing, pp. 209–216, IEEE (Los Alamitos, California), SBAC-PAD 2018, Lyon, France, ISBN: 978-1-5386-7769-8, 2018-09
  Publication details – DOI

• Mission possible: Unify HPC and Big Data stacks towards application-defined blobs at the storage layer (Pierre Matri, Yevhen Alforov, Álvaro Brandon, María S. Pérez, Alexandru Costan, Gabriel Antoniu, Michael Kuhn, Philip Carns, Thomas Ludwig), In Future Generation Computer Systems, (Editors: Peter Sloot), Elsevier, 2018-07
  Publication details – URL – DOI

• A Survey of Storage Systems for High-Performance Computing (Jakob Lüttgau, Michael Kuhn, Kira Duwe, Yevhen Alforov, Eugen Betke, Julian Kunkel, Thomas Ludwig), In Supercomputing Frontiers and Innovations, Series: Volume 5, Number 1, pp. 31–58, (Editors: Jack Dongarra, Vladimir Voevodin), Publishing Center of South Ural State University (454080, Lenin prospekt, 76, Chelyabinsk, Russia), 2018-04
  Publication details – URL – DOI

• Could Blobs Fuel Storage-Based Convergence Between HPC and Big Data? (Pierre Matri, Yevhen Alforov, Alvaro Brandon, Michael Kuhn, Philip Carns, Thomas Ludwig), In 2017 IEEE International Conference on Cluster Computing, pp. 81–86, IEEE Computer Society, CLUSTER 2017, Honolulu, USA, ISBN: 978-1-5386-2326-8, 2017-09
  Publication details – URL – DOI

BigStorage: “Storage-based Convergence between HPC and Cloud to handle Big Data” is a H2020 project, reference number 642963 in the call H2020-MSCA-ITN-2014.

Funding period: 2015–2018

• Prof. Dr. Michael Kuhn (contact person)
• Yevhen Alforov

• http://bigstorage-project.eu/