PIOsimHD/HDTrace

Computer simulation revolutionizes traditional experimentation providing a virtual laboratory. The goal of high-performance computing is a fast execution of applications since this enables rapid experimentation. Performance of parallel applications can be improved by increasing either capability of hardware or execution efficiency. In order to increase utilization of hardware resources, a rich variety of optimization strategies is implemented in both hardware and software layers. The interactions of these strategies, however, result in very complex systems. This complexity makes assessing and understanding the measured performance of parallel applications in real systems exceedingly difficult.

To help in this task, in with PIOsimHD an innovative event-driven simulator for MPI-IO applications and underlying heterogeneous cluster computers is developed which can help us to assess measured performance. The simulator allows conducting MPI-IO application runs in silico, including the detailed simulations of collective communication patterns, parallel I/O and cluster hardware configurations. The simulation estimates the upper bounds for expected performance and therewith facilitates the evaluation of observed performance.

In addition to the simulator, the comprehensive tracing environment HDTrace offers novel capabilities in analyzing parallel I/O. For example, it allows the internal behavior of MPI and the parallel file system PVFS to be traced. While PIOsimHD replays traced behavior of applications on arbitrary virtual cluster environments, in conjunction with HDTrace it is a powerful tool for localizing inefficiencies, conducting research on optimizations for communication algorithms, and evaluating arbitrary and future systems.

Contact: Julian Kunkel

Publications

  • Using Simulation to Validate Performance of MPI(-IO) Implementations (Julian Kunkel), In Supercomputing, Lecture Notes in Computer Science (7905), pp. 181–195, (Editors: Julian Martin Kunkel, Prof. Dr. Thomas Ludwig, Hans Werner Meuer), Springer (Berlin, Heidelberg), ISC 2013, Leipzig, Germany, ISBN: 978-3-642-38749-4, ISSN: 0302-9743, 2013-06
  • Simulating Application and System Interaction with PIOsimHD (Julian Kunkel, Prof. Dr. Thomas Ludwig), In Proceedings of the Work in Progress Session, 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, SEA-Publications (31), (Editors: Erwin Grosspietsch, Konrad Klöckner), Institute for Systems Engineering and Automation (Johannes Kepler University Linz), PDP 2012, Munich Network Management Team, Garching, Germany, ISBN: 978-3-902457-31-8, 2012
  • Visualization of MPI(-IO) Datatypes (Julian Kunkel, Prof. Dr. Thomas Ludwig), In Applications, Tools and Techniques on the Road to Exascale Computing, Advances in Parallel Computing (22), pp. 473–480, (Editors: Koen De Bosschere, Erik H. D'Hollander, Gerhard R. Joubert, David Padua, Frans Peters), IOS Press (Amsterdam, Berlin, Tokyo, Washington DC), ParCo 2011, University of Ghent, ELIS Department, Ghent, Belgium, ISBN: 978-1-61499-040-6, ISSN: 0927-5452, 2012
  • Simulation-Aided Performance Evaluation of Server-Side Input/Output Optimizations (Michael Kuhn, Julian Kunkel, Prof. Dr. Thomas Ludwig), In 20th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, pp. 562–566, (Editors: Rainer Stotzka, Michael Schiffers, Yiannis Cotronis), IEEE Computer Society (Los Alamitos, Washington, Tokyo), PDP 2012, Munich Network Management Team, Garching, Germany, ISBN: 978-0-7695-4633-9, ISSN: 1066-6192, 2012
  • Simulating parallel programs on application and system level (Julian Kunkel), In Computer Science – Research and Development, Series: Volume 28 Number 2-3, Springer (Berlin, Heidelberg), ISSN: 1865-2042, 2012-06
  • Tool Environments to Measure Power Consumption and Computational Performance (Timo Minartz, Daniel Molka, Julian Kunkel, Michael Knobloch, Michael Kuhn, Prof. Dr. Thomas Ludwig), In Handbook of Energy-Aware and Green Computing (Ishfaq Ahmad, Sanjay Ranka), Chapters: 31, pp. 709–743, Chapman and Hall/CRC Press Taylor and Francis Group (6000 Broken Sound Parkway NW, Boca Raton, FL 33487), ISBN: 978-1-4398-5040-4, 2012
  • HDTrace – A Tracing and Simulation Environment of Application and System Interaction (Julian Kunkel), Research Papers (2), Research Group: Scientific Computing, University of Hamburg (Deutsches Klimarechenzentrum GmbH, Bundesstraße 45a, D-20146 Hamburg), 2011-01-23 Link