====== Workshop: Exascale I/O for Unstructured Grids (EIUG) ====== When dealing with large amounts of data, the I/O subsystem of modern Supercomputers is one of the most sensible components with regards to performance. Today, application scientists from different backgrounds have to struggle with the details of parallel file systems to obtain optimal performance. Only to a certain extend the management of the data is supported by libraries like HDF5. An especially challenging example is the storage of results computed on unstructured grids, maybe including grid adaptivity, occurring for example in computations in structural mechanics, cosmology or climate simulations. The workshop will bring together experts from application development, middleware and storage design to discuss the issues of storing and accessing large datasets. While focusing on data for (unstructured) grids, the workshop generally addresses HPC data access. The main goal of the workshop is the identification of future proof strategies for the efficient access of large data sets with respect to an optimal utilization of modern storage subsystems and memory hierarchies. It is intended to publish the results of the workshop in form of a white paper. || Date || September Monday 25th, Tuesday 26th, 2017|| || Venue || [[https://www.dkrz.de/|DKRZ, Hamburg, Germany]] || || Contact || [[http://www.wr.informatik.uni-hamburg.de/people/julian_kunkel|Julian Kunkel]] || Please join our [[https://wr.informatik.uni-hamburg.de/listinfo/workshop-eiug-17|mailing list]]. This workshop is supported by [[http://www.sppexa.de|SPPEXA]], the [[http://www.dkrz.de|DKRZ]] and powered by [[http://www.esiwace.eu|ESiWACE]] ((ESiWACE has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under Grant Agreement No 675191)) and the [[https://www.vi4io.org|Virtual Institute for I/O]]. {{:events:2017:sppexa.png?250&nolink|}} {{:research:projects:esiwace-logo.png?300&nolink|}} {{:events:2017:euflag.jpg?120}} {{:events:2017:vi4io.png?200&nolink|}} ===== Relation to SPPEXA ===== The efficient handling of the parallel I/O of large datasets is a crucial problem for the operation of exascale computers. For large simulations datasets of Tera- to Petabytes have to been handled in one output operation, while a hole simulation run can produce up to Exabytes, which need to be managed effectively. Simulations which potentially produce large datasets, partly on unstructured grids, occur in a number of SPPEXA projects like EXA-DUNE, AIMES, Terra-Neo, EXASTEEL-2, ExaFSA, EXAHD, EXAMAG and ExaDG. There are also SPPEXA projects with focus on solutions for Exascale-I/O like ADA-FS and AIMES. The workshop will intensify contact between the SPPEXA and other experts from the area of parallel I/O, a relevant part of the HPC community, probably not completely aware of SPPEXA up to now. The workshop will also serve as a starting point for cooperation, e.g. common sessions on international conferences. ===== Organization ===== The workshop is organized by * Julian Kunkel (DKRZ), [[kunkel@dkrz.de]] (AIMES project) * Olaf Ippisch (TU Clausthal), [[olaf.ippisch@tu-clausthal.de]] (EXA-Dune project) * Sebastian Oeste (TU Dresden), [[sebastian.oeste@tu-dresden.de]] (ADA-FS project) ===== Registration ===== To participate, please register online here: [[http://goo.gl/7LJyVR]]. Alternatively, you may send an email to [[kunkel@dkrz.de]]. Note that our seats are limited and we have to confirm your attendance (you may also kindly ask about the status). ===== Accommodation ===== Within walking distance, we recommend the: [[http://www.mercure.com/en/hotel-5394-mercure-hotel-hamburg-mitte/index.shtml|Mercure Hotel Hamburg Mitte]] \\ Address: Schroederstiftstr. 3, 20146 Hamburg - Rotherbaum, Deutschland DKRZ has a customer ID that we can provide to you upon request; this should give you a condition of 79€ per night. Please see the {{ :events:2017:hvv.pdf |PDF}} for transportation from airport to the venue/hotel. ===== Agenda ===== The workshop is scheduled for a duration of two days, with extended discussion time between talks to find a common assessment of the challenges and possible solution strategies. ==== Monday ==== * 09:30 //Coffee// and **Welcome** \\ //Julian Kunkel (DKRZ) -- {{ :events:2017:eiug17-kunkel-welcome.pdf |slides}}// * 10:00 **Evolving Sandia National Labs Engineering Unstructured Grids Codes for Exascale** \\ //Jay Lofstead (Sandia)// -- {{ :events:2017:eiug17-lofstead.pptx |slides}} * 10:40 **Parallel IO in the Met Office LFRic Infrastructure** \\ //Samantha Adams (Metoffice)// -- {{ :events:2017:eiug17-adams.pdf |slides}} * 11:20 **Scalable I/O for the earthquake simulation software SeisSol** \\ //Sebastian Rettenberger (TUM)// -- {{ :events:2017:eiug17-rettenberger.pdf |slides}} * 12:00 //Lunch// * 13:00 **IME: Freedom from Filesystem Foibles** \\ //James Coomer (DDN)// -- {{ :events:2017:eiug17-coomer.pdf |slides}} * 13:40 **Campaign Storage** \\ //Peter Braam (Campaign Storage)// -- slides * 14:20 **Looking into Clouds -- Our approach for the visualization and analysis of large ICON data sets** \\ //Niklas Röber (DKRZ)// -- {{ :events:2017:eiug17-niklas.pdf |slides}} * 15:00 //Break// * 15:30 **I/O requirements for Exascale post-processing** \\ //Julien Jomier (Kitware)// -- {{ :events:2017:eiug17-jomier.pdf |slides}} * 16:10 **Overview of file formats for (unstructured) data** \\ //Anastasiia Novikova (Uni Hamburg)// -- {{events:2017:eiug17-novikova.pdf|slides}} * 16:50 **Discussion: Limitations of current systems** \\ Discussion lead: //Jay Lofstead (Sandia)// -- slides * 18:00 //End// * 19:00 Social event -- sponsored dinner in the [[https://www.scandichotels.com/hotels/germany/hamburg/scandic-hamburg-emporio/restaurant-bar/h20-restaurant?_ga=2.63767343.1242666138.1503907732-1629568269.1503907732|Scandic Hamburg Emporio]] ==== Tuesday ==== * 08:55 **Welcome from the director** \\ //Thomas Ludwig (DKRZ)// * 09:00 **I/O and Scheduling aspects in DEEP-EST** \\ //Norbert Eicker (FZ Jülich)// -- {{ :events:2017:eiu17-eicker.pdf |slides}} * 09:40 **Reduction of data sets through high-level I/O interfaces** \\ Yevhen Alforov // (DKRZ)// -- {{ :events:2017:eiug17-alforov.pdf |slides}} * 10:00 **Crowds and clouds - the potential of using data from smartphones in numerical weather prediction** \\ // Kasper Stener Hintz (University of Copenhagen)// -- {{ :events:2017:eiug17-hintz.pdf |slides}} * 10:20 //Break// * 10:50 **I/O Optimizations in ICON by the DWD** \\ //Nathanael Hübbe (Uni Hamburg, former DWD employee)// -- {{ :events:2017:eiug17-huebbe.pdf |slides}} * 11:20 **Practical I/O Analysis** \\ //Sebastian Oeste (TU Dresden)// -- {{ :events:2017:eiug17-oeste.pdf |slides}} * 12:00 //Lunch// * 13:00 **Large scale I/O benchmarking for structured grids and implications for unstructured grids** \\ //Olaf Ippisch (TU Clausthal)// -- slides * 13:40 **Space and Performance Optimizations of NetCDF-Grids** \\ //Eugen Betke (DKRZ)// -- {{ :events:2017:eiug17-betke.pdf |slides}} * 14:20 **Towards new storage interfaces -- chance or curse?** \\ //Julian Kunkel (DKRZ)// -- {{ :events:2017:eiug-kunkel.pdf |slides}} * 15:00 **Discussion: Solutions, emerging questions ** \\ //Julian Kunkel// -- slides * 16:00 //Farewell//