File systems of supercomputers are complex systems of hardware and software. They utilize many optimization techniques such as the cache hierarchy to speed up data access. Unfortunately, this complexity makes assessing I/O difficult. It is impossible to predict the performance of a single I/O operation without knowing the exact system state, as optimizations such as client-side caching of the parallel file system may speed up performance significantly. I/O tracing and characterization tools help capturing the application workload and quantitatively assessing the performance. However, a user has to decide himself if obtained performance is acceptable. In this paper, a density-based method from statistics is investigated to build a model which assists administrators to identify relevant causes (a performance factor). Additionally, the model can be applied to purge unexpectedly slow operations that are caused by significant congestion on a shared resource. It will be sketched, how this could be used in the long term to automatically assess performance and identify the likely cause. The main contribution of the paper is the presentation of a novel methodology to identify relevant performance factors by inspecting the observed execution time on the client side. Starting from a black box model, the methodology is applicable without fully understanding all hardware and software components of the complex system. It then guides the analysis from observations and fosters identification of the most significant performance factors in the I/O path. To evaluate the approach, a model is trained on DKRZ's supercomputer Mistral and validated on synthetic benchmarks. It is demonstrated that the methodology is currently able to distinguish between several client-side storage cases such as sequential and random memory layout, and cached or uncached data, but this will be extended in the future to include server-side I/O factors as well.

@techreport{IRFITIUSMK15,
	author	 = {Julian Kunkel},
	title	 = {{Identifying Relevant Factors in the I/O-Path using Statistical Methods}},
	year	 = {2015},
	month	 = {03},
	publisher	 = {Research Group: Scientific Computing, University of Hamburg},
	address	 = {Deutsches Klimarechenzentrum GmbH, Bundesstraße 45a, D-20146 Hamburg},
	series	 = {Research Papers},
	number	 = {3},
	abstract	 = {File systems of supercomputers are complex systems of hardware and software. They
      utilize many optimization techniques such as the cache hierarchy to speed up data access.
      Unfortunately, this complexity makes assessing I/O difficult. It is impossible to predict the
      performance of a single I/O operation without knowing the exact system state, as optimizations
      such as client-side caching of the parallel file system may speed up performance
      significantly. I/O tracing and characterization tools help capturing the application workload
      and quantitatively assessing the performance. However, a user has to decide himself if
      obtained performance is acceptable. In this paper, a density-based method from statistics is
      investigated to build a model which assists administrators to identify relevant causes (a
      performance factor). Additionally, the model can be applied to purge unexpectedly slow
      operations that are caused by significant congestion on a shared resource. It will be
      sketched, how this could be used in the long term to automatically assess performance and
      identify the likely cause. The main contribution of the paper is the presentation of a novel
      methodology to identify relevant performance factors by inspecting the observed execution time
      on the client side. Starting from a black box model, the methodology is applicable without
      fully understanding all hardware and software components of the complex system. It then guides
      the analysis from observations and fosters identification of the most significant performance
      factors in the I/O path. To evaluate the approach, a model is trained on DKRZ's supercomputer
      Mistral and validated on synthetic benchmarks. It is demonstrated that the methodology is
      currently able to distinguish between several client-side storage cases such as sequential and
      random memory layout, and cached or uncached data, but this will be extended in the future to
      include server-side I/O factors as well.},
}