research:projects:pecoh:start

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
research:projects:pecoh:start [2019-08-07 10:55]
Kai Himstedt
research:projects:pecoh:start [2020-01-17 19:25] (current)
Kai Himstedt [Deliverables]
Line 162: Line 162:
   * {{:​research:​projects:​pecoh:​D1_1-annual-report.pdf|D1.1 Annual Report}}   * {{:​research:​projects:​pecoh:​D1_1-annual-report.pdf|D1.1 Annual Report}}
     * This deliverable contains the first annual report and it includes information about the project progress, status and next steps. For the sake of completeness,​ the report gives a brief overview of the major project goals, the partners and scientific institutions involved in the project. The relevant information about the general organization of the project (e.g. about a delayed project start due to the difficulties in hiring of qualified scientific employees) is also included. The results achieved in the six work packages are listed in more detail in the technical section of the report. The emphasis is on modeling HPC usage costs to calculate costs and statistics for SLURM jobs and on developing an HPC certification program to improve the education of HPC users.     * This deliverable contains the first annual report and it includes information about the project progress, status and next steps. For the sake of completeness,​ the report gives a brief overview of the major project goals, the partners and scientific institutions involved in the project. The relevant information about the general organization of the project (e.g. about a delayed project start due to the difficulties in hiring of qualified scientific employees) is also included. The results achieved in the six work packages are listed in more detail in the technical section of the report. The emphasis is on modeling HPC usage costs to calculate costs and statistics for SLURM jobs and on developing an HPC certification program to improve the education of HPC users.
 +
 +  * {{:​research:​projects:​pecoh:​D2_1-performance-engineering-concepts-and-software-engineering-concepts-for-hpc.pdf|D2.1 Performance Engineering Concepts and Software Engineering Concepts for HPC}}
 +    * This deliverable presents a collection of performance engineering and software engineering concepts that can help scientists to improve the development of scientific software. Additionally included is an evaluation of the concepts against selected criteria in order to show their benefits when applied in scientific programming.
 +
 +  * {{:​research:​projects:​pecoh:​D2_2-code-co-development.pdf|D2.2 Code Co-Development}}
 +    * This deliverable is based on the insight that software engineering methods can increase productivity by providing scaffolding for the collaborative programming,​ reducing the coding errors and increasing the manageability of software. It describes how a subset of software engineering concepts have been chosen for the study that are considered suitable and useful for scientists during their programming tasks. The deliverable also contains a tutorial that teaches the selected concepts. An experience report on the code co-development process is constituted and some solutions for further improvement of the code co-development process are additionally proposed.
  
   * {{:​research:​projects:​pecoh:​D3_1-and-D3_3-modelling-hpc-usage-costs.pdf|D3.1 & D3.3 Modelling HPC Usage Costs}}   * {{:​research:​projects:​pecoh:​D3_1-and-D3_3-modelling-hpc-usage-costs.pdf|D3.1 & D3.3 Modelling HPC Usage Costs}}
     * This deliverable discusses the modelling of HPC usage costs. For this purpose four distinct cost models (from simple to more complex) were developed. Furthermore,​ two tools were developed to apply the four cost models to SLURM jobs: The first tool is supposed to be run from the job-epilogue script and reports the cost of a single job. The second tool is supposed to be run from the command line and calculates costs and statistics for a set of selected jobs based on the accounting records of SLURM.     * This deliverable discusses the modelling of HPC usage costs. For this purpose four distinct cost models (from simple to more complex) were developed. Furthermore,​ two tools were developed to apply the four cost models to SLURM jobs: The first tool is supposed to be run from the job-epilogue script and reports the cost of a single job. The second tool is supposed to be run from the command line and calculates costs and statistics for a set of selected jobs based on the accounting records of SLURM.
  
-  * {{:​research:​projects:​pecoh:​D4_1-hpc-certification-program.pdf|D4.1 HPC Competences and Certi fication ​Program}}+  ​* [[https://​github.com/​pecoh/​cost-modelling|D3.2 Integration of cost-efficiency in SLURM (available via GitHub)]] 
 +    * This deliverable is comprised by the code for reporting job costs. 
 + 
 +  ​* {{:​research:​projects:​pecoh:​D4_1-hpc-certification-program.pdf|D4.1 HPC Competences and Certification ​Program}}
     * This deliverable discusses the identification of HPC competences and a new approach for an HPC certification program. Our approach is based on an HPC skill tree that supports different views on the HPC content by the help of additional attributes to define the level of a skill (Basic, Intermediate,​ or Expert), its suitability for the role of a user, and its suitability for a scientific domain. We strictly separate the certificate definition from content providing (similar to the concept of a "​Zentralabitur"​) and assume that collaborating scientific institutions will complement each other in producing content, whereas the certification board has the power to establish generally accepted certificate definitions and corresponding exams without the burden of being responsible for the content.     * This deliverable discusses the identification of HPC competences and a new approach for an HPC certification program. Our approach is based on an HPC skill tree that supports different views on the HPC content by the help of additional attributes to define the level of a skill (Basic, Intermediate,​ or Expert), its suitability for the role of a user, and its suitability for a scientific domain. We strictly separate the certificate definition from content providing (similar to the concept of a "​Zentralabitur"​) and assume that collaborating scientific institutions will complement each other in producing content, whereas the certification board has the power to establish generally accepted certificate definitions and corresponding exams without the burden of being responsible for the content.
 +
 +  * {{:​research:​projects:​pecoh:​D4_4-online-examination.pdf|D4.4 Online Examination}}
 +    * This deliverable contains the description of the prototypical process to facilitate multiple choice tests for the online examination of HPC skills. It is based on the insight that trust in the generated certificates is important and it describes the overall architecture and security concept of our solution. The process strategy to mitigate potential cheating attempts is based on the creation of a big question pool, the users‘ registration,​ and the technical verifiability,​ based on cryptographic methods, of the tests and the certificates.
 +
 +  * {{:​research:​projects:​pecoh:​D5_1-documentation-of-recommendations.pdf|D5.1 Documentation of Recommendations}}
 +    * This deliverable focuses on increasing the runtime performance of parallel applications by tuning parallel programs without -- or nearly without -- modifying the source code, e.g. by setting appropriate runtime options and selecting the best performing compiler and MPI environment for each specific program. Also included are experiments for finding good settings for using the standard software packages Gaussian and MATLAB in an HPC environment. Furthermore,​ the deliverable describes how switching to an automatic tuning approach using a Black Box Optimizer tool (based on genetic algorithms) can greatly reduce the effort needed in the beginning of the project for manually tuning parallel programs. Recommendations (lessons learned, best practices, ...) are given for the software packages we dealt with.
 +
 +  * [[https://​www.hhcc.uni-hamburg.de/​|D6.1 Web Presence of the Hamburg HPC Competence Center (HHCC)]]
 +    * This deliverable is represented by the HHCC website.
 +
 +  * {{:​research:​projects:​pecoh:​D6_2-collection-of-success-stories.pdf|D6.2 Collection of Success Stories}}
 +    * This deliverable documents the results of pilot studies we established together with scientists as success stories. These studies support re-writing of existing codes and tuning of parallel programs. The pilot studies are selected to represent exemplary applications in order to make the results transferable in generalised form to similar problems. The collection of the success stories covers topics such as encouraging HPC users to use integrated development environments (IDEs) for the program development,​ teaching important software engineering concepts based on a tutorial, finding an insidious bug in a large Fortran program, achieving performance improvements for R programs, and automatically finding the parameter combinations for building and running parallel applications that give the best benchmark results using a Black Box Optimizer Tool, which is based on genetic algorithms.
 +
  
 ===== Publications ===== ===== Publications =====
Line 200: Line 222:
  
 ^ Date ^ File ^ Context ^ Location ^ ^ Date ^ File ^ Context ^ Location ^
 +| 2019-10-18 | {{ :​research:​projects:​pecoh:​gauss2019-pecoh-slides.pdf | Presentation of the status of the project }} | 9th HPC-Status Conference of the Gauß Allianz | Paderborn (Germany) |
 | 2019-07-31 | {{ :​research:​projects:​pecoh:​hamburg2019-pecoh-slides.pdf | Presentation of the PeCoH project, skill tree, and content production workflow}} | Workshop on HPC Training, Education, and Documentation | Hamburg (Germany) | | 2019-07-31 | {{ :​research:​projects:​pecoh:​hamburg2019-pecoh-slides.pdf | Presentation of the PeCoH project, skill tree, and content production workflow}} | Workshop on HPC Training, Education, and Documentation | Hamburg (Germany) |
 | 2019-06-18 | {{ :​research:​projects:​pecoh:​isc2019-hpccf-bof-slides.pdf | Presentation of the skill tree and the PeCoH project }} | BoF ISC 19: International HPC Certification Program | Frankfurt (Germany) | | 2019-06-18 | {{ :​research:​projects:​pecoh:​isc2019-hpccf-bof-slides.pdf | Presentation of the skill tree and the PeCoH project }} | BoF ISC 19: International HPC Certification Program | Frankfurt (Germany) |
research/projects/pecoh/start.1565168112.txt.gz · Last modified: 2019-08-07 10:55 by Kai Himstedt