In-situ data processing

The main aim of this course is to understand how in-situ visualization models work, and to get to grips with a set of tools for implementing them. The course takes the form of a series of practical exercises, culminating in a mini-project (application of what has been learned in the practical exercises). In particular, we'll be using Paraview software and the Catalyst library. The mini-project will consist in setting up in situ instrumentation on a corpuscular gravitational system (a stellar system in this case). Time permitting, we will also try to instrument a tokamak simulation code (plasma evolution within a nuclear fusion reactor).

C++, Python, OpenMP, MPI

The technologies used will be Python, C++, Paraview and Catalyst (some OpenMP+MPI parallelism). The practical exercises will be based on simple examples, in order to fully grasp Catalyst's data structuring logic and Paraview-Python visualization code generation techniques. The second part of the practical sessions will focus on the use of in-situ visualization with distributed data. The mini-project will enable the previous practical sessions to be applied to an existing simulation code.

The ExaScale era will further widen the gap between the speed of simulation data generation and the speed of writing and reading data for post-processing analysis. Time to results will therefore be greatly impacted, and new data processing techniques need to be put in place. In-situ methods reduce the need to write data by analyzing it directly where it is produced. Several techniques are available, running analyses on the same compute nodes as the simulation (in-situ), using dedicated nodes (in-transit) or combining the two approaches (hybrid). Most in-situ methods target simulations that are unable to take 100% advantage of the increasing number of cores per processor.
The module will focus on in-situ data visualization as part of a scientific application.