Kesselheim Team

Team leader

HELMHOLTZ AI CONSULTANTS @ FORSCHUNGSZENTRUM JÜLICH

INFORMATION-FOCUSED AI CONSULTANTS

The Helmholtz AI consultant team @ FZJ, led by Stefan Kesselheim, is located at the Jülich Supercomputing Centre (JSC). The team represents the research field ‘Information’ and is associated with exciting research domains such as Neuroscience, Quantum computing and Material Science. We support scientists with our AI competence in all phases of their projects: from sketch, through implementation to communication.

Our mothership is the Jülich Supercomputing Centre that hosts the largest Supercomputers in Germany. We always ask: When can more computational power improve our methods even further? This stimulates our interest in large-scale pre-training, self-supervised methods and generative models. Furthermore, we are looking for techniques to bring prior physical knowledge into the scene. Therefore, we study methods such as differentiable simulation, simulation based inference.

Furthermore, we give support to the FZJ section of the Helmholtz AI Compute Resources (HAICORE).

SELECTED ONGOING VOUCHER PROJECTS

Our most prominent current challenges.

In the Project "Distributed GANs for synthetic Solar images" we train Generative Adversarial Networks (GANs) that create realistic images of the sun in extreme ultraviolet.
Challenge: The Solar Dynamic Observatory (SDO) takes fascinating, beautiful images of the sun, and allows scientists to generate deep insights into dynamic processes within the sun. Imaging generates excellent qualitative insights, but a quantitative understanding is very difficult, as the space of all possible images forms a very complex distribution. With generative models, it is possible to study and understand this distribution in image space.

Approach: In this project our team develops machine learning models that can create realistic high resolution solar images. By exploring the latent space of the model we can systematically study how the model's hidden variables control qualitative and quantitative feature outputs. We combine different state-of-the art methods to obtain high quality images with multiple consistent channels. We study different GAN architectures (StyleGAN, StyleGAN2, StyleALAE, BIGGAN, Unet BIGGAN) and combine these methods with differentiable data augmentation to increase the data-efficiency. We train models highly parallel on many nodes of the JUWELS Cluster at the Jülich Supercomputing Centre.

Collaborators: Ruggero Vasile, Frederic Effenberger (GFZ Potsdam)
In the Project MLWater we explore different techniques for Learning a Neural Network Potential that corrects a classical water model to obtain accurate potential energy surfaces for water molecules in bulk and at interfaces.
Challenge: The simulation of thermodynamic and structural properties of liquids requires accurate model potentials. In the last decade, machine learning based potentials, including the so called Neural Network (NN) potentials from Behler and Parrinello are gaining influence. Here, the intermolecular potential is constructed from so called Atom-Centered Symmetry Functions that obey translational and rotational symmetry that are fed into a multi-layer Neural Network. Training samples are obtained from quantum mechanical simulations. Our collaborators have recently trained a NN potential for water that recovers experimental thermodynamic properties with high accuracy indicating a high quality result.

Approach: In this project, we jointly continue this investigation under two aspects: (a) How can a NN model be combined with a traditional water model? (b) How can the training be implemented in a differentiable framework so that experimental data can be taken into account? In question (a) we would like to learn only the difference between a classical force field and quantum mechanical simulations. This would allow us to add new atom types on the force field level without retraining and to ensure that the electrostatic force is treated on the classical level, including its long range part. For question (b) we would like to change our training and simulation approach to unlock the potential of differentiable simulation. Recently, two very interesting packages that combine training and execution were released: TorchMD and JAX MD. In this project we will provide a first implementation of a training in each framework as a starting point for future developments.
In the project "Understanding Spin-Tune Variations at COSY" we help understanding interesting measurements of the "Spin Tune" with Machine Learning methods at the particle accelerator COSY in Jülich.
Challenge: The COoler SYnchrtron COSY is a particle accelerator at the Institute for Nuclear Physics at Reserach Center Jülich. It generates proton beams in the energy range between 45 and 2700 MeV or deuteron beams between 90 and 2100 MeV for fundamental science experiments. COSY can create spin-polarized beams, where the spin axis of all particles are aligned. The quantity spin tune is defined as the number of spin precessions per turn in the accelerator. Under ideal conditions the spin tune gives a simple relation for the particle speed and a fundamental property of the particle and can be measured with high accuracy at COSY. However, in recent measurements it was apparent that the spin tune does not obey the simple relationship, but deviates slightly as the measurement conditions deviated from ideal conditions. Measurements show that this deviation has a complex dependency on many other measured quantities and can depend on the operation mode of the accelerator.

Approach: In this project, our team applies machine learning methods to contribute to the understanding of these phenomena. We apply ML pipelines based on dimensionality reduction, regularized linear regressions, and Echo State Networks. A key challenge is that the number of experimental data points is very limited. Therefore, the selection of training and test sets must be done very carefully and regularization is a top priority.

Collaborators: Jörg Pretz (FZ Jülich)
In the project "Atmospheric Machine learning Benchmarking System", we develop video prediction methods for weather forecast that can be trained efficiently on the supercomputers.
Challenge: The Atmospheric Machine learning Benchmarking System (AMBS) aims to use deep learning architectures for weather forecasting. It applies methods, developed originally for video prediction, to the prediction of atmospheric quantities such as temperatures. The AMBS is flexible with respect to the deep learning architecture that is used in training and testing. But, due to big data volumes and the complexity of the deep learning models, AMBS requires parallel data handling and parallel training methods on High Performance Computing (HPC) systems.

Approach: Two challenges must be solved to handle large deep learning models and large data volumes. On the one hand the implementation must be parallelized such that it can work efficiently on large-scale parallel machines. Furthermore, data parallel training alters the training mechanism itself. It requires to increase the batch size of the optimization procedure to grow proportionally to the number of involved accelerators. This can lead to a performance degradation. Our team supports both of these aspects. We parallelize the model training using the horovod library. Furthermore, we apply advanced algorithms to mitigate performance loss, such as the LAMB and LARS algorithms.
In the project AlphaNumerics Zero, we work on Reinforcement Learning methods that accelerate the convergence of a numerics method.
Challenge: Recent successes in Reinforcement Learning, such as AlphaGoZero reaching superhuman Go skills or the OpenAI Five playing Dota 2 on world class level have been very inspiring. Many researchers try to transfer the success to other fields hoping that computers can act as agents with skills superior to humans. For AlphaNumerics Zero this idea is applied to numerical linear algebra. Many numerical algorithms can be optimized by picking the right “magic” parameters. This can dramatically accelerate the convergence and hence reduce the computational effort. In the project AlphaNumerics Zero (αN0), Reinforcement Learning (RL) is used so that for a specified simulation problem, the computer learns to determine the “optimal” numerical solution method and its parameters by itself. The project focuses on iterative time-stepping schemes, i.e. specially spectral deferred correction (SDC) methods which are especially suited for supercomputers. This class of methods serves as a prototype for many different areas: stationary iterative solvers, preconditioning, parallel multigrid techniques, time integrators, and resilient numerical methods. Progress made here can be converted into progress in these other fields, with a very broad impact.

Approach: We support this project with RL expertise and technical support. In intense and fruitful discussions, we discuss options how to formulate the problem as an ML task, directions like reward shaping and how to discretize the action space. We provided a first implementation of the framework in JAX and the PPG algorithm, and support the project in exploiting the possibilities of a fully differentiable formulation of the problem.
In the Jülich Challenges project, we support the creation, deployment and launch of a data challenges website for Forschungszentrum Jülich.
Challenge: Scientists from Forschungszentrum Jülich have had great success in organizing data challenges. Hanno Scharr's group from IBG-2 has attracted a lot of attention and very interesting solutions to a plant phenotyping challenge. Combined effort of several researchers from FZJ is built on the idea of creating a platform that can serve as host for challenges in the future. Researchers from fields as diverse as biology, neuroscience and material science join their efforts for creating the platform. In this way, they aim to increase both the visibility of FZJ's research and the visibility of FZJ as a location of great AI research.

Approach: Our team contributes to the Jülich Challenges in many different ways. We have created the website for Jülich Challenges based on the code of the great EvalAI Website. If you are interested, please visit data-challenges.fz-juelich.de. We support scientists in preparing their dataset for the challenges. Finally, we contribute baseline solutions to some of the challenges and sometimes we try to improve the state of the art.

FEATURED PUBLICATIONS

Muzakka, Khoirul Faiq, Sören Möller, Stefan Kesselheim, Jan Ebert, Alina Bazarova, Helene Hoffmann, Sebastian Starke, and Martin Finsterbusch. 2024. “Analysis of Rutherford Backscattering Spectra with CNN-GRU Mixture Density Network.” Scientific Reports 14 (1): 16983. https://doi.org/10.1038/s41598-024-67629-y.

Pargmann, Max, Jan Ebert, Markus Götz, Daniel Maldonado Quinto, Robert Pitz-Paal, and Stefan Kesselheim. 2024. “Automatic Heliostat Learning for in Situ Concentrating Solar Power Plant Metrology with Differentiable Ray Tracing.” Nature Communications 15 (1): 6997. https://doi.org/10.1038/s41467-024-51019-z.

Siebenmorgen, Till, Filipe Menezes, Sabrina Benassou, Erinc Merdivan, Kieran Didi, André Santos Dias Mourão, Radosław Kitel, et al. 2024. “MISATO: Machine Learning Dataset of Protein–Ligand Complexes for Structure-Based Drug Discovery.” Nature Computational Science 4 (5): 367–78. https://doi.org/10.1038/s43588-024-00627-2.

recent papers RG

Ma, Jun, Ronald Xie, Shamini Ayyadhury, Cheng Ge, Anubha Gupta, Ritu Gupta, Song Gu, et al. 2024. “The Multimodality Cell Segmentation Challenge: Toward Universal Solutions.” Nature Methods 21 (6): 1103–13. https://doi.org/10.1038/s41592-024-02233-6.

Nebli, Ahmed, Christian Schiffer, Meiqi Niu, Nicola Palomero-Gallagher, Katrin Amunts, and Timo Dickscheid. 2024. “Generative Modelling of Cortical Receptor Distributions from Cytoarchitectonic Images in the Macaque Brain.” Neuroinformatics, July. https://doi.org/10.1007/s12021-024-09673-7.

Oberstrass, Alexander, Jordan DeKraker, Nicola Palomero-Gallagher, Sascha E. A. Muenzing, Alan C. Evans, Markus Axer, Katrin Amunts, and Timo Dickscheid. 2024. “Analyzing Regional Organization of The Human Hippocampus in 3D-PLI Using Contrastive Learning and Geometric Unfolding.” In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), 1–5. https://doi.org/10.1109/ISBI56570.2024.10635467.

Kropp, Jan-Oliver, Christian Schiffer, Katrin Amunts, and Timo Dickscheid. 2024. “Denoising Diffusion Probabilistic Models for Image Inpainting of Cell Distributions in The Human Brain.” In 2024 IEEE International Symposium on Biomedical Imaging (ISBI), 1–5. https://doi.org/10.1109/ISBI56570.2024.10635384.

Kesselheim Team // Information

HELMHOLTZ AI CONSULTANTS @ FORSCHUNGSZENTRUM JÜLICH

INFORMATION-FOCUSED AI CONSULTANTS

The team

Stefan Kesselheim

Alexandre Strube

Jan Ebert

Alina Bazarova

Sabrina Benassou

Fritz Niesel

SELECTED ONGOING VOUCHER PROJECTS

FEATURED PUBLICATIONS

Ready to dive in?

Kesselheim Team // Information

HELMHOLTZ AI CONSULTANTS @ FORSCHUNGSZENTRUM JÜLICH

INFORMATION-FOCUSED AI CONSULTANTS

Stefan Kesselheim

Alexandre Strube

Jan Ebert

Alina Bazarova

Sabrina Benassou

Fritz Niesel

Project SunGAN

MLWater - Delta Learning of water potential energy surfaces

Understanding Spin-Tune Variations at COSY

Atmospheric Machine learning Benchmarking System (AMBS)

AlphaNumerics Zero

Jülich Challenges

Our top publications