CISL Visitor Program (CVP) Seminar: On the way to robust methods for training Artificial Neural Networks for time integrating PDEs

CISL Visitor Program (CVP) Seminar - On the way to robust methods for training Artificial Neural Networks for time integrating PDEs

Speaker: Dr. Martin Schreiber, Technical University of Munich

Time integration of PDEs to simulate the atmosphere is a non-trivial task once considering wallclock time as well as accuracy constraints. There are claims about machine learning with neural networks being able to approximate high-dimensional spaces as well as non-linearities due to the activation functions used. This motivates an investigation of their suitability to overcome time step restrictions in the context of dynamical cores. 

This work targets exploiting neural networks for temporal integration of PDEs using large time step sizes beyond the CFL condition. We design the underlying neural network with an inspiration from exponential integrators: Instead of using arbitrary off-the-shelf neural networks in a blackbox fashion, we will design neural networks to reflect as a first step the underlying linear terms.First, purely linear one-dimensional problems will be discussed and problems of high errors when using state-of-the-art gradient-based optimizers, which interestingly already exists for linear optimization of neural networks. As a first step towards a robust training, this will be overcome with a reformulation to a linear optimization problem and using a preconditioned conjugate gradient solvers as an optimizer. As a second part, possible extensions to non-linear problems will be discussed. 


Dr. Martin Schreiber started his doctorate in 2010 at the Technical University of Munich where he developed new and cutting edge algorithms for dynamical adaptive meshes leading to certain quasi-optimal properties and highly efficient parallelization on shared as well as distributed memory systems, hence being ready for future (exascale) HPC architectures. He has investigated and collaborated on resource-aware programming on all levels of computer hardware and software development. This has led to various multi-disciplinary publications. In 2015 he was appointed as a proleptic lecturer at the University of Exeter in the Department of Mathematics. Here, he initialized collaborations across a wide field of areas and also represented the university in the OpenPOWER group. In 2018 he rejoined TUM as a researcher and lecturer. His main research focus is on novel time integration methods, including parallel-in-time methods, for large-scale massively parallel high-performance computing architectures enhancing climate and weather simulations.

Date: Tuesday, January 28, 2020 Time: 2:00 p.m. – 3:00 p.m. Location: Mesa Lab, Chapman Room


Room Number: 
ML 245 -Chapman Room

Type of event:

Will this event be webcast to the public by NCAR|UCAR?: 
Announcement Timing: 
Tuesday, January 21, 2020 to Tuesday, January 28, 2020
Calendar Timing: 
Tuesday, January 28, 2020 - 2:00pm to 3:00pm

Posted by Lisa Larson at ext. 1858,

Quick Facts

NCAR is managed by the nonprofit University Corporation for Atmospheric Research on behalf of NSF and the UCAR university community.

  • NCAR is not a federal agency and its employees are not part of the federal personnel system.
  • Our activities complement those of the federal agencies and we work closely with them.

More about us