talks

Talks

2023

1. SIMULA

   Improving the convergence of pseudo-transient continuation for CFD simulations using neural networks

   Alexander Heinlein

   SciML@Simula workshop (Hybrid), Norway, December 1

   Abs

   Computational fluid dynamics (CFD) simulations are highly relevant for a large range of applications, including but not restricted to the fields of aerospace, environmental, and biological engineering as well as weather predictions and medicine. Modeling Newtonian fluids involves the solution of the Navier-Stokes equations, which, depending on the Reynolds number of the flow, may exhibit a highly nonlinear behavior. The system of nonlinear equations resulting from the discretization of the Navier-Stokes equations can be solved using nonlinear iteration methods, such as Newton’s method. However, fast quadratic convergence is typically only obtained in a local neighborhood of the solution, and for many configurations, the classical Newton iteration does not converge at all. In such cases, so-called globalization techniques may help to improve convergence. In this talk, pseudo-transient continuation is employed in order to improve nonlinear convergence. The classical algorithm is enhanced by a neural network model that is trained to predict a local pseudo-time step. Generalization of the novel approach is facilitated by predicting the local pseudo-time step separately on each element using only local information on a patch of adjacent elements as input. Numerical results for standard benchmark problems, including flow through a backward facing step geometry and Couette flow, show the performance of the machine learning-enhanced globalization approach; as the software for the simulations, the CFD module of COMSOL Multiphysics® is employed.

2. DUCOMS

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Dutch Computational Sciences (DUCOMS) Day 2023, Netherlands, November 11

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems.

3. TUG 2023

   Fast and Robust Overlapping Schwarz (FROSch) Preconditioners in Trilinos – New Developments and Applications

   Alexander Heinlein

   Trilinos User-Developer Group Meeting 2021 (Hybrid), CSRI, Sandia National Laboratories, Albuquerque, USA, October 30-November 2

   URL Slides
4. DAMUT Colloquium

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, October 4

   Abs

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

5. TU Delft

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Numiercal Analysis group coffee talk. Delft University of Technology, Netherlands, September 8
6. ICIAM

   Efficient Schwarz Preconditioning Techniques for Nonlinear Problems Using FROSch

   Alexander Heinlein

   10th International Congress on Industrial and Applied Mathematics (ICIAM), Waseda University, Tokyo, Japan, August 20-25, 2023

   Abs URL Slides

   FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners in Trilinos. Due an algebraic approach, meaning that the preconditioners can be constructed from a fully assembled matrix, FROSch is applicable to a wide range of problems. This talk is focused on the application to nonlinear problems, including computational fluid dynamics and land ice simulations. Techniques for improving the efficiency and the use of GPU architectures are discussed.

7. Workshop

   Multilevel domain decomposition-based architectures for physics-informed neural networks

   Alexander Heinlein

   Invited talk. Workshop on Scientific Learning and Computing, University of Macau, Macao, China, August 17-18

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

8. DLR

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, German Aerospace Center (DLR), July 18
9. TUM

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, Technical University of Munich, July 13

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based network architectures improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. It is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as the residual of the PDE may be used in the loss function for training the network. The second part of the talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

10. EURODYN

    Temporal homogenisation and parallelisation for the numerical simulation of atherosclerotic plaque growth

    Alexander Heinlein

    XII International Conference on Structural Dynamics (EURODYN 2023), Delft University of Technology, Delft, The Netherlands, July 2-5

    Abs URL

    The numerical simulation of atherosclerotic plaque growth is computationally prohibitive since it involves a complex cardiovascular fluid-structure interaction (FSI) problem with a characteristic time scale of milliseconds to seconds as well as a plaque growth process governed by reaction-diffusion equations, which takes place over several months. A resolution of the fast (micro) scale over this period can easily require more than a billion time steps, each corresponding to the solution of a computationally expensive FSI problem. To tackle this problem, we combine a temporal homogenization approach with parallel time-stepping. First, a temporal homogenization approach is developed, which separates the problem in an FSI problem on the micro scale and a reaction-diffusion problem on the macro scale. The approach is analyzed in detail for a simplified flow problem and estimates for the homogenization error and the discretization errors on both time scales are given. Second, a parallel time-stepping approach based on the parareal algorithm is applied on the macro scale of the homogenized system. We investigate modifications in the coarse propagator of the parareal algorithm to further reduce the number of expensive micro problems to be solved and test the numerical algorithms in detailed numerical studies.

11. EuroTUG2023

    Trilinos Tutorial

    Alexander Heinlein

    European Trilinos User Group Meeting 2023 (EuroTUG2023), TU Delft, June 28

    URL Slides
12. 4TU.AMI

    SRI Bridging numerical analysis and machine learninge

    Alexander Heinlein

    4TU.AMI summer event 2023, TU Delft, June 27

    URL
13. COMINDS

    Multilevel domain decomposition-based architectures for physics-informed neural networks

    Alexander Heinlein

    Invited talk. Optimization Workshop (Thematic Einstein Semester on Optimization and Machine Learning), Humboldt Universität zu Berlin, Berlin, June 14-16

    Abs

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

14. HPC Summer School

    Advanced Domain Decomposition Methods – Parallel Schwarz Preconditioning and an Introduction to FROSch

    Alexander Heinlein

    DCSE Summerschool: Numerical Linear Algebra on High Performance Computers, TU Delft, June 5-9

    URL Slides
15. COMINDS

    Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Invited plenary talk. GAMM Workshop on Computational and Mathematical Methods in Data Science, Center for Data and Simulation Science, University of Cologne, Cologne, Germany, May 4-5

    Abs URL

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based network architectures improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. It is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as the residual of the PDE may be used in the loss function for training the network. The second part of the talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

16. CFC 2023

    Parallel Domain Decomposition Preconditioning Techniques for Incompressible Fluid Flow Problems

    Alexander Heinlein

    22nd IACM Computational Fluids Conference (CFC 2023), Cannes, France, April 25-38

    Abs

    Monolithic GDSW (generalized Dryja–Smith–Wildund) preconditioners are two-level Schwarz domain decomposition preconditioners for block systems. They are robust because they account for the coupling terms in the system matrix on both levels, that is, in the local and coarse problems. Block preconditioners, mostly based on block-diagonal and block-triangular preconditioners, such as the famous SIMPLE (semi-implicit method for pressure linked equations) preconditioner, often yield higher iteration counts wile coming at a lower setup cost compared to monolithic approaches. In this talk, the parallel performance of the different preconditioning techniques for incompressible fluid flow problems is investigated and compared using a finite element implementation based on the FEDDLib finite element software and Schwarz preconditioners from the Trilinos package FROSch.

17. CFC 2023

    Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

    Alexander Heinlein

    22nd IACM Computational Fluids Conference (CFC 2023), Cannes, France, April 25-38

    Abs

    Computational fluid dynamics (CFD) simulations are important in many application areas, such as civil and mechanical engineering, meteorology, geosciences, or medical science. However, accurate simulation results come at high computational costs, and they require high-quality meshes describing the computational domain. In this talk, a machine learning-based model order reduction approach for predicting flow fields is presented. After an expensive offline phase, where a surrogate model based on a convolutional neural network (U-Net type architecture) is trained using simulation data, the predictions can be performed much faster than classical numerical simulations. Moreover, the predictions do not require the generation of a complex computational mesh. The surrogate model can be 100 or 10,000 times faster on a CPU or GPU, respectively, of a normal workstation. Numerical results investigating the accuracy and efficiency for varying types of geometries from different application areas are presented. Moreover, different aspects to improve the performance such as the choice of more sophisticated loss functions are discussed.

18. Sandia

    Neural networks with physical constraints, domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Seminar talk, Sandia National Laboratories, USA, April 4

    Abs

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as PDE may be used in the loss function for training the network.

19. IGCM 2023

    Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

    Alexander Heinlein

    Invited plenary lecture. Indo-German conference on Computational Mathematics 2023 (IGCM-2023), Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, India, March 27-30

    Abs

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as PDE may be used in the loss function for training the network.

20. NVIDIA/HLRS

    Neural networks with physical constraints, domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Invited talk. NVIDIA/HLRS SciML GPU Bootcamp, Stuttgart (virtual), Germany, April 26-27

    URL Slides
21. SIAM CSE23

    Domain Decomposition Training Strategies for Physics-Informed Neural Networks

    Alexander Heinlein

    SIAM Conference on Computational Science and Engineering (CSE23). RAI Congress Centre, Amsterdam, The Netherlands, February 26 - March 3

    Abs

    Two-level domain decomposition methods, such as two-level Schwarz, finite element tearing and interconnecting - dual primal (FETI-DP), or balancing domain decomposition by constraints (BDDC) methods, are scalable for a large class of homogeneous problems. However, in the presence of highly-heterogeneous coefficients, convergence generally deteriorates. If the coarse space is enhanced by adapted coarse basis functions, robustness can be regained. Similar observations can be made for highly-heterogeneous nonlinear problems, where both the linear and the nonlinear convergence maybe affected; in this context, a combination of nonlinear domain decomposition methods and robust coarse spaces can recover good linear and nonlinear solver performance. In this talk, preconditioning techniques for highly-heterogeneous linear and nonlinear problems will be discussed and numerical results for various problems will be reported; the focus is on Schwarz domain decomposition methods, but many approaches can be - with small modifications - applied to other domain decomposition methods as well.

22. CTU

    Domain decomposition methods for highly heterogeneous problems - Robust coarse spaces and nonlinear preconditioning

    Alexander Heinlein

    Invited seminar talk. Czech Technical University in Prague, Prague, Czech Republic, February 13

    Abs Slides

    Two-level domain decomposition methods, such as two-level Schwarz, finite element tearing and interconnecting - dual primal (FETI-DP), or balancing domain decomposition by constraints (BDDC) methods, are scalable for a large class of homogeneous problems. However, in the presence of highly-heterogeneous coefficients, convergence generally deteriorates. If the coarse space is enhanced by adapted coarse basis functions, robustness can be regained. Similar observations can be made for highly-heterogeneous nonlinear problems, where both the linear and the nonlinear convergence maybe affected; in this context, a combination of nonlinear domain decomposition methods and robust coarse spaces can recover good linear and nonlinear solver performance. In this talk, preconditioning techniques for highly-heterogeneous linear and nonlinear problems will be discussed and numerical results for various problems will be reported; the focus is on Schwarz domain decomposition methods, but many approaches can be - with small modifications - applied to other domain decomposition methods as well.

23. CDS

    Fast and Robust Overlapping Schwarz Preconditioners in Trilinos – Highly Scalable Algorithms and Their Efficient Implementation

    Alexander Heinlein

    Invited seminar talk. Center for Data and Simulation Science (CDS), Universität zu Köln, Köln, Germany, January 18

    Abs URL Slides

    The Trilinos library is an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems on new and emerging high-performance computing (HPC) architectures. It provides a collection of interoperable software packages enabling the development of algorithms reaching parallel scalability up to the largest supercomputers available. This talk will discuss different aspects of Trilinos for the example of the FROSch (Fast and Robust Overlapping Schwarz) preconditioning framework, which is part of the Trilinos package ShyLU. FROSch implements multilevel Schwarz preconditioners, which are algebraic, i.e., which can be constructed using only the fully assembled parallel distributed system matrix. Making use of the software infrastructure of Trilinos, FROSch allows for the parallel solution of extremely large problems. Numerical results for various problems indicating parallel scalability up to more than 200,000 MPI ranks will be presented. Moreover, node-level parallelization on CPUs as well as GPUs using the Kokkos programming model through the Tpetra linear algebra framework will be discussed.

24. SimTech

    Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

    Alexander Heinlein

    Invited seminar talk. SimTech ML-Session, Universität Stuttgart, Stuttgart, Germany, January 11

    Abs Slides

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. One major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using machine learning models and, in particular, neural networks. The network models be can trained in a data-driven or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: physics-informed neural networks (PINNs) and surrogate models based on convolutional neural networks (CNNs). In PINNs, simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of reference data and the residual of the PDE. Challenging applications, such as multiscale problems, require the use of neural networks with high capacity, and the training of the models is often not robust and may take large iteration counts. Therefore, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on CNNs are discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach and similar to other operator learning approaches, a single network is therefore trained to approximate a variety of boundary value problems. This makes the surrogate modeling approach potentially very efficient. As in the PINN approach, data as well as physics may be used in the loss function for training the network.

2022

1. Seminar

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited talk. Scientific Machine Learning seminar series, University of Strathclyde, Glasgow, UK, December 6

   Abs

   One major branch of scientific machine learning (SciML) is the approximation of the solutions of partial differential equations (PDEs) using neural networks. This can be done in a data-driven or physics-informed way, that is, using reference data from simulations or measurements or by optimizing with respect to the residual of the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: domain decomposition techniques for improving the training of physics-informed neural networks and model order reduction techniques based on convolutional neural networks. Both approaches allow for training using a combination of data and physics.

2. LUH2022

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Seminar talk. Leibniz Universität Hannover, Hannover, Germany, December 1

   Abs

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

3. NVIDIA/HLRS

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited talk. NVIDIA/HLRS SciML GPU Bootcamp, Stuttgart (virtual), Germany, October 24-25

   URL Slides
4. SIAM MDS22

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   SIAM Conference on Mathematics of Data Science, SIAM MDS22, San Diego (Hybrid), USA, September 26 - 30

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities. In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. In particular, the geometry of the computational domain is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented.

5. UDE2022

   Robust, algebraic, and scalable solvers based on Schwarz domain decomposition methods

   Alexander Heinlein

   Kolloquium Ingenieurmathematik, Universität Duisburg-Essen, Essen, Germany, September 15
6. GACM2022

   Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

   Alexander Heinlein

   9th GACM Colloquium on Computational Mechanics 2022, Essen, Germany, September 21-23

   Abs URL

   CFD simulations are very costly to compute and have to be repeated if the geometry changes even slightly. Recently there have been a number of attempts to speed up this process using neural networks. Among these is the use of Convolutional Neural Networks (CNN) as surrogate models for CFD simulations with varying geometries. Here, the model is trained on images of high-fidelity simulation results. However, the generation of training data is expensive and this approach usually requires a large data set. Thus, it is of interest to be able to train a CNN in the absence of abundant training data with the help of physical constraints. First results have already been achieved for the heat equation on a fixed geometry and flow problems in parameterizable geometries. In this talk, we present a physics-aware approach to train CNNs as surrogate models for CFD simulations in varying geometries. The employed CNN takes an imags of the geometry as input and returns images of the associated CFD simulation results, i.e., velocity and pressure, as output. Our CNN architecture is based on the structure of U-Net. Since the model is trained on pixel images, it can be applied to a variety of different geometries. We show results for two-dimensional flows around obstacles of varying size and placement and in non-rectangular geometries, esp. arteries and aneurysms.

7. GAMM2022

   Predicting Cardiovascular Flow Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   92nd GAMM Annual Meeting, Aachen, Germany, August 15-19

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In cardiovascular applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities, such as the maximum velocity or the wall shear stresses at certain locations. In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed, and the architecture and hyper parameters are optimized to this application. Two different approaches are compared: a fully supervised approach, where the model is trained using high fidelity simulation data, and a novel approach, where the model is trained based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Then, the framework is applied to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

8. DD XXVII

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Invited plenary lecture. 27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

   Abs URL Slides

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

9. DD XXVII

   Robust Coarse Spaces for Nonlinear Schwarz Methods

   Alexander Heinlein

   27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

   Abs URL

   In this talk, nonlinear left-preconditioning by nonlinear Schwarz domain decomposition methods is considered. In particular, the focus is on the restricted additive Schwarz preconditioned exact Newton (RASPEN) method. It is based on additive Schwarz preconditioned Newton (ASPEN) method, which has been introduced together with the corresponding inexact method, ASPIN. Nonlinear domain decomposition methods can improve nonlinear as well as linear convergence. However, as in linear domain decomposition methods, scalability and robustness for difficult problems, such as highly heterogeneous problems, generally requires adding a coarse space. The coarse space can be integrated in an additive or multiplicative way using a Galerkin projection. This approach has proven to be very flexible, allowing for the use of different types of coarse spaces, such as classical Lagrangian or multiscale finite element method (MsFEM) type coarse spaces. Furthermore, to obtain robustness for heterogeneous problems, spectral coarse spaces, such as the adaptive generalized Dryja–Smith–Wildund (GDSW) coarse space can be employed. The main goal of this talk is to investigate the effectiveness of different coarse spaces, including GDSW, MsFEM, adaptive GDSW, and approximate component mode synthesis-based (OS-ACMS) adaptive coarse spaces for different nonlinear model problems and depending on the domain decomposition as well as heterogeneity of the problem.

10. DD XXVII

    Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

    Alexander Heinlein

    27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

    Abs URL

    Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a test-bed for the surrogate models, flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

11. Colloquium

    Algebraic Schwarz Domain Decomposition Preconditioners – Parallel Scalability and Robustness

    Alexander Heinlein

    Kolloquium des Fachbereichs Mathematik und Informatik, Universität Marburg, Marburg, Germany, July 18
12. EQUADIFF 15

    Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

    Alexander Heinlein

    Equadiff 15, Brno, Czech Republic, July 11-15

    Abs URL

    Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a test-bed for the surrogate models, flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

13. 4TU.AMI

    Parallel Domain Decomposition Solvers & Scientific Machine Learning

    Alexander Heinlein

    4TU.AMI summer event 2022, Eindhoven, The Netherlands, July 5

    URL
14. ICMS@Strathclyde

    A fully algebraic spectral coarse space for overlapping Schwarz methods

    Alexander Heinlein

    Invited talk. ICMS@Strathclyde: Solvers for frequency-domain wave problems and applications, Glasgow, UK, June 20 - 24

    Abs URL Slides

    Discretizing partial differential equations often results in sparse systems of linear equations. High spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, for instance, in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. The development of spectral coarse spaces for domain decomposition methods has been a very active topic within the last decade. However, until recently, the algebraic construction of robust spectral coarse spaces, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information, has still been on open problem. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. An algebraic and robust spectral coarse space is then constructed by solving two eigenvalue problems on each edge of a two-dimensional domain decomposition. One of them is based on optimal local approximation spaces that are also successfully employed in the construction of multiscale discretizations. The resulting condition number bound is independent of the contrast of the coefficient function, indicating the robustness of the method. In order investigate the robustness of this new method numerically, numerical results for different coefficient distributions with large jumps are presented, including typical examples with channels, random distributions, and coefficient distributions generated from the SPE10 benchmark.

15. ECCOMAS2022

    Parallel Schwarz domain decomposition preconditioning and an introduction to FROSch

    Alexander Heinlein

    Introductory lecture. 8th European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS Congress 2022), Oslo, Norway, June 5 - 9

    URL Slides
16. ECCOMAS2022

    Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

    Alexander Heinlein

    8th European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS Congress 2022), Oslo, Norway, June 5 - 9

    Abs URL

    Computational fluid dynamics (CFD) simulations are important in many application areas, such as civil and mechanical engineering, meteorology, geosciences, or medical science. However, accurate simulation results come at high computational costs, and they require high-quality meshes describing the computational domain. In this talk, a machine learning-based model order reduction approach for predicting flow fields is presented. After an expensive offline phase, where a surrogate model based on a convolutional neural network (U-Net type architecture) is trained using simulation data, the predictions can be performed much faster than classical numerical simulations. Moreover, the predictions do not require the generation of a complex computational mesh. The surrogate model can be 10^2 or 10^4 times faster on a CPU or GPU, respectively, of a normal workstation. Numerical results investigating the accuracy and efficiency for varying types of geometries from different application areas are presented. Moreover, different aspects to improve the performance such as the choice of more sophisticated loss functions are discussed.

17. CM2022

    A fully algebraic spectral coarse space for overlapping Schwarz methods

    Alexander Heinlein

    17th Copper Mountain Conference On Iterative Methods (Virtual), April 4 - April 8

    Abs URL

    Discretizing partial differential equations often results in sparse systems of linear equations. High spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, for instance, in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. The development of spectral coarse spaces for domain decomposition methods has been a very active topic within the last decade. However, until recently, the algebraic construction of robust spectral coarse spaces, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information, has still been on open problem. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. An algebraic and robust spectral coarse space is then constructed by solving two eigenvalue problems on each edge of a two-dimensional domain decomposition. One of them is based on optimal local approximation spaces that are also successfully employed in the construction of multiscale discretizations. The resulting condition number bound is independent of the contrast of the coefficient function, indicating the robustness of the method. In order investigate the robustness of this new method numerically, numerical results for different coefficient distributions with large jumps are presented, including typical examples with channels, random distributions, and coefficient distributions generated from the SPE10 benchmark. Furthermore, randomized eigensolvers are employed to improve the efficiency of solving the eigenvalue problems.

18. SIAM PP22

    New Developments of the FROSch Domain Decomposition Solver Package

    Alexander Heinlein

    SIAM Conference on Parallel Processing for Scientific Computing, SIAM PP22, Virtual, February 23-26

    Abs URL

    Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Wildund (GDSW) type coarse spaces. This talk gives an overview of the FROSch software framework as well as current developments in improving the performance of the solvers, for instance, due to the use of inexact subdomain and coarse solvers. Moreover, recent parallel results for challenging applications, such as coupled multiphysics simulations of land ice in Greenland and Antarctica, are presented.

19. Seminar

    Spectral coarse spaces for overlapping Schwarz methods based on energy-minimizing extensions

    Alexander Heinlein

    Seminar, Ruhr-Universität Bochum, January 13

    Abs

    Discretizing partial differential equations often results in sparse linear equation systems, and high spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, e.g., in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. In particular, the generalized eigenvalue problems are based on energy-minimizing extensions corresponding to the interface components. The presented methods have a provable condition number bound, which is independent of the contrast of the coefficient functions. A great challenge is to construct adaptive coarse spaces that are robust but can be built algebraically, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information. In this talk, a novel adaptive coarse space that is both robust and algebraic is introduced. Furthermore, approaches for combining adaptive coarse spaces with nonlinear preconditioning as well as machine learning techniques are discussed. The talk is based on joint work with Axel Klawonn, Jascha Knepper, Martin Lanser, Janine Weber (University of Cologne), Oliver Rheinbach (TU Bergakademie Freiberg), Kathrin Smetana (Stevens Institute of Technology), and Olof Widlund (New York University).

2021

1. PPS Lecture

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Pretty Porous Science Lecture, Universität Stuttgart, December 7

   Abs

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

2. TUG 2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   Trilinos User-Developer Group Meeting 2021 (Virtual), November 30-December 2

   URL Slides
3. DD Summer School

   Exercise session: FROSch

   Alexander Heinlein

   Summer school on advanced DD methods, Politecnico di Milano, Milano, Italy, November 24-26

   Abs URL Slides

   Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Widlund (GDSW) type coarse spaces. In this lab session, the FROSch software framework will be introduced, and its usage will be explained based on simple model problems. The examples provided will allow investigating the influence of important algorithmic aspects of Schwarz methods, such as the variation of the width of the overlap or adding a coarse level, on the convergence of a preconditioned Krylov solver.

4. CIRM 2021

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Invited Lecture. CIRM Conference “Analysis, Control and Numerics for PDE Models of Interest to Physical and Life Sciences”, Levico Terme, Italy, September 20-24

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

5. YIC 2021

   Using inexact subdomain and coarse solvers in FROSch preconditioners

   Alexander Heinlein

   YIC 2021 (Virtual), July 7-9

   Abs URL

   FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners, which is part of Trilinos. Although being a general framework for the construction and combination of Schwarz operators, FROSch currently focusses on preconditioners that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. The computation of the first level is based on the construction of overlapping subdomains, which can be carried out based on the sparsity pattern of the matrix. In addition to that, robust and scalable coarse spaces are constructed from a partition of unity on the domain decomposition interface and energy-minimizing extensions, without the need for additional information. In particular, GDSW (Generalized Dryja-Smith-Widlund) type coarse spaces are considered. In this talk, the use of inexact solvers for the local overlapping problems and the global coarse problem will be investigated. For this purpose, inexact solvers from different packages from the Trilinos framework will be employed, e.g, from Ifpack2, MueLu, or FROSch itself.

6. COUPLED 2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   9th International Conference on Computational Methods for Coupled Problems in Science and Engineering (Virtual), June 14-16

   Abs URL

   Greenland and Antarctic ice sheets store most of the fresh water on earth and mass loss from these ice sheets significantly contributes to sea-level rise. The simulation of temperature and velocity of the ice sheets gives rise to large highly nonlinear systems of equations. The solution of the associated tangent problems, arising in Newton’s method, is challenging also because of the strong anisotropy of the meshes. We first consider simulations of the ice velocity of Antarctica and the ice temperature of Greenland. We use one-level Schwarz preconditioners as well as GDSW (Generalized Dryja-Smith-Widlund) type preconditioners from the Trilinos package FROSch (Fast and Robust Schwarz), scaling up to 32 k processor cores (8 k MPI ranks and 4 OpenMP threads) for the finest Antarctica mesh; the corresponding velocity problem contains 566 M degrees of freedom. We then study the coupled velocity and temperature problem for the Greenland ice sheet. To the best of our knowledge, it is the first time that a scalable monolithic two-level preconditioner has been used for this multiphysics problem. We present strong scaling results, up to 4 k MPI ranks, using a monolithic GDSW type preconditioner with decoupled extensions from the FROSch package.

7. BaNaNa

   Trilinos

   Alexander Heinlein

   BaNaNa talk, SIAM Student Chapter Delft, May 19

   Abs URL

   Trilinos is a collection of more than 50 open-source software packages that can be used as building blocks for all kinds of scientific applications. For example, packages exist for iterative solvers, PDE-constrained optimization problems and automatic differentiation. This talk will give a brief introduction to Trilinos. Special attention is paid to the Tpetra package that facilitates parallel sparse linear algebra.

8. CM2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   20th Copper Mountain Conference On Multigrid Methods (Virtual), March 29 - April 2

   Abs URL

   Greenland and Antarctic ice sheets store most of the fresh water on earth and mass loss from these ice sheets significantly contributes to sea-level rise. The simulation of temperature and velocity of the ice sheets gives rise to large highly nonlinear systems of equations. The solution of the associated tangent problems, arising in Newton’s method, is challenging also because of the strong anisotropy of the meshes. We first consider simulations of the ice velocity of Antarctica and the ice temperature of Greenland. We use one-level Schwarz preconditioners as well as GDSW (Generalized Dryja-Smith-Widlund) type preconditioners from the Trilinos package FROSch (Fast and Robust Schwarz), scaling up to 32 k processor cores (8 k MPI ranks and 4 OpenMP threads) for the finest Antarctica mesh; the corresponding velocity problem contains 566 M degrees of freedom. We then study the coupled velocity and temperature problem for the Greenland ice sheet. To the best of our knowledge, it is the first time that a scalable monolithic two-level preconditioner has been used for this multiphysics problem. We present strong scaling results, up to 4 k MPI ranks, using a monolithic GDSW type preconditioner with decoupled extensions from the FROSch package.

9. GAMM 2020@21

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   91st GAMM Annual Meeting (Virtual), Kassel, Germany, March 15-19

   Abs URL

   Stress distributions in walls of in vivo arteries (transmural stresses) are a major factor for the course of cardiac diseases; for instance, they are a driving force for the process of arteriosclerosis and arteriogenesis. In order to make accurate predictions of realistic stress distributions using numerical fluid-structure interaction (FSI) simulations, the use of appropriate material models for the arterial walls is essential. In the literature, a wide range of material models is used for this purpose. In this talk, several existing material models for arterial walls are compared based on the curved tube FSI benchmark. In the benchmark configuration, the geometry corresponds to a bended idealized coronary artery with only one material layer. Furthermore, two phases are considered: first, the artery is brought to a physiological pressure of 80 mmHg, and second, the inflow profile of one heartbeat is imposed. As material models, linear elasticity, isotropic hyperelasticity, i.e., a Neo-Hookean material model, as well as the nonlinear anisotropic hyperelastic material models are taken into account. Numerical results for both linear elasticity and Neo-Hookean material models differ significantly from the results obtained with nonlinear anisotropic hyperelastic material models, which have been fitted to experimental data of real arteries. Moreover, the performance of the parallel FSI simulations using different material models is investigated.

10. Seminar

    Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

    Alexander Heinlein

    Numerical Analysis Seminar, TU Delft, Online, February 19

    Abs URL

    FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners, which is part of the Trilinos package ShyLU. Although being a general framework for the construction and combination of Schwarz operators, FROSch currently focusses on preconditioners that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension based coarse spaces, such as GDSW (Generalized Dryja-Smith-Widlund) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW coarse spaces, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a Poisson equation to a coupled multi physics simulations of land ice in Antarctica. Furthermore, a brief overview of some related Schwarz preconditioning techniques, which are not implemented in FROSch yet, will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as the application of machine learning techniques in order to improve the efficiency of these adaptive coarse spaces.

11. Colloquium

    Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

    Alexander Heinlein

    Mathematisches Kolloquium, Universität Stuttgart, Online, February 16

    Abs

    Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Widlund (GDSW) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW preconditioners, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a simple Poisson equation to a coupled multiphysics simulations of land ice in Greenland and Antarctica. Furthermore, a brief overview of related Schwarz preconditioning techniques which are currently being developed but not implemented in FROSch yet will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as novel hybrid preconditioning algorithms combining adaptive coarse spaces and machine learning techniques.

12. Workshop

    Flow Predictions Using Convolutional Neural Networks

    Alexander Heinlein

    GAMM CSE Workshop 2021, Online, January 15, 22, and 29

    Abs URL

    Simulations of fluid flow are generally very costly because too low grid resolutions may even lead to qualitatively incorrect solutions. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of Convolutional Neural Networks (CNNs) to predict fluid flow fields is investigated. Therefore, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are successfully used for image recognition and segmentation tasks, are applied in this context. As a model problem, the flow around obstacles with varying shape and size within a channel is considered. Obstacles of certain type are used as training data, and the generalization of the models to other obstacle geometries and sizes is analyzed. Even though, the training of a neural network is expensive, its evaluation is quite cheap compared to fully resolved Computational Fluid Dynamics (CFD) simulations. This results in a multitude of application possibilities for neural networks in this context, especially in time critical settings.

2020

1. DD XXVI

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   26th International Conference on Domain Decomposition Methods (Virtual), Hong Kong, China, December 7-12
2. DD XXVI

   Flow Predictions Using Convolutional Neural Networks

   Alexander Heinlein

   26th International Conference on Domain Decomposition Methods (Virtual), Hong Kong, China, December 7-12
3. Seminar

   Inaugural Lecture

   Alexander Heinlein

   IANS Highlight Seminar, Universität Stuttgart, Germany, October 26
4. SimTech

   SimTech Quickie

   Alexander Heinlein

   Universität Stuttgart, Germany, October 19
5. Workshop

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   Jahrestreffen GAMM Fachausschuss “Computational Biomechanics” 2020, Bildungszentrum Kloster Banz, Germany, September 21-22
6. Workshop

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 3-6
7. Seminar

   Overlapping Schwarz preconditioning techniques for nonlinear problems

   Alexander Heinlein

   Seminar, Humboldt-Universität zu Berlin, Germany, January 29
8. Workshop

   Flow Predictions Using Convolutional Neural Networks

   Alexander Heinlein

   International Workshop on Scientific Machine Learning, Universität zu Köln, Germany, January 8-10

2019

1. ENUMATH 2019

   Flow predictions using convolutional neural network

   Alexander Heinlein

   ENUMATH Conference 2019, Egmond aan Zee, Netherlands, October 2
2. ENUMATH 2019

   FROSch – A framework for parallel Schwarz preconditioners in Trilinos

   Alexander Heinlein

   ENUMATH Conference 2019, Egmond aan Zee, Netherlands, October 2
3. Seminar

   FROSch – A framework for parallel Schwarz preconditioners in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, September 3
4. MAFELAP 2019

   The FROSch (Fast and Robust Overlapping Schwarz) Package in Cardiovascular Simulations

   Alexander Heinlein

   MAFELAP 2019, Brunel University London, England, June 19
5. EuroTUG

   Fast and Robust Overlapping Schwarz: FROSch

   Alexander Heinlein

   European Trilinos User Group Meeting, ETH Zürich, Switzerland, June 11
6. DAMUT Colloquium

   Designing robust and efficient domain decomposition methods for highly heterogeneous problems using local spectral information and machine learning techniques

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, May 8
7. GAMM 90

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   90th GAMM Annual Meeting, Vienna, Austria, February 18-22

2018

1. Workshop

   The FROSch Package in Cardiovascular Simulations

   Alexander Heinlein

   Workshop on Modeling, Simulation and Optimization of the Cardiovascular System, Magdeburg, Germany, October 22-24
2. Seminar

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, EPFL, Lausanne, Switzerland, August 30
3. DD XXV

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Newfoundland, Canada, July 23-27
4. DD XXV

   Three-Level Extensions of the GDSW Overlapping Schwarz Preconditioner

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Canada, July 23-27
5. DD XXV

   Multiscale Discretizations and Coarse Spaces Based on ACMS

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Canada, July 23-27
6. ECCM 6

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   6th European Conference on Computational Mechanics and 7th European Conference on Computational Fluid Dynamics, Glasgow, UK, June 11-15
7. GAMM 89

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   89th GAMM Annual Meeting, Munich, Germany, March 19-23

2017

1. SNL

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, October 9
2. PASC17

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   PASC17 Conference, Lugano, Switzerland, Juni 26-28
3. DLR

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   Workshop of the German Center for Aerospace and the Mathematical Institute at the Universität zu Köln, Cologne, Germany, March 24
4. GAMM 88

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   88th GAMM Annual Meeting, Weimar, Germany, March 6-10
5. NMM

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 13-15
6. DD XIV

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   24th International Conference on Domain Decomposition Methods, Svalbard, Norway, February 6-10

2016

1. GAMM CSE

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   GAMM CSE Workshop 2016, Kassel, September 8-9
2. DLR

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   Workshop of the German Center for Aerospace and the Mathematical Institute at the Universität zu Köln, Cologne, March 14
3. GAMM 87

   A parallel implementation of the approximate component mode synthesis special finite element method in 2D

   Alexander Heinlein

   Joint Annual Meeting of DMV and GAMM, Braunschweig, Germany, March 7-11
4. NMM

   A parallel implementation of the approximate component mode synthesis special finite element method in 2D

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 22-25

2015

1. IST

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   Workshop on Innovative Modeling Techniques for Predictive Medicine, Lisbon, Portugal, November 12
2. ENUMATH 2015

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   ENUMATH Conference 2015, Ankara, Turkey, September 14-18
3. DD XXIII

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   23rd International Conference on Domain Decomposition Methods, Jeju Island, Korea, July 6-10
4. GAMM 86

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   86th GAMM Annual Meeting, Italy, March 23-27
5. NMM

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 23-25

2014

1. Graz

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   International Workshop on Modelling and Simulation in Biomechanics, Graz, Austria, September 15-17
2. NMM

   GDSW - Domain Decomposition for Fluid-Structure Interaction

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 10-12

2022

1. DD XXVII

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Invited plenary lecture. 27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

   Abs URL Slides

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

2021

1. CIRM 2021

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Invited Lecture. CIRM Conference “Analysis, Control and Numerics for PDE Models of Interest to Physical and Life Sciences”, Levico Terme, Italy, September 20-24

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

Talks

2023

1. DAMUT Colloquium

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, October 4

   Abs

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

2. TU Delft

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Numiercal Analysis group coffee talk. Delft University of Technology, Netherlands, September 8
3. DLR

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, German Aerospace Center (DLR), July 18
4. TUM

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, Technical University of Munich, July 13

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based network architectures improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. It is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as the residual of the PDE may be used in the loss function for training the network. The second part of the talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

5. Sandia

   Neural networks with physical constraints, domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Seminar talk, Sandia National Laboratories, USA, April 4

   Abs

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as PDE may be used in the loss function for training the network.

6. CTU

   Domain decomposition methods for highly heterogeneous problems - Robust coarse spaces and nonlinear preconditioning

   Alexander Heinlein

   Invited seminar talk. Czech Technical University in Prague, Prague, Czech Republic, February 13

   Abs Slides

   Two-level domain decomposition methods, such as two-level Schwarz, finite element tearing and interconnecting - dual primal (FETI-DP), or balancing domain decomposition by constraints (BDDC) methods, are scalable for a large class of homogeneous problems. However, in the presence of highly-heterogeneous coefficients, convergence generally deteriorates. If the coarse space is enhanced by adapted coarse basis functions, robustness can be regained. Similar observations can be made for highly-heterogeneous nonlinear problems, where both the linear and the nonlinear convergence maybe affected; in this context, a combination of nonlinear domain decomposition methods and robust coarse spaces can recover good linear and nonlinear solver performance. In this talk, preconditioning techniques for highly-heterogeneous linear and nonlinear problems will be discussed and numerical results for various problems will be reported; the focus is on Schwarz domain decomposition methods, but many approaches can be - with small modifications - applied to other domain decomposition methods as well.

7. CDS

   Fast and Robust Overlapping Schwarz Preconditioners in Trilinos – Highly Scalable Algorithms and Their Efficient Implementation

   Alexander Heinlein

   Invited seminar talk. Center for Data and Simulation Science (CDS), Universität zu Köln, Köln, Germany, January 18

   Abs URL Slides

   The Trilinos library is an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems on new and emerging high-performance computing (HPC) architectures. It provides a collection of interoperable software packages enabling the development of algorithms reaching parallel scalability up to the largest supercomputers available. This talk will discuss different aspects of Trilinos for the example of the FROSch (Fast and Robust Overlapping Schwarz) preconditioning framework, which is part of the Trilinos package ShyLU. FROSch implements multilevel Schwarz preconditioners, which are algebraic, i.e., which can be constructed using only the fully assembled parallel distributed system matrix. Making use of the software infrastructure of Trilinos, FROSch allows for the parallel solution of extremely large problems. Numerical results for various problems indicating parallel scalability up to more than 200,000 MPI ranks will be presented. Moreover, node-level parallelization on CPUs as well as GPUs using the Kokkos programming model through the Tpetra linear algebra framework will be discussed.

8. SimTech

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited seminar talk. SimTech ML-Session, Universität Stuttgart, Stuttgart, Germany, January 11

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. One major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using machine learning models and, in particular, neural networks. The network models be can trained in a data-driven or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: physics-informed neural networks (PINNs) and surrogate models based on convolutional neural networks (CNNs). In PINNs, simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of reference data and the residual of the PDE. Challenging applications, such as multiscale problems, require the use of neural networks with high capacity, and the training of the models is often not robust and may take large iteration counts. Therefore, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on CNNs are discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach and similar to other operator learning approaches, a single network is therefore trained to approximate a variety of boundary value problems. This makes the surrogate modeling approach potentially very efficient. As in the PINN approach, data as well as physics may be used in the loss function for training the network.

2022

1. Seminar

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited talk. Scientific Machine Learning seminar series, University of Strathclyde, Glasgow, UK, December 6

   Abs

   One major branch of scientific machine learning (SciML) is the approximation of the solutions of partial differential equations (PDEs) using neural networks. This can be done in a data-driven or physics-informed way, that is, using reference data from simulations or measurements or by optimizing with respect to the residual of the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: domain decomposition techniques for improving the training of physics-informed neural networks and model order reduction techniques based on convolutional neural networks. Both approaches allow for training using a combination of data and physics.

2. LUH2022

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Seminar talk. Leibniz Universität Hannover, Hannover, Germany, December 1

   Abs

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

3. UDE2022

   Robust, algebraic, and scalable solvers based on Schwarz domain decomposition methods

   Alexander Heinlein

   Kolloquium Ingenieurmathematik, Universität Duisburg-Essen, Essen, Germany, September 15
4. Colloquium

   Algebraic Schwarz Domain Decomposition Preconditioners – Parallel Scalability and Robustness

   Alexander Heinlein

   Kolloquium des Fachbereichs Mathematik und Informatik, Universität Marburg, Marburg, Germany, July 18
5. Seminar

   Spectral coarse spaces for overlapping Schwarz methods based on energy-minimizing extensions

   Alexander Heinlein

   Seminar, Ruhr-Universität Bochum, January 13

   Abs

   Discretizing partial differential equations often results in sparse linear equation systems, and high spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, e.g., in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. In particular, the generalized eigenvalue problems are based on energy-minimizing extensions corresponding to the interface components. The presented methods have a provable condition number bound, which is independent of the contrast of the coefficient functions. A great challenge is to construct adaptive coarse spaces that are robust but can be built algebraically, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information. In this talk, a novel adaptive coarse space that is both robust and algebraic is introduced. Furthermore, approaches for combining adaptive coarse spaces with nonlinear preconditioning as well as machine learning techniques are discussed. The talk is based on joint work with Axel Klawonn, Jascha Knepper, Martin Lanser, Janine Weber (University of Cologne), Oliver Rheinbach (TU Bergakademie Freiberg), Kathrin Smetana (Stevens Institute of Technology), and Olof Widlund (New York University).

2021

1. PPS Lecture

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Pretty Porous Science Lecture, Universität Stuttgart, December 7

   Abs

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

2. Seminar

   Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

   Alexander Heinlein

   Numerical Analysis Seminar, TU Delft, Online, February 19

   Abs URL

   FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners, which is part of the Trilinos package ShyLU. Although being a general framework for the construction and combination of Schwarz operators, FROSch currently focusses on preconditioners that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension based coarse spaces, such as GDSW (Generalized Dryja-Smith-Widlund) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW coarse spaces, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a Poisson equation to a coupled multi physics simulations of land ice in Antarctica. Furthermore, a brief overview of some related Schwarz preconditioning techniques, which are not implemented in FROSch yet, will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as the application of machine learning techniques in order to improve the efficiency of these adaptive coarse spaces.

3. Colloquium

   Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

   Alexander Heinlein

   Mathematisches Kolloquium, Universität Stuttgart, Online, February 16

   Abs

   Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Widlund (GDSW) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW preconditioners, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a simple Poisson equation to a coupled multiphysics simulations of land ice in Greenland and Antarctica. Furthermore, a brief overview of related Schwarz preconditioning techniques which are currently being developed but not implemented in FROSch yet will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as novel hybrid preconditioning algorithms combining adaptive coarse spaces and machine learning techniques.

2020

1. Seminar

   Inaugural Lecture

   Alexander Heinlein

   IANS Highlight Seminar, Universität Stuttgart, Germany, October 26
2. SimTech

   SimTech Quickie

   Alexander Heinlein

   Universität Stuttgart, Germany, October 19
3. Seminar

   Overlapping Schwarz preconditioning techniques for nonlinear problems

   Alexander Heinlein

   Seminar, Humboldt-Universität zu Berlin, Germany, January 29

2019

1. Seminar

   FROSch – A framework for parallel Schwarz preconditioners in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, September 3
2. DAMUT Colloquium

   Designing robust and efficient domain decomposition methods for highly heterogeneous problems using local spectral information and machine learning techniques

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, May 8

2018

1. Seminar

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, EPFL, Lausanne, Switzerland, August 30

2017

1. SNL

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, October 9