talks

Talks

2025

1. ACOMEN2025

   Adaptive sampling for physics-informed neural networks

   Alexander Heinlein

   9th International Conference on Advanced COmputational Methods in ENgineering and Applied Mathematics, Ghent, Belgium, September 15-19

   Abs URL

2. DD29

   Domain decomposition for physics-informed neural networks: linear and nonlinear function approximation and operator learning

   Alexander Heinlein

   29th International Conference on Domain Decomposition Methods (DD29), Politecnico di Milano, Milan, Italy, June 23-27, 2025

   Abs URL

   Physics-Informed Neural Networks (PINNs) offer a flexible, mesh-free approach to solving differential equations using neural networks. While their implementation is straightforward, training remains challenging due to sensitivity to weight initialization and hyperparameter selection. Additionally, issues such as scalability, spectral bias, and ill-conditioning pose significant challenges. This talk explores how overlapping domain decomposition (DD) techniques can improve the convergence and efficiency of PINNs. Three primary cases are investigated. First, classical PINNs are discussed, where solutions to specific initial-boundary value problems are approximated using standard feedforward neural networks. To enhance performance, DD-based architectures are introduced. Second, randomized neural networks are considered, where hidden layer weights are randomly initialized and fixed, leaving only the final layer to be trained. For linear differential operators, this approach reduces the problem to a linear least-squares formulation, which can be efficiently solved using classical numerical techniques. In this context, DD-based architectures are implemented alongside overlapping Schwarz preconditioning to accelerate convergence. Finally, improvements in physics-informed neural operators through DD-based architectures are studied. Unlike solving a single initial-boundary value problem, neural operators approximate solution operators for parameterized problems. Numerical experiments conducted on various model problems, including multiscale and wave phenomena, demonstrate the effectiveness of domain decomposition techniques for enhancing PINNs. These results highlight the potential of DD methods to improve computational efficiency and accuracy in physics-informed machine learning.

3. DD29

   TBA

   Alexander Heinlein

   29th International Conference on Domain Decomposition Methods (DD29), Politecnico di Milano, Milan, Italy, June 23-27, 2025

   Abs URL

   TBA

4. Workshop

   Domain Decomposition for Physics-Informed Neural Networks: Linear and Nonlinear Function Approximation and Operator Learning

   Alexander Heinlein

   Invited talk. Workshop on the Statistical Theory of Neural Networks, Egmond aan Zee, The Netherlands, May 5 - 8, 2025

   Abs

   Physics-Informed Neural Networks (PINNs) provide a flexible, mesh-free approach to solving differential equations. While their implementation is straightforward and they hold great potential for high-dimensional, inverse, and nonlinear problems, training remains challenging due to their sensitivity to weight initialization, hyperparameter selection, scalability issues, spectral bias, and ill-conditioning. This talk explores how overlapping domain decomposition (DD) techniques can improve the convergence and efficiency of PINNs. Different strategies for integrating DD with PINNs are investigated. First, classical PINNs are enhanced using multilevel DD-based architectures to improve performance. Additionally, this strategy is combined with multifidelity stacking PINNs for time-dependent problems, demonstrating clear improvements over reference results without DD. Second, randomized neural networks are considered, where hidden layer weights are randomly initialized and fixed, reducing the problem to a linear least-squares formulation for linear differential operators. In this setting, DD-based architectures, combined with overlapping Schwarz preconditioning, accelerate convergence. Finally, improvements in physics-informed neural operators through DD-based architectures are explored, enabling the efficient approximation of solution operators for parameterized problems. Numerical experiments on multiscale and wave phenomena demonstrate the effectiveness of DD techniques in PINNs. These results highlight the potential of DD methods to significantly enhance computational efficiency and accuracy in physics-informed machine learning.

5. SDT Talks

   Towards Physics-Informed Machine Learning-Based Surrogate Models for Challenging Problems

   Alexander Heinlein

   Invited seminar talk. Siemens Simulation and Digital Twin Tech Talks (Virtual), Siemens, April 28, 2025

   Abs

   Surrogate models—machine learning models designed to approximate complex and computationally expensive numerical simulations—are a central topic in scientific machine learning. These models are especially relevant for parametrized problems, where the goal is to predict solutions across a range of parameter configurations. While much existing research focuses on low-dimensional parameterizations, such as constant material properties, high-dimensional parameterizations—spatially or temporally varying initial or boundary conditions, material distributions, or source terms—introduce significant methodological and computational challenges. Notably, these cases offer even greater potential benefits, making them particularly important. This talk focuses on the spatial and temporal challenges of machine learning-based surrogate modeling. We examine difficulties in training such models for problems with complex features, including variable geometries and multiscale structures. Key topics include convolutional neural network-based surrogate models tailored to complex domains, adaptive sampling strategies to improve learning efficiency, and the use of spatio-temporal domain decomposition to enhance the scalability and accuracy of operator-based approaches. Applications in multiscale and wave-dominated phenomena, as well as computational fluid dynamics, will illustrate the effectiveness and potential of these methods.

6. GAMM 2025

   Domain Decomposition for Randomized Neural Networks

   Alexander Heinlein

   95th Annual Meeting of the International Association of Applied Mathematics and Mechanics (GAMM 2025), Poznan, Poland, April 7-11

   Abs URL

   Neural network architectures based on overlapping domain decomposition approaches have emerged as a powerful framework for enhancing the efficiency, scalability, and robustness of physics-informed neural networks (PINNs). In this work, we apply this approach to randomized neural networks (RaNNs) for solving partial differential equations (PDEs). A separate neural network is independently initialized on each subdomain using a uniform distribution, and the networks are combined via a partition of unity. Unlike classical PINNs, only the final layers of these networks are trained, which strongly impacts the resulting optimization problem. The resulting optimization problem, for linear PDEs, reduces to a least-squares formulation, which can be solved using direct solvers for small systems or iterative solvers for larger ones. However, the least-squares problems are generally ill-conditioned, and iterative solvers converge slowly without appropriate preconditioning. To address this, we first apply singular value decomposition (SVD) to remove components with low singular values, improving the conditioning of the system. Additionally, we employ a second type of overlapping domain decomposition in the form of additive and restricted additive Schwarz preconditioners for the least-squares problem, further enhancing solver efficiency. Numerical experiments demonstrate that this dual use of domain decomposition significantly reduces computational time while maintaining accuracy, particularly for multi-scale and time-dependent problems.

7. Graphs&Data

   Algebraic Domain Decomposition Solvers for Large-Scale Problems Using Graph Techniques

   Alexander Heinlein

   Invited seminar talk. Graphs&Data@TUDelft Seminar, Delft University of Technology, April 3

   Abs Slides

   Domain decomposition methods (DDMs) solve science and engineering problems by decomposing them into smaller subproblems defined on an overlapping or non-overlapping decomposition of the computational domain. Overlapping Schwarz DDMs are particularly effective for complex problems. However, achieving fast convergence in large-scale problems with large numbers of subdomains requires incorporating additional coarse solver components. The FROSch (Fast and Robust Overlapping Schwarz) package, part of the Trilinos library, implements algebraic domain decomposition methods (DDMs). These methods require only the fully assembled system matrix and minimal geometric information, making them versatile for various applications. The construction of FROSch solvers begins with a non-overlapping domain decomposition obtained via graph partitioning, expanding into overlapping subdomains based on the system matrix’s sparsity pattern. The coarse scale components are constructed from local solutions of the non-overlapping domain decomposition. This talk covers FROSch’s features and recent updates, including the use of inexact local solvers on GPUs and CPUs for faster computation, extending the framework to multiscale and multiphysics problems, and employing machine learning techniques, such as graph neural networks, to improve coarse grid construction. Performance is evaluated on various problems, from simple models to real-world applications.

8. ASML

   Domain Decomposition Methods for Scientific Computing and Machine Learning

   Alexander Heinlein

   Invited seminar talk. Diagnostics & Data Science Seminar, ASML, Eindhoven, March 19

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field that combines scientific computing with machine learning. Addressing complex problems in computational science and engineering (CSE) — often involving multiple scales and physics — requires robust, scalable, and efficient numerical solvers. Domain decomposition methods (DDMs) tackle such problems by decomposing them into smaller subproblems on overlapping or non-overlapping subdomains. DDMs have significantly influenced the development of numerical methods for differential equations for many years. Originating in the 19th century as a theoretical tool for exploring solutions to partial differential equations on complex domains, their divide-and-conquer strategy has proven well-suited for large-scale problems and parallel computing. Nowadays, DDMs form the foundation of many modern numerical solvers of both direct and iterative nature. Recently, DDMs have been integrated with machine learning techniques to enhance their performance and robustness. This underscores the continuing relevance and importance of DDMs in advancing computational methods. This talk discusses overlapping domain decomposition methods for tackling challenging CSE problems, high-performance computing (HPC) for large-scale problems, and state-of-the-art data-driven and physics-informed machine learning methods.

9. ISCL

   Domain decomposition for physics-informed neural networks: linear and nonlinear function approximation and operator learning

   Alexander Heinlein

   Invited seminar talk. ISCL Seminar Series, Pennsylvania State University, USA, March 14

   Abs URL Slides Video

   Physics-Informed Neural Networks (PINNs) provide a flexible, mesh-free approach to solving differential equations. While their implementation is straightforward and they hold great potential for high-dimensional, inverse, and nonlinear problems, training remains challenging due to their sensitivity to weight initialization, hyperparameter selection, scalability issues, spectral bias, and ill-conditioning. This talk explores how overlapping domain decomposition (DD) techniques can improve the convergence and efficiency of PINNs. Different strategies for integrating DD with PINNs are investigated. First, classical PINNs are enhanced using multilevel DD-based architectures to improve performance. Additionally, this strategy is combined with multifidelity stacking PINNs for time-dependent problems, demonstrating clear improvements over reference results without DD. Second, randomized neural networks are considered, where hidden layer weights are randomly initialized and fixed, reducing the problem to a linear least-squares formulation for linear differential operators. In this setting, DD-based architectures, combined with overlapping Schwarz preconditioning, accelerate convergence. Finally, improvements in physics-informed neural operators through DD-based architectures are explored, enabling the efficient approximation of solution operators for parameterized problems. Numerical experiments on multiscale and wave phenomena demonstrate the effectiveness of DD techniques in PINNs. These results highlight the potential of DD methods to significantly enhance computational efficiency and accuracy in physics-informed machine learning.

10. SIAM CSE25

    Domain decomposition for physics-informed neural networks and operators

    Alexander Heinlein

    SIAM Conference on Computational Science and Engineering (CSE25), Fort Worth, Texas, U.S., March 3-7

    Abs URL Slides

    Scientific machine learning (SciML) is a rapidly evolving research field that combines techniques from scientific computing and machine learning. This talk focuses on the application of domain decomposition methods to design neural network architectures and enhance neural network training, Specifically, it explores the use of domain decomposition techniques in neural network-based discretizations for solving partial differential equations with physics-informed neural networks (PINNs) and operator learning, as well as in classical machine learning tasks like semantic image segmentation using convolutional neural networks (CNNs). Computational results show that domain decomposition methods can improve efficiency — both in terms of time and memory — as well as enhance accuracy and robustness.

11. AICOMAS

    Domain Decomposition for Randomized Neural Networks

    Alexander Heinlein

    3rd IACM Digital Twins in Engineering Conference (DTE 2025) & 1st ECCOMAS Artificial Intelligence and Computational Methods in Applied Science (AICOMAS 2025), Paris, France, February 17-21

    Abs URL Slides

    Neural network architectures based on overlapping domain decomposition approaches have emerged as a powerful framework for enhancing the efficiency, scalability, and robustness of physics-informed neural networks (PINNs). In this work, we apply this approach to randomized neural networks (RaNNs) for solving partial differential equations (PDEs). A separate neural network is independently initialized on each subdomain using a uniform distribution, and the networks are combined via a partition of unity. Unlike classical PINNs, only the final layers of these networks are trained, which strongly impacts the resulting optimization problem. The resulting optimization problem, for linear PDEs, reduces to a least-squares formulation, which can be solved using direct solvers for small systems or iterative solvers for larger ones. However, the least-squares problems are generally ill-conditioned, and iterative solvers converge slowly without appropriate preconditioning. To address this, we first apply singular value decomposition (SVD) to remove components with low singular values, improving the conditioning of the system. Additionally, we employ a second type of overlapping domain decomposition in the form of additive and restricted additive Schwarz preconditioners for the least-squares problem, further enhancing solver efficiency. Numerical experiments demonstrate that this dual use of domain decomposition significantly reduces computational time while maintaining accuracy, particularly for multi-scale and time-dependent problems.

12. Seminar

    Localization via Partition of Unity Functions – Coarse Spaces for Domain Decomposition Preconditioners and Multilevel Architectures for Neural Networks

    Alexander Heinlein

    Invited plenary talk. International Multigrid Conference (IMG) 2025, King Abdullah University of Science and Technology (KAUST), Saudi Arabia, February 3-5

    URL Slides
13. Seminar

    Domain decomposition and adaptive sampling for physics-informed neural networks

    Alexander Heinlein

    Invited talk. Workshop "Scientific machine learning: error estimation and analysis", Besançon, France, January 15-16

    URL Slides

2024

1. CASML@2024

   Geometric Challenges in Machine Learning-Based Surrogate Models

   Alexander Heinlein

   Invited plenary talk. International Conference on Applied AI and Scientific Machine Learning (CASML 2024), Indian Institute of Science (IISc), Bangalore, India, December 14-18

   Abs URL Slides

   Surrogate models—machine learning models designed to approximate complex and computationally expensive numerical simulations—are a key topic in scientific machine learning. These models are particularly relevant for parametrized problems, predicting solutions across a range of parameter configurations. While much of the existing work focuses on low-dimensional parameterizations, such as constant material properties, higher-dimensional parameterizations—spatially or temporally varying initial or boundary conditions, material distributions, or source terms—present significant methodological and computational challenges. Moreover, these cases are particularly important, as the potential benefits of the resulting surrogate models are even greater. This talk explores geometric aspects of machine learning-based surrogate modeling, focusing on convolutional neural networks and neural operators, such as deep operator networks (DeepONets). Both data-driven and physics-informed loss terms are being considered, and some novel directions are motivated for physics-informed neural networks (PINNs). The presentation focuses on strategies for learning complex spatial solution behaviors, addressing variations in the computational domain, and improving operator performance and scalability through spatio-temporal decomposition. Applications in multiscale and wave problems, as well as computational fluid dynamics, illustrate the potential of these methods.

2. Seminar

   When One Level Is Not Enough – Multilevel Domain Decomposition Methods for Physics and Data-Driven Problems

   Alexander Heinlein

   Invited seminar talk. Heidelberg University, Heidelberg, Germany, December 5

   Abs Slides

   Domain decomposition methods (DDMs) solve boundary value problems by decomposing them into smaller subproblems defined on an overlapping or non-overlapping decomposition of the computational domain. Their divide-and-conquer approach makes DDMs well-suited for large-scale problems and parallel computing. However, achieving robust convergence for challenging problems and scalability to large numbers of subdomains generally requires (global) information transport. This can be achieved by incorporating well-designed coarse levels, transforming DDMs from one- into multi-level algorithms. This talk highlights the importance of using multiple levels in domain decomposition methods. In the first part of the talk, coarse spaces for domain decomposition-based preconditioners will be discussed. They can provide robustness and scalability to Schwarz preconditioners for a wide range of challenging problems exhibiting, for instance, strong heterogeneities, multiple coupled physics, and/or strong nonlinearities. Numerical results using the FROSch (Fast and Robust Overlapping Schwarz) package, which is part of the Trilinos library, demonstrate the effectiveness and efficiency of these Schwarz preconditioners. The second part of the talk will explore the application of DDMs to neural networks (NNs), demonstrating improvements in terms of accuracy, computation time, and/or memory efficiency. Similar to classical domain decomposition methods, coarse levels, here in the form of small global NNs, ensure global information transport, enabling scalability. This talk will cover the application of DDMs in solving partial differential equations using physics-informed NNs (PINNs) and in image segmentation using convolutional NNs (CNNs).

3. CASA

   When One Level Is Not Enough – Multilevel Domain Decomposition Methods for Physics and Data-Driven Problems

   Alexander Heinlein

   Invited seminar talk. CASA colloquium, TU Eindhoven, Eindhoven, The Netherlands, November 20

   Abs URL Slides

   Domain decomposition methods (DDMs) solve boundary value problems by decomposing them into smaller subproblems defined on an overlapping or non-overlapping decomposition of the computational domain. Their divide-and-conquer approach makes DDMs well-suited for large-scale problems and parallel computing. However, achieving robust convergence for challenging problems and scalability to large numbers of subdomains generally requires (global) information transport. This can be achieved by incorporating well-designed coarse levels, transforming DDMs from one- into multi-level algorithms. This talk highlights the importance of using multiple levels in domain decomposition methods. In the first part of the talk, coarse spaces for domain decomposition-based preconditioners will be discussed. They can provide robustness and scalability to Schwarz preconditioners for a wide range of challenging problems exhibiting, for instance, strong heterogeneities, multiple coupled physics, and/or strong nonlinearities. Numerical results using the FROSch (Fast and Robust Overlapping Schwarz) package, which is part of the Trilinos library, demonstrate the effectiveness and efficiency of these Schwarz preconditioners. The second part of the talk will explore the application of DDMs to neural networks (NNs), demonstrating improvements in terms of accuracy, computation time, and/or memory efficiency. Similar to classical domain decomposition methods, coarse levels, here in the form of small global NNs, ensure global information transport, enabling scalability. This talk will cover the application of DDMs in solving partial differential equations using physics-informed NNs (PINNs) and in image segmentation using convolutional NNs (CNNs).

4. Seminar

   Domain Decomposition for Neural Networks: Physics-Informed Neural Networks, Operator Learning, and Image Segmentation

   Alexander Heinlein

   Invited seminar talk. Mathematics of Data Science seminar, University of Twente, Enschede, The Netherlands, November 4

   Abs

   Scientific machine learning (SciML) is a rapidly evolving research field that combines techniques from scientific computing and machine learning. This talk focuses on the application of domain decomposition methods to design neural network architectures and enhance neural network training. In particular, it explores how domain decomposition techniques can be employed in neural network-based discretizations to address forward and inverse problems involving partial differential equations using physics-informed neural networks (PINNs) as well as in neural operators. Furthermore, the talk explores the use of domain decomposition methods for traditional machine learning tasks, such as semantic image segmentation with convolutional neural networks (CNNs). Computational results show that domain decomposition methods can improve efficiency—both in terms of time and memory—as well as enhance accuracy and robustness.

5. Seminar

   Domain decomposition for neural networks

   Alexander Heinlein

   Invited seminar talk. University of Konstanz, Konstanz, Germany, October 30

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving research field that combines techniques from scientific computing and machine learning. This talk focuses on the application of domain decomposition methods to design neural network architectures and enhance neural network training, Specifically, it explores the use of domain decomposition techniques in neural network-based discretizations for solving partial differential equations with physics-informed neural networks (PINNs) and operator learning, as well as in classical machine learning tasks like semantic image segmentation using convolutional neural networks (CNNs). Computational results show that domain decomposition methods can improve efficiency - both in terms of time and memory - as well as enhance accuracy and robustness.

6. IGHASC

   Domain decomposition for neural networks

   Alexander Heinlein

   Indo-German Workshop on Hardware-aware Scientific Computing. Heidelberg University, Heidelberg, Germany, October 28-30

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. In this context, this talks focuses on the enhancement of machine learning using classical numerical methods, in particular, on improving neural networks using domain decomposition-inspired architectures. In the first part of this talk, the domain decomposition paradigm is applied to the approximation of the solutions of partial differential equations (PDEs) using physics-informed neural networks (PINNs). It is observed that network architectures inspired by multi-level Schwarz domain decomposition methods can improve the performance for certain challenging problems, such as multiscale problems. Moreover, a classical machine learning task is considered, that is, image segmentation using convolutional neural networks (CNNs). Domain decomposition techniques offer a way of scaling up common CNN architectures, such as the U-Net. In particular, local subdomain networks learn local features and are coupled via a coarse network which incorporates global features.

7. TU Delft

   Why Domain Decomposition Preconditioning for Highly Heterogeneous Problems is Challenging and How Machine Learning Can Help

   Alexander Heinlein

   Numerical Analysis group coffee talk. Delft University of Technology, Netherlands, October 11
8. Colloquium

   Domain decomposition techniques for high-performance scientific computing

   Alexander Heinlein

   Kolloquium des Fachbereichs Mathematik und Informatik, Universität Marburg, Marburg, Germany, October 10
9. NHR2024

   Domain decomposition for neural networks

   Alexander Heinlein

   Invited talk. NHR Conference 2024, Darmstadt, Germany, September 9-12

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving research field that combines techniques from scientific computing and machine learning. This talk focuses on the application of domain decomposition methods to design neural network architectures and enhance neural network training, Specifically, it explores the use of domain decomposition techniques in neural network-based discretizations for solving partial differential equations with physics-informed neural networks (PINNs) and operator learning, as well as in classical machine learning tasks like semantic image segmentation using convolutional neural networks (CNNs). Computational results show that domain decomposition methods can improve efficiency — both in terms of time and memory — as well as enhance accuracy and robustness.

10. Seminar

    Fast and Robust Overlapping Schwarz (FROSch) Domain Decomposition Preconditioners

    Alexander Heinlein

    Invited seminar talk. University of Macau, Macao, China, August 7

    Abs Slides

    The Schwarz domain decomposition framework is a powerful algorithmic framework for efficiently solving partial differential equations by decomposing a complex global problem into smaller, local subproblems. The FROSch (Fast and Robust Overlapping Schwarz) package, which is part of the Trilinos software library, leverages this framework. Moreover, FROSch employs extension-based coarse spaces to allow for constructing scalable and algebraic multilevel Schwarz preconditioners. In this context, "algebraic" means that the preconditioners can be constructed using only the fully assembled, parallel distributed system matrix. This talk gives an overview of the capabilities of FROSch and delves into recent developments. This includes: 1) exploring the use of inexact local solvers on both GPUs and CPUs to improve computing times; 2) the development of monolithic and adaptive coarse spaces to broaden the range of problems FROSch can tackle; 3) investigations into utilizing machine learning techniques, such as graph neural networks, to improve the construction of coarse bases. The performance of these approaches is evaluated across various problem types, encompassing simple model problems as well as complex multi-physics problems that address real-world applications.

11. Seminar

    Domain decomposition for neural networks

    Alexander Heinlein

    Invited seminar talk. Shenzhen Institutes of Advanced Technology, China, July 31

    Abs Slides

    Scientific machine learning (SciML) is a rapidly evolving research field that combines techniques from scientific computing and machine learning. This talk specifically addresses the application of domain decomposition methods to design neural network architectures and enhance neural network training. The discussion will explore the use of these techniques in neural network-based discretizations for solving partial differential equations with physics-informed neural networks (PINNs) and operator learning, as well as in classical machine learning tasks like semantic image segmentation using convolutional neural networks (CNNs). Computational results show that domain decomposition methods can improve efficiency — both in terms of time and memory — as well as enhance accuracy and robustness.

12. SU

    Domain decomposition for neural networks

    Alexander Heinlein

    Invited talk. Workshop on Scientific Machine Learning, Strasbourg University, Strasbourg, France, July 8-12

    Abs URL Slides

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. In this context, this talks focuses on the enhancement of machine learning using classical numerical methods, in particular, on improving neural networks using domain decomposition-inspired architectures. In the first part of this talk, the domain decomposition paradigm is applied to the approximation of the solutions of partial differential equations (PDEs) using physics-informed neural networks (PINNs). It is observed that network architectures inspired by multi-level Schwarz domain decomposition methods can improve the performance for certain challenging problems, such as multiscale problems. Moreover, a classical machine learning task is considered, that is, image segmentation using convolutional neural networks (CNNs). Domain decomposition techniques offer a way of scaling up common CNN architectures, such as the U-Net. In particular, local subdomain networks learn local features and are coupled via a coarse network which incorporates global features.

13. XJU

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited talk. International Workshop on Deep Learning and Numerical Methods for PDEs, Xi’an, China, June 21-23

    Abs Slides

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. PINNs have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require careful hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. One way to incorporate the domain decomposition approach is to introduce an outer Schwarz iteration and solve the local subdomain problems using a PINN model. This approach has been introduced as the deep learning-based domain decomposition (DeepDDM) method. On the other hand, in the finite basis physics-informed neural networks (FBPINNs) approach, the domain decomposition is introduced via the network architecture, and the coupling is performed implicitly without introducing additional loss terms. Inspired by the space decomposition of classical Schwarz methods, a general framework, that also allows for multilevel extensions, can be introduced. The multilevel FBPINN method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Finally, the combination of the multilevel domain decomposition strategy with multifidelity stacking PINNs, introduced as stacking FBPINNs for time-dependent problems, will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

14. XUT

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited talk. Xi’an University of Technology, Xi’an, China, June 21

    Abs Slides

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. PINNs have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require careful hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. One way to incorporate the domain decomposition approach is to introduce an outer Schwarz iteration and solve the local subdomain problems using a PINN model. This approach has been introduced as the deep learning-based domain decomposition (DeepDDM) method. On the other hand, in the finite basis physics-informed neural networks (FBPINNs) approach, the domain decomposition is introduced via the network architecture, and the coupling is performed implicitly without introducing additional loss terms. Inspired by the space decomposition of classical Schwarz methods, a general framework, that also allows for multilevel extensions, can be introduced. The multilevel FBPINN method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Finally, the combination of the multilevel domain decomposition strategy with multifidelity stacking PINNs, introduced as stacking FBPINNs for time-dependent problems, will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

15. Precond 24

    The importance of coarse levels for domain decomposition methods

    Alexander Heinlein

    Invited plenary lecture. International Conference On Preconditioning Techniques For Scientific and Industrial Applications (Precond 24), Georgia Institute of Technology, Atlanta, USA , June 10-12

    Abs URL Slides

    Domain decomposition methods (DDMs) solve boundary value problems by decomposing them into smaller subproblems defined on an overlapping or non-overlapping decomposition of the computational domain. Their divide-and-conquer approach makes them well-suited for parallel computing. However, achieving robust convergence for challenging problems and scalability to large numbers of subdomains generally requires (global) information transport. This can be achieved by incorporating a well-designed coarse level, transforming DDMs from one- to multi-level algorithms. This talk highlights the importance of coarse levels in domain decomposition methods. First, the algorithmic framework of extension-based coarse spaces will be discussed. They provide robustness and scalability to Schwarz preconditioners for a wide range of challenging problems exhibiting, for instance, strong heterogeneities, multiple coupled physics, and/or strong nonlinearities. Numerical results using the FROSch (Fast and Robust Overlapping Schwarz) package, which is part of the Trilinos library, demonstrate the effectiveness and efficiency of these Schwarz preconditioners. The second part of the talk will explore the application of DDMs to neural networks (NNs), demonstrating improvements in terms of accuracy, computation time, and/or memory efficiency. Similar to classical domain decomposition methods, coarse levels, here in the form of small global NNs, ensure global information transport, enabling scalability. This talk will cover the application of DDMs in solving partial differential equations using physics-informed NNs (PINNs) and in image segmentation using convolutional NNs (CNNs).

16. ECCOMAS2024

    Domain decomposition for neural networks

    Alexander Heinlein

    9th European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS Congress 2024), Lisbon, Portugal, June 3-7

    Abs URL Slides

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. In this context, this talks focuses on the enhancement of machine learning using classical numerical methods, in particular, on improving neural networks using domain decomposition-inspired architectures. In the first part of this talk, the domain decomposition paradigm is applied to the approximation of the solutions of partial differential equations (PDEs) using physics-informed neural networks (PINNs). It is observed that network architectures inspired by multi-level Schwarz domain decomposition methods can improve the performance for certain challenging problems, such as multiscale problems. This part of the talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). Moreover, a classical machine learning task is considered, that is, image segmentation using convolutional neural networks (CNNs). Domain decomposition techniques offer a way of scaling up common CNN architectures, such as the U-Net. In particular, local subdomain networks learn local features and are coupled via a coarse network which incorporates global features. The second part of this talk is based on joint work with Eric Cyr (Sandia National Laboratories) and Corné Verburg (Delft University of Technology).

17. Seminar

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited seminar talk. Institute of Mathematics of the Czech Academy of Sciences, Prague, Czech Republic, May 24

    Abs URL Slides

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. PINNs have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require careful hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. One way to incorporate the domain decomposition approach is to introduce an outer Schwarz iteration and solve the local subdomain problems using a PINN model. This approach has been introduced as the deep learning-based domain decomposition (DeepDDM) method. On the other hand, in the finite basis physics-informed neural networks (FBPINNs) approach, the domain decomposition is introduced via the network architecture, and the coupling is performed implicitly without introducing additional loss terms. Inspired by the space decomposition of classical Schwarz methods, a general framework, that also allows for multilevel extensions, can be introduced. The multilevel FBPINN method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Finally, the combination of the multilevel domain decomposition strategy with multifidelity stacking PINNs, introduced as stacking FBPINNs for time-dependent problems, will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

18. HPCSE 2024

    Fast and Robust Overlapping Schwarz (FROSch) Domain Decomposition Preconditioners

    Alexander Heinlein

    Invited plenary lecture. High Performance Computing in Science and Engineering 2024 conference (HPCSE 2024), Beskydy, Czech Republic, May 20 - 23

    Abs URL Slides

    The Schwarz domain decomposition framework is a powerful algorithmic framework for efficiently solving partial differential equations by decomposing a complex global problem into smaller, local subproblems. The FROSch (Fast and Robust Overlapping Schwarz) package, which is part of the Trilinos software library, leverages this framework. Moreover, FROSch employs extension-based coarse spaces to allow for constructing scalable and algebraic multilevel Schwarz preconditioners. In this context, "algebraic" means that the preconditioners can be constructed using only the fully assembled, parallel distributed system matrix. This talk gives an overview of the capabilities of FROSch and delves into recent developments. This includes: 1) exploring the use of inexact local solvers on both GPUs and CPUs to improve computing times; 2) the development of monolithic and adaptive coarse spaces to broaden the range of problems FROSch can tackle; 3) investigations into utilizing machine learning techniques, such as graph neural networks, to improve the construction of coarse bases. The performance of these approaches is evaluated across various problem types, encompassing simple model problems as well as complex multi-physics problems that address real-world applications.

19. CRUNCH

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited seminar talk. CRUNCH seminar, CRUNCH Group, Division of Applied Mathematics, Brown University, USA, March 22

    Abs URL Slides Video

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. They have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require a lot of hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. Compared with other domain decomposition techniques for PINNs, in the finite basis physics-informed neural networks (FBPINNs) approach, the coupling is done implicitly via the overlapping regions and does not require additional loss terms. Using the classical Schwarz domain decomposition framework, a very general framework, that also allows for mult-level extensions, can be introduced. The method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Furthermore, the combination of the multi-level domain decomposition strategy with multifidelity stacking PINNs for time-dependent problems will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

20. Sandia

    Fast and Robust Overlapping Schwarz (FROSch) Preconditioners in Trilinos

    Alexander Heinlein

    Seminar talk, Sandia National Laboratories, USA, March 19

    Abs Slides

    Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework. FROSch mostly focuses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Wildund (GDSW) type coarse spaces. This talk gives an overview of the FROSch software framework and summarizes current developments.

21. SIAMPP

    Efficient Schwarz Preconditioning Techniques on Current Hardware Using FROSch

    Alexander Heinlein

    SIAM Conference on Parallel Processing for Scientific Computing (PP24), Baltimore, USA, March 5 - 8

    Abs URL Slides

    FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners in Trilinos. It is applicable to a wide range of problems due to its algebraic approach, which allows the preconditioners to be constructed from a fully assembled, parallel distributed matrix. This is enabled by the use of extension-based coarse spaces instead of classical coarse spaces. FROSch also features a variety of algorithmic variants that extend its applicability and scalability. These include monolithic preconditioning for block systems and multi-level extensions. This talk will focus on recent developments in FROSch that improve the efficiency of Schwarz preconditioners for current hardware architectures. These include techniques for reducing communication and global work, lower precision preconditioning, as well as techniques for facilitating GPUs. The performance of the different techniques will be demonstrated for different application problems and using different state-of-the-art supercomputers.

22. Seminar

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited seminar talk. Séminaire d’analyse numérique, Université de Genève, Geneva, Switzerland, February 20

    Abs URL Slides

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. They have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require a lot of hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. Compared with other domain decomposition techniques for PINNs, in the finite basis physics-informed neural networks (FBPINNs) approach, the coupling is done implicitly via the overlapping regions and does not require additional loss terms. Using the classical Schwarz domain decomposition framework, a very general framework, that also allows for mult-level extensions, can be introduced. The method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Furthermore, the combination of the multi-level domain decomposition strategy with multifidelity stacking PINNs for time-dependent problems will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

23. ANCS

    Domain decomposition for physics-informed neural networks

    Alexander Heinlein

    Invited seminar talk. ANCS Seminar, Laboratoire de Mathématiques, Besançon, France, February 16

    Abs Slides

    Physics-informed neural networks (PINNs) are a class of methods for solving differential equation-based problems using a neural network as the discretization. They have been introduced by Raissi et al. and combine the pioneering collocation approach for neural network functions introduced by Lagaris et al. with the incorporation of data via an additional loss term. PINNs are very versatile as they do not require an explicit mesh, allow for the solution of parameter identification problems, and are well-suited for high-dimensional problems. However, the training of a PINN model is generally not very robust and may require a lot of hyper parameter tuning. In particular, due to the so-called spectral bias, the training of PINN models is notoriously difficult when scaling up to large computational domains as well as for multiscale problems. In this talk, overlapping domain decomposition-based techniques for PINNs are being discussed. Compared with other domain decomposition techniques for PINNs, in the finite basis physics-informed neural networks (FBPINNs) approach, the coupling is done implicitly via the overlapping regions and does not require additional loss terms. Using the classical Schwarz domain decomposition framework, a very general framework, that also allows for mult-level extensions, can be introduced. The method outperforms classical PINNs on several types of problems, including multiscale problems, both in terms of accuracy and efficiency. Furthermore, the combination of the multi-level domain decomposition strategy with multifidelity stacking PINNs for time-dependent problems will be discussed. It can be observed that the combination of multifidelity stacking PINNs with a domain decomposition in time clearly improves the reference results without a domain decomposition.

24. DD28

    Scalable coarse spaces for monolithic Schwarz preconditioners

    Alexander Heinlein

    28th International Conference on Domain Decomposition Methods (DD28), King Abdullah University of Science and Technology (KAUST), Saudi Arabia, January 28 - February 1

    Abs URL

    Monolithic GDSW (generalized Dryja–Smith–Wildund) preconditioners are scalable two-level Schwarz preconditioners for block systems. Compared with block domain decomposition preconditioners, monolithic domain decomposition preconditioners often yield faster convergence since they account for the coupling within the subdomain and coarse problems. In this talk, the focus is on computational fluid dynamics (CFD) simulations, in particular, saddle point problem resulting from the discretization of the (Navier-)Stokes equations. In this case, numerical results show that fast convergence and scalability of monolithic Schwarz preconditioners strongly depends on the choice of an appropriate pair of the velocity and pressure coarse spaces; in particular, there seem to be parallels to inf-sup stable discretizations. Whereas inf-sup stable Lagrangian coarse spaces, for instance, based on Taylor-Hood elements, yield scalable monolithic preconditioners, they are limited in their applicability to complicated geometries and unstructured domain decompositions. GDSW-type coarse spaces are well-suited for those cases, but, in general, an extension of the GDSW approach is required to construct appropriate pairs of velocity and pressure coarse spaces. Therefore, in this talk, a wider class of extension-based coarse spaces using various interface partitions of unity is explored. The numerical and parallel scalability for (Navier-)Stokes problems is then investigated using the parallel FROSch (Fast and Robust Overlapping Schwarz) preconditioner framework in Trilinos.

25. DD28

    Multi-level domain decomposition-based physics-informed neural networks

    Alexander Heinlein

    28th International Conference on Domain Decomposition Methods (DD28), King Abdullah University of Science and Technology (KAUST), Saudi Arabia, January 28 - February 1

    Abs URL Slides

    Scientific machine learning (SciML) is a rapidly growing field that combines scientific computing and machine learning to solve complex scientific problems. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. One popular example are physics-informed neural networks (PINNs), which are trained by minimizing a loss function that may include both a data error and a residual error, the latter of which enforces the governing PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity and often lead to challenges in the training. In this talk, multi-level domain decomposition-based approaches for PINNs will be discussed. Therefore, on each level, the domain is decomposed into overlapping subdomains, and a separate network is constructed on each subdomain. The networks are then trained using a monolithic optimization loop, where the subdomain networks are coupled in an additive way using partition of unity functions. Numerical results for several model problems, including challenging multiscale problems, that demonstrate the robust convergence of the multi-level domain decomposition approach are presented.

26. DD28

    Efficient Schwarz preconditioning techniques for nonlinear problems

    Alexander Heinlein

    28th International Conference on Domain Decomposition Methods (DD28), King Abdullah University of Science and Technology (KAUST), Saudi Arabia, January 28 - February 1

    Abs URL Slides

    This talk discusses several techniques for efficiently solving nonlinear problems using Schwarz methods. Newton-Krylov methods are widely used to solve nonlinear problems numerically. These methods linearize the system using Newton’s method and solve the resulting linearized systems using a (preconditioned) Krylov method. When using domain decomposition preconditioners, the efficiency can be enhanced if information about the preconditioner from previous linearizations is reused. This includes structural information, such as the domain decomposition, certain index sets, and symbolic factorizations of the subdomain and coarse problems, but also certain components of the preconditioner, such as the coarse space. This typically leads to a speedup in the setup phase, resulting in a dominance of the time required for the solve phase; the solve phase can then be sped up by assign computations to GPUs. As an alternative to Newton-Krylov methods, nonlinear Schwarz preconditioning techniques can be employed to speed up both the nonlinear and linear convergence. In order to facilitate numerical scalability to large numbers of subdomains, a multi-level approach has to be employed; this leads to nonlinear multi-level Schwarz methods. The aforementioned approaches are being discussed and investigated in numerical experiments.

2023

1. SIMULA

   Improving the convergence of pseudo-transient continuation for CFD simulations using neural networks

   Alexander Heinlein

   Invited talk. SciML@Simula workshop (Hybrid), Oslo, Norway, December 1

   Abs

   Computational fluid dynamics (CFD) simulations are highly relevant for a large range of applications, including but not restricted to the fields of aerospace, environmental, and biological engineering as well as weather predictions and medicine. Modeling Newtonian fluids involves the solution of the Navier-Stokes equations, which, depending on the Reynolds number of the flow, may exhibit a highly nonlinear behavior. The system of nonlinear equations resulting from the discretization of the Navier-Stokes equations can be solved using nonlinear iteration methods, such as Newton’s method. However, fast quadratic convergence is typically only obtained in a local neighborhood of the solution, and for many configurations, the classical Newton iteration does not converge at all. In such cases, so-called globalization techniques may help to improve convergence. In this talk, pseudo-transient continuation is employed in order to improve nonlinear convergence. The classical algorithm is enhanced by a neural network model that is trained to predict a local pseudo-time step. Generalization of the novel approach is facilitated by predicting the local pseudo-time step separately on each element using only local information on a patch of adjacent elements as input. Numerical results for standard benchmark problems, including flow through a backward facing step geometry and Couette flow, show the performance of the machine learning-enhanced globalization approach; as the software for the simulations, the CFD module of COMSOL Multiphysics® is employed.

2. DUCOMS

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Dutch Computational Sciences (DUCOMS) Day 2023, Netherlands, November 11

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems.

3. TUG 2023

   Fast and Robust Overlapping Schwarz (FROSch) Preconditioners in Trilinos – New Developments and Applications

   Alexander Heinlein

   Trilinos User-Developer Group Meeting 2023 (Hybrid), CSRI, Sandia National Laboratories, Albuquerque, USA, October 30-November 2

   URL Slides
4. DAMUT Colloquium

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, October 4

   Abs URL

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

5. TU Delft

   Decomposing physics-informed neural networks

   Alexander Heinlein

   Numerical Analysis group coffee talk. Delft University of Technology, Netherlands, September 8
6. ICIAM

   Efficient Schwarz Preconditioning Techniques for Nonlinear Problems Using FROSch

   Alexander Heinlein

   10th International Congress on Industrial and Applied Mathematics (ICIAM), Waseda University, Tokyo, Japan, August 20-25

   Abs URL Slides

   FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners in Trilinos. Due an algebraic approach, meaning that the preconditioners can be constructed from a fully assembled matrix, FROSch is applicable to a wide range of problems. This talk is focused on the application to nonlinear problems, including computational fluid dynamics and land ice simulations. Techniques for improving the efficiency and the use of GPU architectures are discussed.

7. Workshop

   Multilevel domain decomposition-based architectures for physics-informed neural networks

   Alexander Heinlein

   Invited talk. Workshop on Scientific Learning and Computing, University of Macau, Macao, China, August 17-18

   Abs URL Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

8. DLR

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, German Aerospace Center (DLR), July 18
9. TUM

   Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

   Alexander Heinlein

   Invited seminar talk, Technical University of Munich, July 13

   Abs Slides

   Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based network architectures improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. It is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as the residual of the PDE may be used in the loss function for training the network. The second part of the talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

10. EURODYN

    Temporal homogenisation and parallelisation for the numerical simulation of atherosclerotic plaque growth

    Alexander Heinlein

    XII International Conference on Structural Dynamics (EURODYN 2023), Delft University of Technology, Delft, The Netherlands, July 2-5

    Abs URL

    The numerical simulation of atherosclerotic plaque growth is computationally prohibitive since it involves a complex cardiovascular fluid-structure interaction (FSI) problem with a characteristic time scale of milliseconds to seconds as well as a plaque growth process governed by reaction-diffusion equations, which takes place over several months. A resolution of the fast (micro) scale over this period can easily require more than a billion time steps, each corresponding to the solution of a computationally expensive FSI problem. To tackle this problem, we combine a temporal homogenization approach with parallel time-stepping. First, a temporal homogenization approach is developed, which separates the problem in an FSI problem on the micro scale and a reaction-diffusion problem on the macro scale. The approach is analyzed in detail for a simplified flow problem and estimates for the homogenization error and the discretization errors on both time scales are given. Second, a parallel time-stepping approach based on the parareal algorithm is applied on the macro scale of the homogenized system. We investigate modifications in the coarse propagator of the parareal algorithm to further reduce the number of expensive micro problems to be solved and test the numerical algorithms in detailed numerical studies.

11. EuroTUG2023

    Trilinos Tutorial

    Alexander Heinlein

    European Trilinos User Group Meeting 2023 (EuroTUG2023), TU Delft, June 28

    URL Slides
12. 4TU.AMI

    SRI Bridging numerical analysis and machine learninge

    Alexander Heinlein

    4TU.AMI summer event 2023, TU Delft, June 27

    URL
13. COMINDS

    Multilevel domain decomposition-based architectures for physics-informed neural networks

    Alexander Heinlein

    Invited talk. Optimization Workshop (Thematic Einstein Semester on Optimization and Machine Learning), Humboldt Universität zu Berlin, Berlin, June 14-16

    Abs URL

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. In classical physics-informed neural networks (PINNs), simple feed-forward neural networks are employed to discretize a PDE. The loss function may include a combination of data (e.g., initial, boundary, and/or measurement data) and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large numbers of iterations. In this talk, domain decomposition-based network architectures for PINNs using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In particular, the global network function is constructed as a combination of local network functions defined on an overlapping domain decomposition. Similar to classical domain decomposition methods, the one-level method generally lacks scalability, but scalability can be achieved by introducing a multi-level hierarchy of overlapping domain decompositions. The performance of the multi-level FBPINN method will be investigated based on numerical results for several model problems, showing robust convergence for up to 64 subdomains on the finest level and challenging multi-frequency problems. This talk is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich).

14. HPC Summer School

    Advanced Domain Decomposition Methods – Parallel Schwarz Preconditioning and an Introduction to FROSch

    Alexander Heinlein

    DCSE Summerschool: Numerical Linear Algebra on High Performance Computers, TU Delft, June 5-9

    URL Slides
15. COMINDS

    Neural networks with physical constraints – Domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Invited talk. GAMM Workshop on Computational and Mathematical Methods in Data Science, Center for Data and Simulation Science, University of Cologne, Cologne, Germany, May 4-5

    Abs URL

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based network architectures improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. It is based on joint work with Victorita Dolean (University of Strathclyde, Côte d’Azur University), Siddhartha Mishra, and Ben Moseley (ETH Zürich). In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as the residual of the PDE may be used in the loss function for training the network. The second part of the talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

16. CFC 2023

    Parallel Domain Decomposition Preconditioning Techniques for Incompressible Fluid Flow Problems

    Alexander Heinlein

    22nd IACM Computational Fluids Conference (CFC 2023), Cannes, France, April 25-28

    Abs URL

    Monolithic GDSW (generalized Dryja–Smith–Wildund) preconditioners are two-level Schwarz domain decomposition preconditioners for block systems. They are robust because they account for the coupling terms in the system matrix on both levels, that is, in the local and coarse problems. Block preconditioners, mostly based on block-diagonal and block-triangular preconditioners, such as the famous SIMPLE (semi-implicit method for pressure linked equations) preconditioner, often yield higher iteration counts wile coming at a lower setup cost compared to monolithic approaches. In this talk, the parallel performance of the different preconditioning techniques for incompressible fluid flow problems is investigated and compared using a finite element implementation based on the FEDDLib finite element software and Schwarz preconditioners from the Trilinos package FROSch.

17. CFC 2023

    Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

    Alexander Heinlein

    22nd IACM Computational Fluids Conference (CFC 2023), Cannes, France, April 25-38

    Abs URL

    Computational fluid dynamics (CFD) simulations are important in many application areas, such as civil and mechanical engineering, meteorology, geosciences, or medical science. However, accurate simulation results come at high computational costs, and they require high-quality meshes describing the computational domain. In this talk, a machine learning-based model order reduction approach for predicting flow fields is presented. After an expensive offline phase, where a surrogate model based on a convolutional neural network (U-Net type architecture) is trained using simulation data, the predictions can be performed much faster than classical numerical simulations. Moreover, the predictions do not require the generation of a complex computational mesh. The surrogate model can be 100 or 10,000 times faster on a CPU or GPU, respectively, of a normal workstation. Numerical results investigating the accuracy and efficiency for varying types of geometries from different application areas are presented. Moreover, different aspects to improve the performance such as the choice of more sophisticated loss functions are discussed.

18. Sandia

    Neural networks with physical constraints, domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Seminar talk, Sandia National Laboratories, USA, April 4

    Abs

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as PDE may be used in the loss function for training the network.

19. IGCM 2023

    Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

    Alexander Heinlein

    Invited plenary lecture. Indo-German Conference on Computational Mathematics 2023 (IGCM-2023), Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, India, March 27-30

    Abs URL

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. A major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using neural networks. The network models be can trained in a data-driven and/or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In physics-informed neural networks (PINNs), simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of data and the residual of the PDE. Challenging applications, such as multiscale problems, require neural networks with high capacity, and the training is often not robust and may take large iteration counts. Therefore, in the first part of the talk, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on convolutional neural networks (CNNs) will be discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach, a single network is trained to approximate a variety of boundary value problems. This makes the approach potentially very efficient. As in the PINN approach, data as well as PDE may be used in the loss function for training the network.

20. NVIDIA/HLRS

    Neural networks with physical constraints, domain decomposition-based network architectures, and model order reduction

    Alexander Heinlein

    Invited talk. NVIDIA/HLRS SciML GPU Bootcamp, Stuttgart (virtual), Germany, April 26-27

    URL Slides Video
21. SIAM CSE23

    Domain Decomposition Training Strategies for Physics-Informed Neural Networks

    Alexander Heinlein

    SIAM Conference on Computational Science and Engineering (CSE23). RAI Congress Centre, Amsterdam, The Netherlands, February 26 - March 3

    Abs URL

    Two-level domain decomposition methods, such as two-level Schwarz, finite element tearing and interconnecting - dual primal (FETI-DP), or balancing domain decomposition by constraints (BDDC) methods, are scalable for a large class of homogeneous problems. However, in the presence of highly-heterogeneous coefficients, convergence generally deteriorates. If the coarse space is enhanced by adapted coarse basis functions, robustness can be regained. Similar observations can be made for highly-heterogeneous nonlinear problems, where both the linear and the nonlinear convergence maybe affected; in this context, a combination of nonlinear domain decomposition methods and robust coarse spaces can recover good linear and nonlinear solver performance. In this talk, preconditioning techniques for highly-heterogeneous linear and nonlinear problems will be discussed and numerical results for various problems will be reported; the focus is on Schwarz domain decomposition methods, but many approaches can be - with small modifications - applied to other domain decomposition methods as well.

22. CTU

    Domain decomposition methods for highly heterogeneous problems - Robust coarse spaces and nonlinear preconditioning

    Alexander Heinlein

    Invited seminar talk. Czech Technical University in Prague, Prague, Czech Republic, February 13

    Abs Slides Video

    Two-level domain decomposition methods, such as two-level Schwarz, finite element tearing and interconnecting - dual primal (FETI-DP), or balancing domain decomposition by constraints (BDDC) methods, are scalable for a large class of homogeneous problems. However, in the presence of highly-heterogeneous coefficients, convergence generally deteriorates. If the coarse space is enhanced by adapted coarse basis functions, robustness can be regained. Similar observations can be made for highly-heterogeneous nonlinear problems, where both the linear and the nonlinear convergence maybe affected; in this context, a combination of nonlinear domain decomposition methods and robust coarse spaces can recover good linear and nonlinear solver performance. In this talk, preconditioning techniques for highly-heterogeneous linear and nonlinear problems will be discussed and numerical results for various problems will be reported; the focus is on Schwarz domain decomposition methods, but many approaches can be - with small modifications - applied to other domain decomposition methods as well.

23. CDS

    Fast and Robust Overlapping Schwarz Preconditioners in Trilinos – Highly Scalable Algorithms and Their Efficient Implementation

    Alexander Heinlein

    Invited seminar talk. Center for Data and Simulation Science (CDS), Universität zu Köln, Köln, Germany, January 18

    Abs URL Slides

    The Trilinos library is an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific problems on new and emerging high-performance computing (HPC) architectures. It provides a collection of interoperable software packages enabling the development of algorithms reaching parallel scalability up to the largest supercomputers available. This talk will discuss different aspects of Trilinos for the example of the FROSch (Fast and Robust Overlapping Schwarz) preconditioning framework, which is part of the Trilinos package ShyLU. FROSch implements multilevel Schwarz preconditioners, which are algebraic, i.e., which can be constructed using only the fully assembled parallel distributed system matrix. Making use of the software infrastructure of Trilinos, FROSch allows for the parallel solution of extremely large problems. Numerical results for various problems indicating parallel scalability up to more than 200,000 MPI ranks will be presented. Moreover, node-level parallelization on CPUs as well as GPUs using the Kokkos programming model through the Tpetra linear algebra framework will be discussed.

24. SimTech

    Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

    Alexander Heinlein

    Invited seminar talk. SimTech ML-Session, Universität Stuttgart, Stuttgart, Germany, January 11

    Abs Slides

    Scientific machine learning (SciML) is a rapidly evolving field of research that combines techniques from scientific computing and machine learning. One major branch of SciML is the approximation of the solutions of partial differential equations (PDEs) using machine learning models and, in particular, neural networks. The network models be can trained in a data-driven or physics-informed way, that is, using reference data (from simulations or measurements) or a loss function based on the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: physics-informed neural networks (PINNs) and surrogate models based on convolutional neural networks (CNNs). In PINNs, simple feedforward neural networks are employed to discretize the PDEs, and a single network is trained to approximate the solution of one specific boundary value problem. The loss function may include a combination of reference data and the residual of the PDE. Challenging applications, such as multiscale problems, require the use of neural networks with high capacity, and the training of the models is often not robust and may take large iteration counts. Therefore, domain decomposition-based training strategies improving the training performance using the finite basis physics-informed neural network (FBPINN) approach will be discussed. In the second part of the talk, surrogate models for computational fluid dynamics (CFD) simulations based on CNNs are discussed. In particular, the network is trained to approximate a solution operator, taking a representation of the geometry as input and the solution field(s) as output. In contrast to the classical PINN approach and similar to other operator learning approaches, a single network is therefore trained to approximate a variety of boundary value problems. This makes the surrogate modeling approach potentially very efficient. As in the PINN approach, data as well as physics may be used in the loss function for training the network.

2022

1. Seminar

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited talk. Scientific Machine Learning seminar series, University of Strathclyde, Glasgow, UK, December 6

   Abs

   One major branch of scientific machine learning (SciML) is the approximation of the solutions of partial differential equations (PDEs) using neural networks. This can be done in a data-driven or physics-informed way, that is, using reference data from simulations or measurements or by optimizing with respect to the residual of the PDE, respectively. In this talk, two approaches for approximating the solutions of PDEs using neural networks are discussed: domain decomposition techniques for improving the training of physics-informed neural networks and model order reduction techniques based on convolutional neural networks. Both approaches allow for training using a combination of data and physics.

2. LUH2022

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Seminar talk. Leibniz Universität Hannover, Hannover, Germany, December 1

   Abs

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

3. NVIDIA/HLRS

   Neural networks with physical constraints, domain decomposition-based training strategies, and model order reduction

   Alexander Heinlein

   Invited talk. NVIDIA/HLRS SciML GPU Bootcamp, Stuttgart (virtual), Germany, October 24-25

   URL Slides Video
4. SIAM MDS22

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   SIAM Conference on Mathematics of Data Science, SIAM MDS22, San Diego (Hybrid), USA, September 26 - 30

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities. In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. In particular, the geometry of the computational domain is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented.

5. UDE2022

   Robust, algebraic, and scalable solvers based on Schwarz domain decomposition methods

   Alexander Heinlein

   Kolloquium Ingenieurmathematik, Universität Duisburg-Essen, Essen, Germany, September 15
6. GACM2022

   Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

   Alexander Heinlein

   9th GACM Colloquium on Computational Mechanics 2022, Essen, Germany, September 21-23

   Abs URL

   CFD simulations are very costly to compute and have to be repeated if the geometry changes even slightly. Recently there have been a number of attempts to speed up this process using neural networks. Among these is the use of Convolutional Neural Networks (CNN) as surrogate models for CFD simulations with varying geometries. Here, the model is trained on images of high-fidelity simulation results. However, the generation of training data is expensive and this approach usually requires a large data set. Thus, it is of interest to be able to train a CNN in the absence of abundant training data with the help of physical constraints. First results have already been achieved for the heat equation on a fixed geometry and flow problems in parameterizable geometries. In this talk, we present a physics-aware approach to train CNNs as surrogate models for CFD simulations in varying geometries. The employed CNN takes an imags of the geometry as input and returns images of the associated CFD simulation results, i.e., velocity and pressure, as output. Our CNN architecture is based on the structure of U-Net. Since the model is trained on pixel images, it can be applied to a variety of different geometries. We show results for two-dimensional flows around obstacles of varying size and placement and in non-rectangular geometries, esp. arteries and aneurysms.

7. GAMM 2022

   Predicting Cardiovascular Flow Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   92nd GAMM Annual Meeting, Aachen, Germany, August 15-19

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In cardiovascular applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities, such as the maximum velocity or the wall shear stresses at certain locations. In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed, and the architecture and hyper parameters are optimized to this application. Two different approaches are compared: a fully supervised approach, where the model is trained using high fidelity simulation data, and a novel approach, where the model is trained based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Then, the framework is applied to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

8. DD XXVII

   Robust, algebraic, and scalable Schwarz preconditioners with extension-based coarse spaces

   Alexander Heinlein

   Invited plenary lecture. 27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

   Abs URL Slides Video

   Coarse spaces are an important component in domain decomposition preconditioners since they are, in general, necessary to obtain numerical scalability and robustness. This talk will focus on presenting an algorithmic framework for the construction of coarse spaces for Schwarz preconditioners which is based on the generalized Dryja–Smith–Widlund (GDSW) coarse space. In particular, the coarse basis functions are extensions from the interface into the interior of a corresponding nonoverlapping domain decomposition by solving the homogeneous partial differential equation (PDE) with a given trace on the interface. This makes the coarse spaces very practical since they can be constructed in an algebraic way, that is, using only the fully assembled system matrix. This algorithmic framework is the basis for the highly-scalable FROSch (Fast and Robust Schwarz) solver package, which is part of the Trilinos software library. The FROSch package provides a parallel software framework to construct algebraic Schwarz preconditioners that are robust as well as numerically and parallel scalable for a wide range of problems. It provides a multi-level implementation to extend parallel scalability. Moreover, using extension-based coarse spaces, robust monolithic solvers for saddle-point problems, such as the (Navier–)Stokes equations and multi-physics problems in land ice simulations, can be constructed. Solving appropriate local eigenvalue problems on a partition of the interface to compute the traces of the coarse basis yields provably robust extension-based coarse spaces, even for highly heterogeneous model problems with large coefficient jumps. These adaptive coarse spaces are generally not algebraic since the eigenvalue problems require local Neumann matrices. However, by solving two local eigenvalue problems for each interface component instead, an algebraic and adaptive coarse space can be obtained. Finally, the framework can also be employed in nonlinear Schwarz preconditioning.

9. DD XXVII

   Robust Coarse Spaces for Nonlinear Schwarz Methods

   Alexander Heinlein

   27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

   Abs URL

   In this talk, nonlinear left-preconditioning by nonlinear Schwarz domain decomposition methods is considered. In particular, the focus is on the restricted additive Schwarz preconditioned exact Newton (RASPEN) method. It is based on additive Schwarz preconditioned Newton (ASPEN) method, which has been introduced together with the corresponding inexact method, ASPIN. Nonlinear domain decomposition methods can improve nonlinear as well as linear convergence. However, as in linear domain decomposition methods, scalability and robustness for difficult problems, such as highly heterogeneous problems, generally requires adding a coarse space. The coarse space can be integrated in an additive or multiplicative way using a Galerkin projection. This approach has proven to be very flexible, allowing for the use of different types of coarse spaces, such as classical Lagrangian or multiscale finite element method (MsFEM) type coarse spaces. Furthermore, to obtain robustness for heterogeneous problems, spectral coarse spaces, such as the adaptive generalized Dryja–Smith–Wildund (GDSW) coarse space can be employed. The main goal of this talk is to investigate the effectiveness of different coarse spaces, including GDSW, MsFEM, adaptive GDSW, and approximate component mode synthesis-based (OS-ACMS) adaptive coarse spaces for different nonlinear model problems and depending on the domain decomposition as well as heterogeneity of the problem.

10. DD XXVII

    Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

    Alexander Heinlein

    27th International Conference on Domain Decomposition Methods, Prague, Czech Republic, July 25-29

    Abs URL

    Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a test-bed for the surrogate models, flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

11. Colloquium

    Algebraic Schwarz Domain Decomposition Preconditioners – Parallel Scalability and Robustness

    Alexander Heinlein

    Kolloquium des Fachbereichs Mathematik und Informatik, Universität Marburg, Marburg, Germany, July 18
12. EQUADIFF 15

    Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

    Alexander Heinlein

    Equadiff 15, Brno, Czech Republic, July 11-15

    Abs URL

    Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed. Using this approach, it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. In order to construct accurate surrogate models, U-Net type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. After that, a novel approach is introduced, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a test-bed for the surrogate models, flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes.

13. 4TU.AMI

    Parallel Domain Decomposition Solvers & Scientific Machine Learning

    Alexander Heinlein

    4TU.AMI summer event 2022, Eindhoven, The Netherlands, July 5

    URL
14. ICMS@Strathclyde

    A fully algebraic spectral coarse space for overlapping Schwarz methods

    Alexander Heinlein

    Invited talk. ICMS@Strathclyde: Solvers for frequency-domain wave problems and applications, Glasgow, UK, June 20-24

    Abs URL Slides

    Discretizing partial differential equations often results in sparse systems of linear equations. High spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, for instance, in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. The development of spectral coarse spaces for domain decomposition methods has been a very active topic within the last decade. However, until recently, the algebraic construction of robust spectral coarse spaces, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information, has still been on open problem. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. An algebraic and robust spectral coarse space is then constructed by solving two eigenvalue problems on each edge of a two-dimensional domain decomposition. One of them is based on optimal local approximation spaces that are also successfully employed in the construction of multiscale discretizations. The resulting condition number bound is independent of the contrast of the coefficient function, indicating the robustness of the method. In order investigate the robustness of this new method numerically, numerical results for different coefficient distributions with large jumps are presented, including typical examples with channels, random distributions, and coefficient distributions generated from the SPE10 benchmark.

15. ECCOMAS2022

    Parallel Schwarz domain decomposition preconditioning and an introduction to FROSch

    Alexander Heinlein

    Introductory lecture. 8th European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS Congress 2022), Oslo, Norway, June 5-9

    URL Slides
16. ECCOMAS2022

    Surrogate Models for CFD Simulations Based on Convolutional Neural Networks

    Alexander Heinlein

    8th European Congress on Computational Methods in Applied Sciences and Engineering (ECCOMAS Congress 2022), Oslo, Norway, June 5-9

    Abs URL

    Computational fluid dynamics (CFD) simulations are important in many application areas, such as civil and mechanical engineering, meteorology, geosciences, or medical science. However, accurate simulation results come at high computational costs, and they require high-quality meshes describing the computational domain. In this talk, a machine learning-based model order reduction approach for predicting flow fields is presented. After an expensive offline phase, where a surrogate model based on a convolutional neural network (U-Net type architecture) is trained using simulation data, the predictions can be performed much faster than classical numerical simulations. Moreover, the predictions do not require the generation of a complex computational mesh. The surrogate model can be 10^2 or 10^4 times faster on a CPU or GPU, respectively, of a normal workstation. Numerical results investigating the accuracy and efficiency for varying types of geometries from different application areas are presented. Moreover, different aspects to improve the performance such as the choice of more sophisticated loss functions are discussed.

17. CM2022

    A fully algebraic spectral coarse space for overlapping Schwarz methods

    Alexander Heinlein

    17th Copper Mountain Conference On Iterative Methods (Virtual), April 4 - April 8

    Abs URL

    Discretizing partial differential equations often results in sparse systems of linear equations. High spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, for instance, in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. The development of spectral coarse spaces for domain decomposition methods has been a very active topic within the last decade. However, until recently, the algebraic construction of robust spectral coarse spaces, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information, has still been on open problem. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. An algebraic and robust spectral coarse space is then constructed by solving two eigenvalue problems on each edge of a two-dimensional domain decomposition. One of them is based on optimal local approximation spaces that are also successfully employed in the construction of multiscale discretizations. The resulting condition number bound is independent of the contrast of the coefficient function, indicating the robustness of the method. In order investigate the robustness of this new method numerically, numerical results for different coefficient distributions with large jumps are presented, including typical examples with channels, random distributions, and coefficient distributions generated from the SPE10 benchmark. Furthermore, randomized eigensolvers are employed to improve the efficiency of solving the eigenvalue problems.

18. SIAM PP22

    New Developments of the FROSch Domain Decomposition Solver Package

    Alexander Heinlein

    SIAM Conference on Parallel Processing for Scientific Computing, SIAM PP22, Virtual, February 23-26

    Abs URL

    Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Wildund (GDSW) type coarse spaces. This talk gives an overview of the FROSch software framework as well as current developments in improving the performance of the solvers, for instance, due to the use of inexact subdomain and coarse solvers. Moreover, recent parallel results for challenging applications, such as coupled multiphysics simulations of land ice in Greenland and Antarctica, are presented.

19. Seminar

    Spectral coarse spaces for overlapping Schwarz methods based on energy-minimizing extensions

    Alexander Heinlein

    Seminar, Ruhr-Universität Bochum, January 13

    Abs

    Discretizing partial differential equations often results in sparse linear equation systems, and high spatial resolutions lead to large systems, which can be solved efficiently using iterative methods. A suitable class of solvers are Krylov methods preconditioned by domain decomposition preconditioners, which are scalable and robust for a wide range of problems. Unfortunately, highly heterogeneous problems arising, e.g., in the simulation of composite materials or porous media generally lead to unfavorable distributions of the eigenvalues of the system matrix that cause slow convergence for many solvers, including classical domain decomposition preconditioners. In order to retain robustness of domain decomposition methods, the coarse space can be enriched by additional coarse basis functions computed from eigenmodes of local generalized eigenvalue problems, leading to so-called spectral or adaptive coarse spaces. This talk deals with a specific class of adaptive coarse spaces for overlapping Schwarz methods which are based on a partition of the interface of the corresponding nonoverlapping domain decomposition. In particular, the generalized eigenvalue problems are based on energy-minimizing extensions corresponding to the interface components. The presented methods have a provable condition number bound, which is independent of the contrast of the coefficient functions. A great challenge is to construct adaptive coarse spaces that are robust but can be built algebraically, that is, using only the fully assembled system matrix without additional Neumann matrices or geometrical information. In this talk, a novel adaptive coarse space that is both robust and algebraic is introduced. Furthermore, approaches for combining adaptive coarse spaces with nonlinear preconditioning as well as machine learning techniques are discussed. The talk is based on joint work with Axel Klawonn, Jascha Knepper, Martin Lanser, Janine Weber (University of Cologne), Oliver Rheinbach (TU Bergakademie Freiberg), Kathrin Smetana (Stevens Institute of Technology), and Olof Widlund (New York University).

2021

1. PPS Lecture

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Pretty Porous Science Lecture, Universität Stuttgart, December 7

   Abs

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

2. TUG 2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   Trilinos User-Developer Group Meeting 2021 (Virtual), November 30-December 2

   URL Slides
3. DD Summer School

   Exercise session: FROSch

   Alexander Heinlein

   Summer school on advanced DD methods, Politecnico di Milano, Milano, Italy, November 24-26

   Abs URL Slides

   Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Widlund (GDSW) type coarse spaces. In this lab session, the FROSch software framework will be introduced, and its usage will be explained based on simple model problems. The examples provided will allow investigating the influence of important algorithmic aspects of Schwarz methods, such as the variation of the width of the overlap or adding a coarse level, on the convergence of a preconditioned Krylov solver.

4. CIRM 2021

   Surrogate Models for Computational Fluid Dynamics Simulations Using Convolutional Autoencoder Neural Networks and Physical Constraints

   Alexander Heinlein

   Invited Lecture. CIRM Conference “Analysis, Control and Numerics for PDE Models of Interest to Physical and Life Sciences”, Levico Terme, Italy, September 20-24

   Abs URL

   Simulations of fluid flow are generally very costly because high grid resolutions are not only required to obtain quantitatively accurate results, but too low grid resolutions may also lead to qualitatively incorrect results. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of convolutional autoencoder neural networks to construct efficient reduced order surrogate models for high fidelity computational fluid dynamics (CFD) simulations is discussed; cf. [Eichinger et al., 2021] and [Eichinger et al., submitted 2020]. Using this approach, which is inspired by [Guo et al., 2016], it is possible to build surrogate models for varying geometries. In particular, the geometry is the input of the neural network, and the flow and pressure fields are the output. The proposed framework is very general since the geometry and solution fields are simply encoded as pixel images, which allows the application to various types of geometries; also the extension to other physics is, in principle, straight forward. In order to construct accurate surrogate models, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are very successful in image recognition and segmentation tasks, are employed and the architecture and hyper parameters are optimized to this application. As a first step, a fully supervised approach, which requires the availability of simulation results as the training data, is presented. Here, the choice of an appropriate loss function is crucial to obtain good results. After that, results for a newer unsupervised approach, which does not require CFD simulation results but is based on introducing physical constraints via the loss function. As a testbed for the surrogate models, the flow around obstacles with varying shape and size within a channel is considered. Moreover, results for the application to geometries of arteries with aneurysms are presented. In both cases, only two-dimensional configurations are considered for now. The results show that the surrogate models provide good predictions of the flow and pressure fields while being computationally much cheaper compared to classical CFD codes. In particular, the predictions can be performed on a workstation within a fraction of a second, without requiring the generation of a computational mesh. When evaluated on GPUs, the prediction time can be further reduced by factor of more than ten. The talk is based on joint work with Matthias Eichinger, Viktor Grimm, and Axel Klawonn (University of Cologne).

5. YIC 2021

   Using inexact subdomain and coarse solvers in FROSch preconditioners

   Alexander Heinlein

   YIC 2021 (Virtual), July 7-9

   Abs URL

   FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners, which is part of Trilinos. Although being a general framework for the construction and combination of Schwarz operators, FROSch currently focusses on preconditioners that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. The computation of the first level is based on the construction of overlapping subdomains, which can be carried out based on the sparsity pattern of the matrix. In addition to that, robust and scalable coarse spaces are constructed from a partition of unity on the domain decomposition interface and energy-minimizing extensions, without the need for additional information. In particular, GDSW (Generalized Dryja-Smith-Widlund) type coarse spaces are considered. In this talk, the use of inexact solvers for the local overlapping problems and the global coarse problem will be investigated. For this purpose, inexact solvers from different packages from the Trilinos framework will be employed, e.g, from Ifpack2, MueLu, or FROSch itself.

6. COUPLED 2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   9th International Conference on Computational Methods for Coupled Problems in Science and Engineering (Virtual), June 14-16

   Abs URL

   Greenland and Antarctic ice sheets store most of the fresh water on earth and mass loss from these ice sheets significantly contributes to sea-level rise. The simulation of temperature and velocity of the ice sheets gives rise to large highly nonlinear systems of equations. The solution of the associated tangent problems, arising in Newton’s method, is challenging also because of the strong anisotropy of the meshes. We first consider simulations of the ice velocity of Antarctica and the ice temperature of Greenland. We use one-level Schwarz preconditioners as well as GDSW (Generalized Dryja-Smith-Widlund) type preconditioners from the Trilinos package FROSch (Fast and Robust Schwarz), scaling up to 32 k processor cores (8 k MPI ranks and 4 OpenMP threads) for the finest Antarctica mesh; the corresponding velocity problem contains 566 M degrees of freedom. We then study the coupled velocity and temperature problem for the Greenland ice sheet. To the best of our knowledge, it is the first time that a scalable monolithic two-level preconditioner has been used for this multiphysics problem. We present strong scaling results, up to 4 k MPI ranks, using a monolithic GDSW type preconditioner with decoupled extensions from the FROSch package.

7. BaNaNa

   Trilinos

   Alexander Heinlein

   BaNaNa talk, SIAM Student Chapter Delft, May 19

   Abs URL

   Trilinos is a collection of more than 50 open-source software packages that can be used as building blocks for all kinds of scientific applications. For example, packages exist for iterative solvers, PDE-constrained optimization problems and automatic differentiation. This talk will give a brief introduction to Trilinos. Special attention is paid to the Tpetra package that facilitates parallel sparse linear algebra.

8. CM2021

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   20th Copper Mountain Conference On Multigrid Methods (Virtual), March 29 - April 2

   Abs URL

   Greenland and Antarctic ice sheets store most of the fresh water on earth and mass loss from these ice sheets significantly contributes to sea-level rise. The simulation of temperature and velocity of the ice sheets gives rise to large highly nonlinear systems of equations. The solution of the associated tangent problems, arising in Newton’s method, is challenging also because of the strong anisotropy of the meshes. We first consider simulations of the ice velocity of Antarctica and the ice temperature of Greenland. We use one-level Schwarz preconditioners as well as GDSW (Generalized Dryja-Smith-Widlund) type preconditioners from the Trilinos package FROSch (Fast and Robust Schwarz), scaling up to 32 k processor cores (8 k MPI ranks and 4 OpenMP threads) for the finest Antarctica mesh; the corresponding velocity problem contains 566 M degrees of freedom. We then study the coupled velocity and temperature problem for the Greenland ice sheet. To the best of our knowledge, it is the first time that a scalable monolithic two-level preconditioner has been used for this multiphysics problem. We present strong scaling results, up to 4 k MPI ranks, using a monolithic GDSW type preconditioner with decoupled extensions from the FROSch package.

9. GAMM 2020@21

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   91st GAMM Annual Meeting (Virtual), Kassel, Germany, March 15-19

   Abs URL

   Stress distributions in walls of in vivo arteries (transmural stresses) are a major factor for the course of cardiac diseases; for instance, they are a driving force for the process of arteriosclerosis and arteriogenesis. In order to make accurate predictions of realistic stress distributions using numerical fluid-structure interaction (FSI) simulations, the use of appropriate material models for the arterial walls is essential. In the literature, a wide range of material models is used for this purpose. In this talk, several existing material models for arterial walls are compared based on the curved tube FSI benchmark. In the benchmark configuration, the geometry corresponds to a bended idealized coronary artery with only one material layer. Furthermore, two phases are considered: first, the artery is brought to a physiological pressure of 80 mmHg, and second, the inflow profile of one heartbeat is imposed. As material models, linear elasticity, isotropic hyperelasticity, i.e., a Neo-Hookean material model, as well as the nonlinear anisotropic hyperelastic material models are taken into account. Numerical results for both linear elasticity and Neo-Hookean material models differ significantly from the results obtained with nonlinear anisotropic hyperelastic material models, which have been fitted to experimental data of real arteries. Moreover, the performance of the parallel FSI simulations using different material models is investigated.

10. Seminar

    Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

    Alexander Heinlein

    Numerical Analysis Seminar, TU Delft, Online, February 19

    Abs URL

    FROSch (Fast and Robust Overlapping Schwarz) is a framework for parallel Schwarz domain decomposition preconditioners, which is part of the Trilinos package ShyLU. Although being a general framework for the construction and combination of Schwarz operators, FROSch currently focusses on preconditioners that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension based coarse spaces, such as GDSW (Generalized Dryja-Smith-Widlund) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW coarse spaces, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a Poisson equation to a coupled multi physics simulations of land ice in Antarctica. Furthermore, a brief overview of some related Schwarz preconditioning techniques, which are not implemented in FROSch yet, will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as the application of machine learning techniques in order to improve the efficiency of these adaptive coarse spaces.

11. Colloquium

    Fast and Robust Overlapping Schwarz Methods – New Developments and an Efficient Parallel Implementation in Trilinos

    Alexander Heinlein

    Mathematisches Kolloquium, Universität Stuttgart, Online, February 16

    Abs

    Schwarz methods are an algorithmic framework for a large class of domain decomposition methods. The software FROSch (Fast and Robust Overlapping Schwarz), which is part of the Trilinos package ShyLU, provides a highly scalable implementation of the Schwarz framework, and the resulting solvers are based on the construction and combination of the relevant Schwarz operators. FROSch currently focusses on Schwarz operators that are algebraic in the sense that they can be constructed from a fully assembled, parallel distributed matrix. This is facilitated by the use of extension-based coarse spaces, such as generalized Dryja-Smith-Widlund (GDSW) type coarse spaces. This talk will cover several Schwarz preconditioning techniques which are currently being developed based on the FROSch package: reduced dimension coarse spaces, multilevel GDSW preconditioners, and monolithic preconditioners for block systems. These approaches are introduced and applied to different problems ranging from a simple Poisson equation to a coupled multiphysics simulations of land ice in Greenland and Antarctica. Furthermore, a brief overview of related Schwarz preconditioning techniques which are currently being developed but not implemented in FROSch yet will be given. This includes nonlinear two-level Schwarz preconditioning techniques based on Galerkin projections, adaptive coarse spaces for highly heterogeneous problems, as well as novel hybrid preconditioning algorithms combining adaptive coarse spaces and machine learning techniques.

12. Workshop

    Flow Predictions Using Convolutional Neural Networks

    Alexander Heinlein

    GAMM CSE Workshop 2021, Online, January 15, 22, and 29

    Abs URL

    Simulations of fluid flow are generally very costly because too low grid resolutions may even lead to qualitatively incorrect solutions. In applications, however, one is often not interested in accurate approximations of the complete flow field but only in the qualitative behavior of the flow or in individual quantities (e.g., maximum velocity, pressure drop within a section of a pipe, or wall shear stresses at certain locations). In this talk, the use of Convolutional Neural Networks (CNNs) to predict fluid flow fields is investigated. Therefore, U-Net [Ronneberger et al., 2015] type convolutional neural networks, which are successfully used for image recognition and segmentation tasks, are applied in this context. As a model problem, the flow around obstacles with varying shape and size within a channel is considered. Obstacles of certain type are used as training data, and the generalization of the models to other obstacle geometries and sizes is analyzed. Even though, the training of a neural network is expensive, its evaluation is quite cheap compared to fully resolved Computational Fluid Dynamics (CFD) simulations. This results in a multitude of application possibilities for neural networks in this context, especially in time critical settings.

2020

1. DD XXVI

   FROSch Preconditioners for Land Ice Simulations of Greenland and Antarctica

   Alexander Heinlein

   26th International Conference on Domain Decomposition Methods (Virtual), Hong Kong, China, December 7-12

   URL
2. DD XXVI

   Flow Predictions Using Convolutional Neural Networks

   Alexander Heinlein

   26th International Conference on Domain Decomposition Methods (Virtual), Hong Kong, China, December 7-12

   URL
3. Seminar

   Inaugural Lecture

   Alexander Heinlein

   IANS Highlight Seminar, Universität Stuttgart, Germany, October 26
4. SimTech

   SimTech Quickie

   Alexander Heinlein

   Universität Stuttgart, Germany, October 19
5. Workshop

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   Jahrestreffen GAMM Fachausschuss “Computational Biomechanics” 2020, Bildungszentrum Kloster Banz, Germany, September 21-22
6. Workshop

   Comparing Arterial Wall Models for the Curved Tube Fluid-Structure Interaction Benchmark

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 3-6
7. Seminar

   Overlapping Schwarz preconditioning techniques for nonlinear problems

   Alexander Heinlein

   Seminar, Humboldt-Universität zu Berlin, Germany, January 29
8. Workshop

   Flow Predictions Using Convolutional Neural Networks

   Alexander Heinlein

   International Workshop on Scientific Machine Learning, Universität zu Köln, Germany, January 8-10

   URL

2019

1. ENUMATH 2019

   Flow predictions using convolutional neural network

   Alexander Heinlein

   ENUMATH Conference 2019, Egmond aan Zee, Netherlands, October 2

   URL
2. ENUMATH 2019

   FROSch – A framework for parallel Schwarz preconditioners in Trilinos

   Alexander Heinlein

   ENUMATH Conference 2019, Egmond aan Zee, Netherlands, October 2

   URL
3. Seminar

   FROSch – A framework for parallel Schwarz preconditioners in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, September 3
4. MAFELAP 2019

   The FROSch (Fast and Robust Overlapping Schwarz) Package in Cardiovascular Simulations

   Alexander Heinlein

   MAFELAP 2019, Brunel University London, England, June 19

   URL
5. EuroTUG

   Fast and Robust Overlapping Schwarz: FROSch

   Alexander Heinlein

   European Trilinos User Group Meeting 2019, ETH Zürich, Switzerland, June 11

   URL
6. DAMUT Colloquium

   Designing robust and efficient domain decomposition methods for highly heterogeneous problems using local spectral information and machine learning techniques

   Alexander Heinlein

   Invited talk. DAMUT Colloquium, University of Twente, Netherlands, May 8

   URL
7. GAMM 2019

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   90th GAMM Annual Meeting, Vienna, Austria, February 18-22

   URL

2018

1. Workshop

   The FROSch Package in Cardiovascular Simulations

   Alexander Heinlein

   Workshop on Modeling, Simulation and Optimization of the Cardiovascular System, Magdeburg, Germany, October 22-24
2. Seminar

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, EPFL, Lausanne, Switzerland, August 30
3. DD XXV

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Newfoundland, Canada, July 23-27

   URL
4. DD XXV

   Three-Level Extensions of the GDSW Overlapping Schwarz Preconditioner

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Canada, July 23-27

   URL
5. DD XXV

   Multiscale Discretizations and Coarse Spaces Based on ACMS

   Alexander Heinlein

   25th International Conference on Domain Decomposition Methods, St. John’s, Canada, July 23-27

   URL
6. ECCM 6

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   6th European Conference on Computational Mechanics and 7th European Conference on Computational Fluid Dynamics, Glasgow, UK, June 11-15

   URL
7. GAMM 2018

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   89th GAMM Annual Meeting, Munich, Germany, March 19-23

   URL

2017

1. SNL

   FROSch – A Parallel Implementation of the GDSW Domain Decomposition Preconditioner in Trilinos

   Alexander Heinlein

   Seminar, Sandia National Laboratories, USA, October 9
2. PASC17

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   PASC17 Conference, Lugano, Switzerland, Juni 26-28

   URL
3. DLR

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   Workshop of the German Center for Aerospace and the Mathematical Institute at the Universität zu Köln, Cologne, Germany, March 24
4. GAMM 2017

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   88th GAMM Annual Meeting, Weimar, Germany, March 6-10

   URL
5. NMM

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 13-15
6. DD XIV

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   24th International Conference on Domain Decomposition Methods, Svalbard, Norway, February 6-10

   URL

2016

1. GAMM CSE

   Domain-Decomposition-Based Fluid-Structure Interaction Methods Using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   GAMM CSE Workshop 2016, Kassel, September 8-9
2. DLR

   An Adaptive Coarse Space for the GDSW Algorithm

   Alexander Heinlein

   Workshop of the German Center for Aerospace and the Mathematical Institute at the Universität zu Köln, Cologne, March 14
3. GAMM 2016

   A parallel implementation of the approximate component mode synthesis special finite element method in 2D

   Alexander Heinlein

   Joint Annual Meeting of DMV and GAMM, Braunschweig, Germany, March 7-11

   URL
4. NMM

   A parallel implementation of the approximate component mode synthesis special finite element method in 2D

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 22-25

2015

1. IST

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   Workshop on Innovative Modeling Techniques for Predictive Medicine, Lisbon, Portugal, November 12
2. ENUMATH 2015

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   ENUMATH Conference 2015, Ankara, Turkey, September 14-18
3. DD XXIII

   Domain-Decomposition-Based Fluid-Structure Interaction Methods using Nonlinear Anisotropic Arterial Wall Models

   Alexander Heinlein

   23rd International Conference on Domain Decomposition Methods, Jeju Island, Korea, July 6-10

   URL
4. GAMM 2015

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   86th GAMM Annual Meeting, Lecce, Italy, March 23-27

   URL
5. NMM

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 23-25

2014

1. Graz

   Fluid-Structure Interaction in Hemodynamics Using Nonlinear, Anisotropic Hyperelastic Wall Models

   Alexander Heinlein

   International Workshop on Modelling and Simulation in Biomechanics, Graz, Austria, September 15-17
2. NMM

   GDSW - Domain Decomposition for Fluid-Structure Interaction

   Alexander Heinlein

   Workshop on Numerical Mathematics and Mechanics, Kevelaer, Germany, February 10-12