Alexander Heinlein

Randomized neural networks (RaNNs), characterized by fixed hidden layers after random initialization, offer a computationally efficient alternative to fully parameterized neural networks trained using stochastic gradient descent-type algorithms. In this paper, we integrate RaNNs with overlapping Schwarz domain decomposition in two primary ways: firstly, to formulate the least-squares problem with localized basis functions, and secondly, to construct effective overlapping Schwarz preconditioners for solving the resulting linear systems. Specifically, neural networks are randomly initialized in each subdomain following a uniform distribution, and these localized solutions are combined through a partition of unity, providing a global approximation to the solution of the partial differential equation. Boundary conditions are imposed via a constraining operator, eliminating the necessity for penalty methods. Furthermore, we apply principal component analysis (PCA) within each subdomain to reduce the number of basis functions, thereby significantly improving the conditioning of the resulting linear system. By constructing additive Schwarz (AS) and restricted AS preconditioners, we efficiently solve the least-squares problems using iterative solvers such as the Conjugate Gradient (CG) and generalized minimal residual methods. Numerical experiments clearly demonstrate that the proposed methodology substantially reduces computational time, particularly for multi-scale and time-dependent PDE problems. Additionally, we present a three-dimensional numerical example illustrating the superior efficiency of employing the CG method combined with an AS preconditioner over direct methods like QR decomposition for solving the associated least-squares system.

Multiscale problems are challenging for neural network-based discretizations of differential equations, such as physics-informed neural networks (PINNs) and opera-tor networks. This can be (partly) attributed to the so-called spectral bias of neural net-works. To improve the performance of PINNs for time-dependent problems, a combination of multifidelity stacking PINNs and domain decomposition-based finite basis PINNsis employed. Inparticular, to learn thehigh-fidelity part of the multifidelity model, a do-main decomposition in time is employed. The performance is investigated for a pendulum and a two-frequency problem as well as the Allen–Cahn equation. It can be observed that the domain decomposition approach clearly improves the PINN and stacking PINN approaches. Finally, it is demonstrated that the FBPINN approach can be extended tomultifidelity physics-informed deep operator networks.

The segmentation of ultra-high resolution images poses challenges such as loss of spatial information or computational inefficiency. In this work, a novel approach that combines encoder-decoder architectures with domain decomposition strategies to address these challenges is proposed. Specifically, a domain decomposition-based U-Net (DDU-Net) architecture is introduced, which partitions input images into non-overlapping patches that can be processed independently on separate devices. A communication network is added to facilitate inter-patch information exchange to enhance the understanding of spatial context. Experimental validation is performed on a synthetic dataset that is designed to measure the effectiveness of the communication network. Then, the performance is tested on the DeepGlobe land cover classification dataset as a real-world benchmark data set. The results demonstrate that the approach, which includes inter-patch communication for images divided into 16\times16 non-overlapping subimages, achieves a 2-3\,% higher intersection over union (IoU) score compared to the same network without inter-patch communication. The performance of the network which includes communication is equivalent to that of a baseline U-Net trained on the full image, showing that our model provides an effective solution for segmenting ultra-high-resolution images while preserving spatial context. The code is available at https://github.com/corne00/DDU-Net.

Abstract. Two-level domain decomposition preconditioners lead to fast convergence and scalability of iterative solvers. However, for highly heterogeneous problems with a rapidly varying coefficient function, the condition number of the preconditioned system generally depends on the contrast of the coefficient function. As a result, the convergence may deteriorate. Enhancing the coarse space by functions constructed from suitable local eigenvalue problems restores robust, contrast-independent convergence; these coarse spaces are often denoted as adaptive or spectral coarse spaces. However, these eigenvalue problems typically rely on nonalgebraic information such that the adaptive coarse spaces cannot be constructed from the fully assembled system matrix. In this paper, a novel algebraic adaptive coarse space which relies on the \(a\)-orthogonal decomposition of (local) finite element (FE) spaces into functions that solve the elliptic PDE with some trace and FE functions that are zero on the boundary is proposed. In particular, the basis is constructed from eigenmodes of two types of local eigenvalue problems associated with the edges of the domain decomposition. To approximate functions that solve the PDE locally, we employ a transfer eigenvalue problem which has originally been proposed for the construction of optimal local approximation spaces for multiscale methods. In addition, we make use of a Dirichlet eigenvalue problem that is a slight modification of the Neumann eigenvalue problem used in the adaptive generalized Dryja–Smith–Widlund (AGDSW) coarse space. Both eigenvalue problems rely solely on local Dirichlet matrices, which can be extracted from the fully assembled system matrix, allowing for an algebraic construction. By combining arguments from multiscale and domain decomposition methods, we derive a contrast-independent upper bound for the condition number. While we restrict ourselves here to a two-dimensional diffusion problem discretized by low-order FEs on regular meshes, the proposed framework is general, and we conjecture that the approach can be readily extended, for instance, to other elliptic problems, three dimensions, or, under mild assumptions, higher-order discretizations. The robustness of the method is confirmed numerically for a variety of heterogeneous coefficient distributions, including binary random distributions and a coefficient function constructed from the SPE10 benchmark. The results are comparable to those of the nonalgebraic AGDSW coarse space as well as for those cases where the convergence of the classical algebraic generalized Dryja–Smith–Widlund coarse space deteriorates. Moreover, the coarse space dimension is the same as or comparable to the AGDSW coarse space for all numerical experiments.

We enhance machine learning algorithms for learning model parameters in complex systems represented by differential equations with domain decomposition methods. The study evaluates the performance of two approaches, namely (vanilla) Physics-Informed Neural Networks (PINNs) and Finite Basis Physics-Informed Neural Networks (FBPINNs), in learning the dynamics of test models with a quasi-stationary longtime behavior. We test the approaches for data sets in different dynamical regions and with varying noise level. As results, the FBPINN approach better captures the overall dynamical behavior compared to the vanilla PINN approach, even in cases with data only from a time domain with quasi-stationary dynamics.

In recent years, the concept of introducing physics to machine learning has become widely popular. Most physics-inclusive ML-techniques however are still limited to a single geometry or a set of parametrizable geometries. Thus, there remains the need to train a new model for a new geometry, even if it is only slightly modified. With this work we introduce a technique with which it is possible to learn approximate solutions to the steady-state Navier–Stokes equations in varying geometries without the need of parametrization. This technique is based on a combination of a U-Net-like CNN and well established discretization methods from the field of the finite difference method. The results of our physics-aware CNN are compared to a state-of-the-art data-based approach. Additionally, it is also shown how our approach performs when combined with the data-based approach.

Approximate solutions of partial differential equations (PDEs) obtained by neural networks are highly affected by hyper parameter settings. For instance, the model training strongly depends on loss function design, including the choice of weight factors for different terms in the loss function, and the sampling set related to numerical integration; other hyper parameters, like the network architecture and the optimizer settings, also impact the model performance. On the other hand, suitable hyper parameter settings are known to be different for different model problems and currently no universal rule for the choice of hyper parameters is known. In this paper, for second order elliptic model problems, various hyper parameter settings are tested numerically to provide a practical guide for efficient and accurate neural network approximation. While a full study of all possible hyper parameter settings is not possible, we focus on studying the formulation of the PDE loss as well as the incorporation of the boundary conditions, the choice of collocation points associated with numerical integration schemes, and various approaches for dealing with loss imbalances will be extensively studied on various model problems; in addition to various Poisson model problems, also a nonlinear and an eigenvalue problem are considered.

Trilinos is a community-developed, open-source software framework that facilitates building large-scale, complex, multiscale, multiphysics simulation code bases for scientific and engineering problems. Since the Trilinos framework has undergone substantial changes to support new applications and new hardware architectures, this document is an update to “An Overview of the Trilinos project” by Heroux et al. (ACM Transactions on Mathematical Software, 31(3):397–423, 2005). It describes the design of Trilinos, introduces its new organization in product areas, and highlights established and new features available in Trilinos. Particular focus is put on the modernized software stack based on the Kokkos ecosystem to deliver performance portability across heterogeneous hardware architectures. This paper also outlines the organization of the Trilinos community and the contribution model to help onboard interested users and contributors.

During the COVID-19 crisis, mechanistic models have been proven fundamental to guide evidence-based decision making. However, time-critical decisions in a dynamically changing environment restrict the time available for modelers to gather supporting evidence. As infectious disease dynamics are often heterogeneous on a spatial or demographic scale, models should be resolved accordingly. In addition, with a large number of potential interventions, all scenarios can barely be computed on time, even when using supercomputing facilities. We suggest to combine complex mechanistic models with data-driven surrogate models to allow for on-the-fly model adaptations by public health experts. We build upon a spatially and demographically resolved infectious disease model and train a graph neural network for data sets representing early phases of the pandemic. The resulting networks reached an execution time of less than a second, a significant speedup compared to the metapopulation approach. The suggested approach yields potential for on-the-fly execution and, thus, integration of disease dynamics models in low-barrier website applications. For the approach to be used with decision-making, datasets with larger variance will have to be considered.