Sparse Grids and Applications - Munich 2018 (Lecture Notes in Computational Science and Engineering, 144) 3030813614, 9783030813611

Citation preview

144

Hans-Joachim Bungartz Jochen Garcke · Dirk Pflüger Editors

Sparse Grids and Applications – Munich 2018 Editorial Board T. J.Barth M.Griebel D.E.Keyes R.M.Nieminen D.Roose T.Schlick

Lecture Notes in Computational Science and Engineering Volume 144

Series Editors Timothy J. Barth, NASA Ames Research Center, Moffett Field, CA, USA Michael Griebel, Institut für Numerische Simulation, Universität Bonn, Bonn, Germany David E. Keyes, Applied Mathematics and Computational Science, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia Risto M. Nieminen, Department of Applied Physics, Aalto University School of Science & Technology, Aalto, Finland Dirk Roose, Department of Computer Science, Katholieke Universiteit Leuven, Leuven, Belgium Tamar Schlick, Courant Institute of Mathematical Sciences, New York University, New York, NY, USA

This series contains monographs of lecture notes type, lecture course material, and high-quality proceedings on topics described by the term “computational science and engineering”. This includes theoretical aspects of scientific computing such as mathematical modeling, optimization methods, discretization techniques, multiscale approaches, fast solution algorithms, parallelization, and visualization methods as well as the application of these approaches throughout the disciplines of biology, chemistry, physics, engineering, earth sciences, and economics.

More information about this series at https://link.springer.com/bookseries/3527

Hans-Joachim Bungartz • Jochen Garcke • Dirk Pflüger Editors

Sparse Grids and Applications - Munich 2018

Editors Hans-Joachim Bungartz Institut für Informatik TU München Garching, Germany

Jochen Garcke Institut für Numerische Simulation Universität Bonn Bonn, Germany Fraunhofer SCAI Sankt Augustin, Germany

Dirk Pflüger Institut für Parallele und Verteilte Systeme (IPVS) Universität Stuttgart Stuttgart, Germany

ISSN 1439-7358 ISSN 2197-7100 (electronic) Lecture Notes in Computational Science and Engineering ISBN 978-3-030-81361-1 ISBN 978-3-030-81362-8 (eBook) https://doi.org/10.1007/978-3-030-81362-8 Mathematics Subject Classification: 65D99, 65M12, 65N99, 65Y20, 65N12, 62H99 © Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Today, sparse grids are a well-established approach to the efficient numerical treatment of high-dimensional problems. While most classical numerical discretization schemes fail in more than three or four dimensions, sparse grids make it possible to overcome the so-called “curse of dimensionality” to some degree—extending the tractability of problems, i.e. the number of dimensions that can be dealt with. This property makes sparse grids, in their different flavors, frequently the method of choice, be it spatially adaptive in the hierarchical basis or via the dimensionally adaptive combination technique. The Fifth Workshop on Sparse Grids and Applications (SGA2018) took place in Munich, Bavaria, Germany, from July 23 to 27, 2018. This means that the workshop series returned to the birthplace of sparse grids. Once again, the event demonstrated the importance of this numerical approach and of the core algorithmic ideas behind it. Organized by Hans-Joachim Bungartz, Jochen Garcke, Michael Griebel, Markus Hegland, Dirk Pflüger, Clayton Webster, and Guannan Zhang, the workshop brought together almost 50 participants from four different continents, who presented and discussed the current state of the art of sparse grids and their various applications. Thirty-five talks covered aspects of numerical analysis as well as efficient data structures and new forms of adaptivity, and a range of applications from clustering and model order reduction to uncertainty quantification settings and optimization. Moreover, more physics- and engineering-based applications such as aerospace or plasma physics simulations were addressed. As a novelty, a software session was added to the format. This session gave a nice overview of prominent software developments in the sparse grid community, such as SG++ or the sparse grids MATLAB kit. This volume of LNCSE collects selected contributions from attendees of the workshop. Almost 30 years after the term “sparse grids” had been coined by Christoph Zenger in Munich, the SGA was hosted by his former institution, the Department of Informatics at the Technical University of Munich, together with the TUM’s Institute for Advanced Study (IAS). We want to thank the IAS and the Priority Program “Software for Exascale Computing” (SPPEXA) of the German Research

v

vi

Preface

Foundation (DFG) for the financial and organizational support. Furthermore, and in particular, we thank Michael Obersteiner and Kilian Röhner for their effort and enthusiasm in the local organization of the workshop, and the IAS staff for their always helpful assistance. Garching, Germany Bonn, Germany Stuttgart, Germany April 2021

Hans-Joachim Bungartz Jochen Garcke Dirk Pflüger

Contents

On Expansions and Nodes for Sparse Grid Collocation of Lognormal Elliptic PDEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Oliver G. Ernst, Björn Sprungk, and Lorenzo Tamellini

1

Sparse Grids Approximation of Goldstone Diagrams in Electronic Structure Calculations .. . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Heinz-Jürgen Flad and Gohar Flad-Harutyunyan

33

Generalized Sparse Grid Interpolation Based on the Fast Discrete Fourier Transform .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Michael Griebel and Jan Hamaekers

53

Fast Sparse Grid Operations Using the Unidirectional Principle: A Generalized and Unified Framework . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . David Holzmüller and Dirk Pflüger

69

Propagation of Uncertainties in Density-Driven Flow . . .. . . . . . . . . . . . . . . . . . . . 101 Alexander Litvinenko, Dmitry Logashenko, Raul Tempone, Gabriel Wittum, and David Keyes A Posteriori Error Estimation for the Stochastic Collocation Finite Element Approximation of the Heat Equation with Random Coefficients .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 127 Fabio Nobile and Eva Vidliˇcková A Spatially Adaptive Sparse Grid Combination Technique for Numerical Quadrature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 161 Michael Obersteiner and Hans-Joachim Bungartz Hierarchical Extended B-splines for Approximations on Sparse Grids. . . . 187 Michael F. Rehme, Stefan Zimmer, and Dirk Pflüger Analysis of Sparse Grid Multilevel Estimators for Multi-Dimensional Zakai Equations . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 205 Christoph Reisinger and Zhenru Wang vii

viii

Contents

Efficiently Transforming from Values of a Function on a Sparse Grid to Basis Coefficients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 229 Robert Wodraszka and Tucker Carrington Jr. A Sparse-Grid Probabilistic Scheme for Approximation of the Runaway Probability of Electrons in Fusion Tokamak Simulation . . . . . . . . 245 Minglei Yang, Guannan Zhang, Diego del-Castillo-Negrete, Miroslav Stoyanov, and Matthew Beidler

On Expansions and Nodes for Sparse Grid Collocation of Lognormal Elliptic PDEs Oliver G. Ernst, Björn Sprungk, and Lorenzo Tamellini

Abstract This work is a follow-up to our previous contribution (“Convergence of sparse collocation for functions of countably many Gaussian random variables (with application to elliptic PDEs)”, SIAM J. Numer. Anal., 2018), and contains further insights on some aspects of the solution of elliptic PDEs with lognormal diffusion coefficients using sparse grids. Specifically, we first focus on the choice of univariate interpolation rules, advocating the use of Gaussian Leja points as introduced by Narayan and Jakeman (“Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional approximation”, SIAM J. Sci. Comput., 2014) and then discuss the possible computational advantages of replacing the standard Karhunen-Loève expansion of the diffusion coefficient with the Lévy-Ciesielski expansion, motivated by theoretical work of Bachmayr, Cohen, DeVore, and Migliorati (“Sparse polynomial approximation of parametric elliptic PDEs. part II: lognormal coefficients”, ESAIM: M2AN, 2016). Our numerical results indicate that, for the problem under consideration, Gaussian Leja collocation points outperform Gauss–Hermite and Genz–Keister nodes for the sparse grid approximation and that the Karhunen–Loève expansion of the log diffusion coefficient is more appropriate than its Lévy–Ciesielski expansion for purpose of sparse grid collocation.

O. G. Ernst Department of Mathematics, TU Chemnitz, Chemnitz, Germany e-mail: [email protected] B. Sprungk Faculty of Mathematics and Computer Science, TU Bergakademie Freiberg, Freiberg, Germany e-mail: [email protected] L. Tamellini () Istituto di Matematica Applicata e Tecnologie Informatiche “E. Magenes”, Consiglio Nazionale delle Ricerche, Pavia, Italy e-mail: [email protected] © Springer Nature Switzerland AG 2021 H.-J. Bungartz et al. (eds.), Sparse Grids and Applications - Munich 2018, Lecture Notes in Computational Science and Engineering 144, https://doi.org/10.1007/978-3-030-81362-8_1

1

2

O. G. Ernst et al.

1 Introduction We consider the sparse polynomial collocation method for approximating the solution of a random elliptic boundary value problem with lognormal diffusion coefficient, a well-studied model problem for uncertainty quantification in numerous physical systems such as stationary groundwater flow in an uncertain aquifer. The assumption of a lognormal diffusion coefficient, i.e., that its logarithm is a Gaussian random field, is a common, quite simple approach for modeling uncertain conductivities with large variability in practice (a discussion on this and other, more sophisticated models for the conductivity of aquifers can be found e.g. in [34], empirical evidence for lognormality is discussed in [17]), but already yields an interesting setting from a mathematical point of view. For instance, a lognormal diffusion coefficient introduces challenges, e.g., for stochastic Galerkin methods [18, 23, 32] due to the unboundedness of the coefficient and the necessity of solving large coupled linear systems. By contrast, stochastic collocation based on sparse grids [1, 36, 37, 48] has been established as a powerful and flexible non-intrusive approximation method in high dimensions for functions of weighted mixed Sobolev regularity. The fact that solutions of lognormal diffusion problems belong to this function class has been shown under suitable assumptions in [2]. Based on the analysis in [2], we have established in [14] a dimension-independent convergence rate for sparse polynomial collocation given a mild condition on the univariate node sets. This condition is, for instance, satisfied by the classical Gauss-Hermite nodes [14]. In related work, dimension-independent convergence has also been shown for sparse grid quadrature [10]. This work is a follow-up on our previous contribution [14] and provides further discussion, insights and numerical results concerning two important design decisions for sparse polynomial collocation applied to differential equations with Gaussian random fields. The first concerns the representation of the Gaussian random field by a series expansion. A common choice is to use the Karhunen-Loève expansion [21] of the random field. Although it represents the spectral, and thus L2 -optimal, expansion of the input field, it is not necessarily the most efficient parametrization for approximating the solution field of the equation. In particular, in [2, 3] the authors advocate using wavelet-based expansions with localized basis functions. A classical example of this type is the Lévy-Ciesielski (LC) expansion of Brownian motion or a Brownian bridge [7, 11], which employs hat functions, whereas the KL expansion of the same random fields results in sinusoidal (hence smoother and globally supported) basis functions. A theoretical advantage of localized expansions of Gaussian random fields is that for these it is easier to verify the (sufficient) condition for weighted mixed Sobolev regularity of the solution of the associated lognormal diffusion problem. In this work, we conduct numerical experiments with the KL and LC expansions of a Brownian bridge as the lognormal coefficient in an elliptic diffusion equation in order to study their relative merits for sparse grid

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

3

collocation of the resulting solution. We note that finding optimal representations of the random inputs is a topic of ongoing research, see e.g. [8, 39, 46]. The second design decision we investigate is the choice of the univariate polynomial interpolation node sequences which form the building blocks of sparse grid collocation. Established schemes are Lagrange interpolation based on Gauss– Hermite or Genz–Keister nodes. However, the former are non-nested and the latter grow rapidly in number and are only available up to a certain level. In recent work, weighted Leja nodes [33] have been advocated as a suitable nested and slowly increasing node family for sparse grid approximations, see, e.g., [15, 28, 47] for recent applications in uncertainty quantification. However, so far there exist only preliminary results regarding the numerical analysis of weighted Leja points on unbounded domains, e.g., [27]. We provide numerical evidence that Gaussian Leja nodes, i.e., weighted Leja nodes with Gaussian weight, satisfy as well the sufficient condition given in [14] for dimension-independent sparse polynomial collocation. Moreover, we compare the performance of sparse grid collocation based on Gaussian Leja, Gauss–Hermite and Genz–Keister nodes for the approximation of the solution of a lognormal random diffusion equation. The remainder of the paper is organized as follows. In Sect. 2 we provide the necessary fundamentals on lognormal diffusion problems and discuss the classical Karhunen–Loève expansion of random fields and expansions based on wavelets. Sparse polynomial collocation using sparse grids are introduced in Sect. 3, where we also recall our convergence results from [14]. Moreover, we discuss the use of Gaussian Leja points for quadrature and sparse grid collocation in connection with Gaussian distributions in Sect. 3.2. Finally, in Sect. 4, we present our numerical results for sparse polynomial collocation applied to lognormal diffusion problems using the above-mentioned univariate node families and expansion variants for random fields. We draw final conclusions in Sect. 5.

2 Lognormal Elliptic Partial Differential Equations We consider a random elliptic boundary value problem on a bounded domain D ⊂ Rd with smooth boundary ∂D, − ∇ · (a(ω) ∇u(ω)) = f

in D,

u(ω) = 0 on ∂D,

P-a.s. ,

(1)

with a random diffusion coefficient a : D × Ω → R w.r.t. an underlying probability space (Ω, A , P). If a(·, ω) : D → R satisfies the conditions of the Lax–Milgram lemma [22] P-almost surely, then a pathwise solution u : Ω → H01 (D) of (1) exists. Under suitable assumptions on the integrability of amin(ω) := ess infx∈D a(x, ω) one can show that u belongs to a Lebesgue–Bochner space p LP (Ω; H01(D)) consisting of all random functions v : Ω → H01 (D) with vLp := 1/p   p v(ω) P(dω) . 1 Ω H0 (D)

4

O. G. Ernst et al.

In this paper, we consider lognormal random coefficients a, i.e., where log a : D × Ω → R is a Gaussian random field which   is uniquely determined by its mean function φ0 : D → R, φ0 (x) := E log a(x) and its covariance function c : D × D → R, c(x, x  ) := Cov(log a(x), log a(x  )). If the Gaussian random field log a has continuous paths the existence of a weak solution u : Ω → H01 (D) can be ensured. Proposition 1 ([9, Section 2]) Let log a in (1) be a Gaussian random field with a(·, ω) ∈ C (D) almost surely. Then a unique solution u : Ω → H01 (D) of (1) exists p such that u ∈ LP (Ω; H01(D)) for any p > 0. A Gaussian random field log a : D × Ω → R can be represented as a series expansion of the form log a(x, ω) = φ0 (x) +



ξm ∼ N(0, 1) i.i.d.,

φm (x) ξm (ω),

(2)

m≥1

with suitably chosen φ0 , φm ∈ L∞ (D), m ≥ 1. In general, several such expansions or expansion bases {φm }m∈N , respectively, can be constructed, cf. Sect. 2.2— thus raising the question of whether certain bases {φm }m∈N are better suited for parametrizing random fields than others. Conversely, given an appropriate system {φm }m∈N , the construction (2) will yield a Gaussian random field if we ensure that the expansion in (2) converges P-almost surely pointwise or in L∞ (D), i.e., that the Gaussian coefficient sequence (ξm )m∈N in RN with distribution μ :=  m∈N N(0, 1) satisfies μ(Γ ) = 1

where

∞

 φm ξm L∞ (D) < ∞ . Γ := ξ ∈ RN : 

(3)

m=1

We remark that Γ is a linear subspace of RN . The basic condition (3) is satisfied, for instance, if  φm L∞ (D) < ∞ (4) m≥1

and (2) then yields a Gaussian random variable in L∞ (D), see [42, Lemma 2.28] or [43, Section 2.2.1]. Given the assumption (3) we can view the random function a in (2) and the resulting pathwise solution u of (1) as functions in L∞ (D) and H01 (D), respectively, depending on the random parameter ξ ∈ Γ , i.e., a : Γ → L∞ (D) and u : Γ → H01 (D). In particular, by the Lax–Milgram lemma we have that u(ξ ) ∈ H01 (D) is well-defined for ξ ∈ Γ and u(ξ )H 1 (D) ≤ 0

CD f L2 (D) , amin (ξ )

amin (ξ ) := ess inf a(x, ξ ). x∈D

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

5

In the following subsection, we provide sufficient conditions on the series representation in (2) such that (3) holds and that the solution u : Γ → H01 (D) of (1) belongs p to a Lebesgue–Bochner space Lμ (Γ ; H01 (D)). Moreover, we discuss the regularity of the solution u of the random PDE (1) as a function of the variable ξ ∈ Γ , which governs approximability by polynomials in ξ .

2.1 Integrability and Regularity of the Solution A first result concerning the integrability of u given log a as in (2) is the following. Proposition 2 ([42, Proposition 2.34]) If the functions φm , m ∈ N, in (2) satisfy (4), then (3) holds and the solution u : Γ → H01 (D) of (1) with diffusion coefficient p a as in (2) satisfies u ∈ Lμ (Γ ; H01 (D)) for any p > 0. In [2, Corollary 2.1] the authors establish the same statements as in Proposition 2 but under the assumption that there exists a strictly positive sequence (τm )m∈N such that   sup τm |φm (x)| < ∞, exp(−τm2 ) < ∞. (5) x∈D m≥1

m≥1

Compared with (4), this relaxes the summability condition if the functions φm have local support. On the other hand, (5) implies that (|φm (x)|)m∈N decays√slightly faster than a general 1 (N)-sequence due to the required growth of τm ≥ C log m. The authors of [2] further establish a particular weighted Sobolev regularity of the solution u : Γ → H01 (D) of (1) w.r.t. ξ or ξm , respectively, assuming a stronger version of (5). To state their result, we introduce further notation. We define the partial derivative ∂ξm v(ξ ) for a function v : Γ → H01 (D) by ∂ξm v(ξ ) := lim

h→0

v(ξ + hem ) − v(ξ ) , h

when it exists, where em denotes the m-th unit vector in RN . Higher derivatives ∂ξkm v(ξ ) are defined inductively. Thus, for any k ∈ N we have ∂ξkm v : Γ → H01 (D), assuming its existence on Γ . In order to denote arbitrary mixed derivatives we introduce the set  F := k ∈ NN 0 : |k|0 < ∞ ,

|k|0 := |{m ∈ N : km > 0}|,

(6)

6

O. G. Ernst et al.

of finitely supported multi-index sequences k ∈ NN 0 . For k ∈ F we can then define 1 k the partial derivative ∂ v : Γ → H0 (D) of a function v : Γ → H01 (D) by ⎛ ∂ k v(ξ ) := ⎝



⎞ ∂ξkmm ⎠ v(ξ ),

m≥1

where the product is, in fact, finite due to the definition of F . Remark 1 It was shown in [2] that the partial derivative ∂ k u(ξ ) ∈ H01 (D), k ∈ F , of the solution u of (1) can itself be characterized as the solution of a variational problem in H01 (D):    k a(ξ ) ∇[∂ u(ξ )] · ∇v dx = φ k−i a(ξ )∇[∂ i u(ξ )] · ∇v dx i D D



k

∀v ∈ H01 (D)

ik

where i  k denotes that im ≤ km for all m ∈ N but i = k and φ i , i ∈ F , is a  im shorthand notation for the finite product m≥1 φm ∈ L∞ (D). We now state the regularity result in [2] which uses a slightly stronger assumption than (5). Theorem 1 ([2, Theorem 4.2]) Let r ∈ N and let there exist strictly positive weights τm > 0, m ∈ N such that for the functions φm , m ∈ N, in (2) and for a p > 0 we have 



log 2 τm |φm (x)| < √ r x∈D m≥1 sup

−p

τm < ∞.

(7)

m≥1

Then the solution u : Γ → H01 (D) of (1) with coefficient a as in (2) satisfies  τ 2k ∂ k u2L2 < ∞, μ k!

k∈F , |k|∞ ≤r

where τ k =

 m≥1

τmkm and k! =



km ! .

(8)

m≥1

This theorem tells us that, given (7), the partial derivatives ∂ k u : Γ → H01 (D) exist for any k ∈ F with |k|∞ < ∞ and belong to L2μ (Γ ; H01(D)). Moreover, their L2μ -norm decays faster than τ −2k —otherwise (8) would not hold. In particular, Theorem 1 establishes a weighted mixed Sobolev regularity of the solution u : Γ → H01 (D) of maximal degree r ∈ N and with increasing weights τm ≥ Cm1/p . As it turns out, it is such a regularity which ensures dimension-independent convergence rates for polynomial sparse grid collocation approximations—see the next section. Moreover, the condition (7) seems to favor localized basis functions ∞ ∞ φm for which τ |φ (x)| reduces to a summation over a subsequence m=1 m m k=1 τmk |φmk (x)|

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

7

such that (7) is easier to verify. In view of this, the authors of [2, 3] proposed using wavelet-based expansions for Gaussian random fields with sufficiently localized φm in place of the globally supported eigenmodes φm in the Karhunen–Loève (KL) expansion. In fact, condition (7) fails to hold for the KL expansion of some rough Gaussian processes (Example 1 below), but can be established if the process is sufficiently smooth (Example 2). We will discuss KL and wavelet-based expansions of Gaussian processes in more detail in the next subsection.

2.2 Choice of Expansion Bases Given a Gaussian random field log a : D × Ω → R with mean φ0 : D → R and covariance function c : D × D → R we seek a representation as an expansion (2). We explain in the following how such expansions can be derived in general. To this end, we assume that the random field has P-almost surely continuous paths, i.e., log a : Ω → C (D), and a continuous covariance function c ∈ C (D × D). Thus, we can view log a also as a Gaussian random variable with values in the separable Banach space C (D) or, by continuous embedding, with values in the separable 2 Hilbert space L2 (D). The covariance operator C : L2 (D) →  L (D) of the random 2 variable log a : Ω → L (D) is then given by (Cf )(x) := D c(x, y) f (y) dy. This operator is of trace class and induces a dense subspace HC := range C 1/2 ⊂ L2 (D), which equipped with the inner product u, vC := C −1/2 u, C −1/2 vL2 (D) , forms again a Hilbert space, called the Cameron–Martin space (CMS) of log a. The CMS plays a crucial role for series representations (2) of log a. Specifically, it is shown in [30] that (2) holds almost surely in C(D) if and only if the system {φm }m∈N is a so-called Parseval frame or (super) tight frame in the CMS of log a, i.e., if {φm }m∈N ⊂ HC and 

|φm , f C |2 = f 2C

∀f ∈ HC .

m≥1

We discuss two common choices for such frames below. Karhunen–Loève Expansions This expansion is based on the eigensystem (λm , ψm )m∈N of the compact and selfadjoint covariance operator C : L2 (D) → L2 (D) of log a. Thus, let ψm ∈ L2 (D) satisfy Cψm = λm ψm with λm > 0. Since the covariance function c is a continuous function on D × D, we have ψm ∈ C (D) and (2) holds almost surely in C (D) −1/2 with φm := λm ψm , because {φm }m∈N ⊂ HC is a complete orthonormal system (CONS) of HC . In fact, the KL basis {φm }m∈N represents the only CONS of HC that is also L2 (D)-orthogonal. In addition, as the spectral expansion of log a in L2P (L2 (D)), it is the optimal basis in this space in the sense that the truncation

8

O. G. Ernst et al.

 error after M terms  log a − φ0 − M m=1 φm ξm L2P (L2 (D)) is the smallest among all truncated expansions of length M of the form log a(x, ω) = φ0 (x) +

M 

φ˜ m (x)ξ˜m (ω).

m=1

Under additional assumptions the KL expansion also yields optimal rates of the truncation error in L2P (C (D)), see again [30]. However, the KL modes φm typically have global support on D, which often makes it difficult to verify a condition like (7). Nonetheless, for particular covariance functions, such as the Matérn kernels, bounds on the norms φm L∞ (D) are known, see, e.g., [24]. Wavelet-Based Expansions We now consider expansions in orthonormal wavelet bases {ψm }m∈N of L2 (D). Given a factorization C = SS ∗ , S : L2 (D) → L2 (D), of the covariance operator C (e.g., S = S ∗ = C 1/2 ), we can set φm := Sψm and obtain a CONS {φm }m∈N of the CMS HC , see [30]. Thus, (2) holds almost surely in C (D) with φm = Sψm . The advantage of wavelet-based expansions is that the resulting φm often inherit the localized behavior of the underlying ψm , cf. Example 1, which then facilitates verification of the sufficient condition (7) for the weighted Sobolev regularity of the solution u of (1). For instance, we refer to [3] for Meyer wavelet-based expansions of Gaussian random fields with Matérn covariance functions satisfying (7). There, the authors use a periodization approach and construct the φm via their Fourier transforms. Further work on constructing and analyzing wavelet-based expansions of Gaussian random fields includes, e.g., [6, 12, 13]. Example 1 (Brownian Bridge) A simple but useful example is the standard Brownian bridge B : D × Ω → R on D = [0, 1]. This is a Gaussian process with mean φ0 ≡ 0 and covariance function c(x, x  ) = min(x, x  ) − xx  . The associated CMS is given by HC = H01 (D) with u, vC = ∇u, ∇vL2 (D) and we have C = SS ∗ with 

 1[0,x] (y) − x f (y) dy,

1

Sf (x) := 0

f ∈ L2 (D).

The KL expansion of the Brownian bridge is given by B(x, ω) =

 m≥1 √

√

2 sin(πmx)ξ(ω), πm

ξm ∼ N(0, 1) i.i.d. , √

(9)

i.e., we have φm (x) = πm2 sin(πmx) and φm L∞ (D) = πm2 . Although the functions φm do not satisfy the assumptions of Proposition 2, existence and integrability of the solution u of (1) for log a = B is guaranteed by Proposition 1, since B has almost surely continuous paths. Concerning the condition (7) it can be

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

9

 shown that m≥1 τm |φm (x)| converges pointwise to a (discontinuous) function if τm ∈ o(m−1 ), i.e., (τm−1 )m∈N ∈ p (N) only for a p > 1, see the Appendix. However, this function turns out to be unbounded in a neighborhood of x = 0 if (τm−1 )m∈N ∈ p (N) for p ≤ 2, and numerical evidence suggests that it is also unbounded if (τm−1 )m∈N ∈ p (N) for p > 2, again see the Appendix. Thus, the KL expansion of the Brownian bridge does not satisfy the conditions of Theorem 1 for the weighted Sobolev regularity of u : Γ → H01 (D). Another classical series representation of the Brownian bridge is the Lévy– Ciesielski expansion [11]. This wavelet-based expansions uses the Haar wavelets ψm (x) = 2 /2 ψ(2 x − j ) where ψ(x) = 1[0,1/2](x) − 1(1/2,1](x) is the mother wavelet and m = 2 + j for level ≥ 0 and shift j = 0, . . . , 2 − 1. Since the Haar wavelets form a CONS of L2 (D) we obtain a Parseval frame of the CMS of the Brownian bridge by taking φm = Sψm , which yields a Schauder basis consisting of the hat functions φm (x) := 2− /2φ(2 x−j ),

φ(x) := max(0, 1−|2x−1|),

m = 2 +j,

(10)

with j = 0, . . . , 2 − 1 and ≥ 0. Hence, for log a = B the series representation (2) also holds almost surely in C (D)with φm as in (10), see  also [7, Section IX.1]. Moreover, we have φm L∞ = 2−log2 m/2 , resulting in m≥1 φm L∞ = ∞. On the other hand, due to the localization of φm we have that for any fixed x ∈ D and each level ≥ 0 there exists only one k ∈ {0, . . . , 2 − 1} such that φ2 +k (x) = 0. In particular, it can be shown that the LC expansion of the Brownian bridge satisfies √ the conditions of Theorem 1 for any p > 2, since for τm = κ log2 m with |κ| < 2 we get 

κ log2 m |φm (x)| =

m≥1



κ /2|φ2 +k (x)| ≤

l≥0

√ ( 0.5ρ) < ∞ l≥0

and for p > logκ 2 > 2  m≥1

−p

τm =



2l κ − p =

l≥0

  2κ −p < ∞. l≥0

Example 2 (Smoothed Brownian Bridge) Based on the explicit KL expansion of the Brownian bridge we can construct Gaussian random fields with smoother realizations by Bq (x, ω) =

 m≥1

√ 2 sin(πmx)ξ(ω), (πm)q

ξm ∼ N(0, 1) i.i.d. ,

q > 1. (11)

10

O. G. Ernst et al. 10 -3

0.04

10 -3

5

2

0.02 0

0

0 l=-1, k=0 l=0, k=0

-0.02 0

0.5

-2

-5

l=1, k=0 l=1, k=1

k=0

1

k=1

0

k=2

0.5

x

k=3

1

-4

k=0 k=1

0

k=2 k=3

k=4 k=5

0.5

k=6 k=7

1

x

x

Fig. 1 Expansion functions resulting from applying C 1/2 as in Example 2 for q = 3 to the Haar wavelets ψm , m = 2 + k, with level ∈ {−1, 0, 1} (left), = 2 (middle), and = 3 (right)

Now, the resulting φm =

√ 2 (πm)q

sin(πm·) indeed satisfy the assumptions of

1 Proposition 2 for any q > 1, since φm L∞ (D) ∝ m−q . Moreover, for p > q−1 the expansion (11) satisfies the assumptions of Theorem 1 with τm = m(1+)/p for sufficiently small . For  this Gaussian random field Bq the covariance function is given by c(x, y) = 2 m≥1 (πm)−2q sin(πmx) sin(πmy) and we can express C 1/2 via   C 1/2 f (x) = k(x, y) f (y) dy, k(x, y) = 2 (πm)−q sin(πmx) sin(πmy). D

m≥1

Thus, we could construct alternative expansion bases for Bq via φm = C 1/2 ψm given a wavelet CONS {ψm }m∈N of L2 (D). However, in this case the resulting φm do not necessarily have a localized support. For instance, when taking Haar wavelets ψm the C 1/2 ψm have global support in D = [0, 1], see Fig. 1.

3 Sparse Grid Approximation In [14] we presented a solution approach for solving random elliptic PDEs based on sparse polynomial collocation derived from tensorized interpolation at GaussHermite nodes. The problem is cast as that of approximating the solution u of (1) as a function u : Γ → H01 (D) by solving for realizations of u associated with judiciously chosen collocation points {ξ j }N j =1 ⊂ Γ . Sparse polynomial collocation operators are constructed by tensorizing univariate Lagrange interpolation sequences (Uk )k∈N0 defined as (Uk f )(ξ ) =

k  i=0

f (ξi(k) ) L(k) i (ξ ),

f : R → R,

(12)

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

11

(k)

where {Li }ki=0 denote the Lagrange fundamental polynomials of degree k associ  ated with a set of k + 1 distinct interpolation nodes Ξ (k) := ξ0(k) , ξ1(k) , . . . , ξk(k) ⊂ R and L0 ≡ 1. For any k  ∈ F (cf. (6)), the associated tensorized Lagrange interpolation operator Uk := m∈N Ukm is given by  (Uk f )(ξ ) =



 Ukm f

(ξ ) =



(k)

f : RN → R,

(k)

f (ξ i )Li (ξ ),

(13)

i≤k

m∈N

in terms of the tensorized Lagrange polynomials L(k) i (ξ ) := multivariate interpolation nodes RΓ → Qk , where

(k) ξi

∈

Ξ (k)

:=

×m∈N



(km ) m∈N Lim (ξm )

Ξ (km ) .

Qk := span{ξ i : 0 ≤ im ≤ km , m ∈ N},

with

We thus have Uk :

k ∈ F,

denotes the multivariate tensor-product polynomial space of maximal degree km in the m-th variable in the countable set of variables ξ = (ξm ) ∈ RN . Sparse polynomial spaces can be constructed by tensorizing the univariate detail operators Δk := Uk − Uk−1 ,

k ≥ 0,

U−1 :≡ 0,

(14)

giving Δk :=



Δkm : RΓ → Qk .

m∈N

A sparse polynomial collocation operator is then obtained by fixing a suitable set of multi-indices Λ ⊂ F and setting UΛ :=



Δi : RΓ → PΛ ,

where PΛ :=

i∈Λ



Qi .

(15)

i∈Λ

It is shown in [14] that if Λ is finite and monotone (meaning that i ∈ Λ implies that any j ∈ F for which j ≤ i holds componentwise also belongs to Λ), then UΛ is the identity on PΛ and Δi vanishes on PΛ for any i ∈ Λ. The construction of UΛ f for f : Γ → R consists of a linear combination of tensor product interpolation operators requiring the evaluation of f at certain multivariate nodes. It can be shown that for i ∈ F the detail operators have the representation Δi f =

 m≥1

 (Uim − Uim −1 ) f =



(−1)|i−k|1

i−1≤k≤i

 m≥1

 Ukm f,

12

O. G. Ernst et al.

leading to an alternative representation of UΛ for monotone finite subsets Λ ⊂ F known as the combination technique: UΛ =



c(i; Λ) Ui ,



c(i; Λ) :=

e∈{0,1}N :

i∈Λ

(−1)|e|1 .

(16)

i+e∈Λ

We refer to the collection of nodes appearing in the tensor product interpolants Ui as the sparse grid ΞΛ ⊂ Γ associated with Λ: ΞΛ =



Ξ (i) .

(17)

i∈Λ

In the same way, when approximating the solution u : Γ → H01 (D) of (1) by u(ξ ) ≈ (UΛ u)(ξ ), each evaluation u(ξ j ) at a sparse grid point ξ j ∈ ΞΛ represents the solution of the PDE for the coefficient a = a(ξ j ). Remark 2 Let us provide some further comments. 1. The univariate interpolation operators Uk in (12), on which the sparse grid collocation construction is based, will have degree of exactness k, as the associated sets of interpolation nodes Ξ (k) have cardinality k + 1. Although we do not consider this here, allowing nodal sets to grow faster than this may bring some advantages. Such an example is the sequence of Clenshaw–Curtis nodes (cf. [37]), for which |Ξ (0) | = 1 and |Ξ (k) | = 1 + 2k . 2. The Clenshaw–Curtis doubling scheme generates nested node sets Ξ (k) ⊂ Ξ (k+1) . This has the advantage that higher order collocation approximations may re-use function evaluations of previously computed lower-order approximations. Moreover, it was shown in [5] that sparse grid collocation based on nested node sequences are interpolatory. By contrast, the sequence of Gauss–Hermite nodes with |Ξ (k) | = k + 1 results in disjoint consecutive nodal sets. The number of new nodes added by each consecutive set is referred to as the granularity of the node sequence. 3. Two heuristic approaches for constructing monotone multi-index sets Λ for sparse polynomial collocation for lognormal random diffusion equations are presented in [14]. Further details are given in Sect. 4. In [14], a convergence theory for sparse polynomial collocation approximations f ≈ UΛ f of functions in f ∈ L2μ (Γ, H01 (D)) was given based on the expansion f (ξ ) =

 k∈F

 fk Hk (ξ ),

fk =

f (ξ )Hk (ξ )μ(dξ ), Γ

 in tensorized Hermite polynomials Hk (ξ ) = m∈N Hkm (ξm ), k ∈ F , with Hkm denoting the univariate Hermite orthogonal polynomial of degree km , which are known to form an orthonormal basis of L2μ (Γ ; H01(D)).

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

13

Under assumptions to be detailed below, it was shown [14, Theorem 3.12] that there exists a nested sequence of monotone multi-index sets ΛN ⊂ F , where N = |ΛN |, such that the sparse grid collocation error of the approximation UΛN f satisfies 

  f − UΛ f  2 ≤ C(1 + N)− N L μ

1 1 p−2



,

(18)

for certain values of p ∈ (0, 2) with a constant C. The precise assumptions under which (18) was shown to hold are as follows: (1) The condition μ(Γ ) = 1 on the domain of f (cf. (3)). (2) An assumption of weighted L2μ -summability on the derivatives of f : specifically, there exists r ∈ N0 such that ∂ k f ∈ L2μ (RN ; H01 (D)) for all k ∈ F with |k|∞ ≤ r and a sequence of positive numbers (τm−1 )m∈N ∈ p (N), p ∈ (0, 2), such that relation (8) holds. (3) An assumption on the univariate sequence of interpolation nodes: there exist constants θ ≥ 0 and c ≥ 1 such that the univariate detail operators (14) satisfy max Δi Hk L2μ ≤ (1 + ck)θ , i∈N0

k ∈ N0 .

(19)

In order for (18) to hold true, it is sufficient that (8) be satisfied for r > 2(θ +1)+ p2 . It was shown in [14, Lemma 3.13] that (19) holds with θ = 1 for the detail operators Δk = Uk − Uk−1 associated with univariate Lagrange interpolation operators Uk at Gauss-Hermite nodes, i.e., the zeros of the univariate Hermite polynomial of degree k + 1.

3.1 Gaussian Leja Nodes Leja points for interpolation on a bounded interval I ⊂ R are defined recursively by fixing an arbitrary initial point ξ0 ∈ I and setting ξk+1 := arg max ξ ∈I

k 

|ξ − ξi |,

k ∈ N0 .

(20)

i=1

They are seen to be nested, possessing the lowest possible granularity and have been shown to have an asymptotically optimal distribution [41, Chapter 5]. The quantity maximized in the extremal problem (20) is not finite for unbounded sets I , which arise, e.g., when an interpolation problem is posed on the entire real line. Such is the case with parameter variables ξm which follow a Gaussian distribution. By adding a weight function vanishing at infinity faster than polynomials grow, one can generalize the Leja construction to unbounded domains (cf. [29]). Different ways of

14

O. G. Ernst et al.

incorporating weights in (20) have also been proposed in the bounded case, cf. e.g. [41, p. 258], [4], and [31]. In [33], it was shown that for weighted Leja sequences generated on unbounded intervals I by solving the extremal problem ξk+1 = arg max ρ(ξ ) ξ ∈I

k 

|ξ − ξi |,

(21)

i=0

where ρ is a probability density function on I , their asymptotic distribution coincides with the probability distribution associated with ρ. This is shown in [33] for the generalized Hermite, generalized Laguerre and Jacobi weights, corresponding to a generalized Gaussian, Gamma and Beta distributions. Subsequently, the result of [45] on the subexponential growth of the Lebesgue constant of bounded unweighted Leja sequences was generalized to the unbounded weighted case in [27]. If we choose ρ(ξ ) = exp(−ξ 2 /2) and I = R in (21) and set ξ0 = 0, then we shall refer to the resulting weighted Leja nodes also Gaussian Leja nodes in view of their asymptotic distribution. Unfortunately, the result in [27] does not imply a bound like (19) for univariate interpolation using Gaussian Leja nodes. However, we provide numerical evidence in Fig. 2 suggesting that (19) is also satisfied for Gaussian Leja nodes with θ = 1. In the next subsection we compare the performance of Gaussian Leja nodes for quadrature and interpolation purposes to that of Gauss–Hermite and Genz–Keister nodes [19], which represent another common univariate node family for quadrature w.r.t. a Gaussian weight. Although a comparison of Gaussian Leja with Genz–Keister points is already available in [33] and a comparison between Gauss–Hermite and Genz–Keister points is reported in [10, 35], the joint comparison of the three choices has not been reported in literature to the best of our knowledge.

5 Gauss-Hermite Gaussian Leja

3

i

H k || L2

4

max i ||

Fig. 2 Comparison of maxi Δi Hk L2μ , k = 1, . . . , 39, for Gauss–Hermite and Gaussian Leja nodes

2 1 0

0

10 20 30 Degree k of Hermite Polynomial

40

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

15

3.2 Performance Comparison of Common Univariate Nodes In this section we investigate and compare the performance of numerical quadrature and interpolation of uni- and multivariate functions (M = 2, 6, 9 variables) using either Gauss–Hermite, Genz–Keister or Gaussian Leja nodes. As a measure of performance we consider the achieved error in relation to the number of employed quadrature or interpolation nodes, respectively. Quadrature is carried out with respect to a standard (multivariate) Gaussian measure μ and the interpolation error is measured in L2μ . The functions we consider in this section were previously proposed in [44] for the purpose of comparing univariate quadrature with Gauss–Hermite and Genz–Keister points and are included in the figures displaying the results. Quadrature results are reported in Fig. 3. In the univariate case, Gauss–Hermite nodes perform best, and Genz–Keister nodes also show good performance, which is not surprising given that they are constructed as nested extensions of the Gauss– Hermite points with maximal degree of exactness. The Gaussian Leja nodes, by comparison, perform poorly. This should not surprise, however, given that Gaussian Leja points are determined by minimizing Lebesgue constants, i.e., they are conceived as interpolation points rather than quadrature points. In the multivariate case, however, the situation changes and Gauss–Hermite nodes are the worst performing. This is due to their non-nestedness, which tends to introduce unnecessary quadrature nodes into the quadrature scheme. Note that in this case we are simply using the standard Smolyak sparse multi-index set in M dimensions in Eq. (15), ! " M  M Λw = i ∈ N : im ≤ w ,

for some w ∈ N,

m=1

i.e., we are not tailoring the sparse grid either to the function to be integrated nor to the univariate points. The Gaussian Leja and Genz–Keister points show a faster decay of the quadrature error, due to their nestedness. This is remarkable in particular for Gaussian–Leja, given that they were proposed in literature as univariate interpolation points, as already discussed. Overall, the Genz–Keister points show the best performance as expected, but it is important to recall that only a limited number of Genz–Keister nodes is available, i.e., no nested Genz–Keister quadrature formula with real quadrature weights and more than 35 nodes is known in literature, [19, 26, 44]. In particular, the plots report the largest standard sparse grids that can be built with these rules before running out of tabulated Genz–Keister points. We remark that introducing a Genz–Keister quadrature formula with more than 35 nodes is not a simple matter of investing more computational effort and tabulating more points, but it would entail some “trial and error” phase to look for a suitable sequence of so-called “generators”, see e.g. [44] for more details. This activity

16

O. G. Ernst et al.

10 0

10 0

10 0

10 -5

10 -5

10 -5

10 -10

10 -10

10 -10

10 -15

10 -15

Gauss-Hermite Gaussian Leja Genz-Keister

10 -15 0

20

40

0

0

20

40

0

0

10

-3

10 -3

10 -6

10 -6

10 -6

10 -9

10 -9

10 -9

10 -12

10 -12

10 10

10

-3

10

10 -12

Gauss-Hermite Gaussian Leja Genz-Keister

10 -15 10 0

10 2

10 -15 10 4

10 6

10 0

10 4

10 6

10 0

10 -1

10 -1

10 -3

10 -3

10 -3

10 -5

10 -5

10 -5

10 -7

10 -7

10 -7

10 -9

10 -9

10 0

Gauss-Hermite Gaussian Leja Genz-Keister

10 2

10 4

10 6

10 0

10 2

10 4

10 6

10 0

10 -1

10 -1

10 -1

10 -3

10 -3

10 -3

10 -5

10 -5

10 -5

10 -7

10 -7

10 -9

10 -9

10 -7 10 -9 10 0

Gauss-Hermite Gaussian Leja Genz-Keister

10 2

10 4

10 6

10 0

40

10 -15 10 2

10 -1

10 -9

20

0

10 2

10 4

10 6

Fig. 3 Results for univariate and multivariate quadrature test

10 0

10 2

10 4

10 6

10 2

10 4

10 6

10 2

10 4

10 6

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

17

exceeds the scope of this paper. Moreover, Genz–Keister nodes are significantly less granular, which could be a disadvantage in certain situations: indeed, the cardinalities of the univariate Genz–Keister node sets are |Ξ (k) | = 1, 3, 9, 19, 35 for k = 0, . . . , 4 (and a sequence of Genz–Keister sets exceeding 35 nodes might be even less granular, e.g., jumping from 1 to 5 or 7). Next, we turn to comparing the performance of the different node families for interpolation. Here, Gaussian Leja nodes are expected to be best (or close-to-best) performing, given their specific design. Measuring interpolation error on unbounded domains with a Gaussian measure (or any non-uniform measure for that matter) is a delicate task, as one would need to choose a proper weight to ensure boundedness of the pointwise error, see e.g. [25, 35]. In this contribution, we actually discuss the L2μ approximation error of the interpolant, which we compute as follows: we sample K independent batches of M-variate Gaussian random variables, with P points each, Bk = {ξ i }Pi=1 , ξi,m ∼ N(0, 1), m = 1, . . . , M, k = 1, . . . , K; we construct a sequence of increasingly accurate sparse grids UΛw [f ] and evaluate them on each random batch; we then approximate the L2μ error for each sparse grid on each batch by Monte Carlo, Errk (UΛw [f ]) =

P 1  (f (ξ i ) − UΛw [f ](ξ i ))2 P i=1

and then we show the convergence of the median value of the L2μ error for each sparse grid over the K repetitions.1 The results are reported in Fig. 4. The plots indicate that the convergence of interpolation degrades significantly as the number of dimension M increases (due to the simple choice of index-set Λw ), and in particular the convergence of grids based on Gauss–Hermite points is always the worst among those tested (due again to their non-nestedness), so that using nested points such as Gaussian Leja or Genz–Keister becomes mandatory. The performance of Genz–Keister points is surprisingly good, even better than Gaussian Leja at times, despite the fact that they are designed for quadrature rather than interpolation. However, the rapid growth and the limited availability of Genz– Keister points still are substantial drawbacks. To this end, we remark that also in these plots we are showing the largest grid that we could compute before running out of Genz–Keister points.

1

Exchanging the median value with the mean value does not significantly change the plots, which means that the errors are distributed symmetrically around the median. For brevity, we do not report these plots here. We have also checked that the distribution of the errors is not too spread, by adding boxplots to the convergence lines. Again, we do not show these plots for brevity. Finally, observe that we could have also employed a sparse grid to compute the L2μ error, but we chose Monte Carlo quadrature to minimize the chance that the result depends on the specific grid employed.

18

O. G. Ernst et al.

10 0

10 0

10 0

10 -5

10 -5

10 -5

10 -10

10 -10

10 -10

10 -15

10 -15

Gauss-Hermite Gaussian Leja Genz-Keister

10 -15 0

10

20

30

40

0

10

20

30

40

0

10 0

10 0

10 0

10 -3

10 -3

10 -3

10 -6

10 -6

10 -6

10 -9

10 -9

10 -9

10 -12

10 -12

10 -12

Gauss-Hermite Gaussian Leja Genz-Keister

10 -15 10 0

10 2

10 4

10 -15 10 6

10 0

10 4

10 6

10 0

10 -1

10 -1

10 -3

10 -3

10 -3

10 -5

10 -5

10 -7 10 0

Gauss-Hermite Gaussian Leja Genz-Keister

10 2

10 4

10 6

10 -7 10 0

10 2

10 4

10 6

10 -7 10 0

10 -1

10 -1

10 -1

10 -3

10 -3

10 -3

10 -5

10 -5

10 -5

10 -7 10 0

Gauss-Hermite Gaussian Leja Genz-Keister

10 2

10 4

10 6

10 -7 10 0

20

30

10 2

10 4

10 6

10 2

10 4

10 6

10 2

10 4

10 6

40

10 -15 10 2

10 -1

10 -5

10

10 2

10 4

10 6

10 -7 10 0

Fig. 4 L2μ error for univariate and multivariate interpolation. The results for the univariate test were produced with K = 30 repetitions, each with P = 100 samples. The results for the multivariate test were produced with K = 50 repetitions, each with P = 500 samples

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

19

4 Numerical Results We now perform numerical tests solving the elliptic PDE introduced in Sect. 2, with the aim of extending the numerical evidence obtained in [14]. In that paper, we assessed: • the sharpness of the predicted rate for the a-priori sparse grid construction (both with respect to the number of multi-indices in the set and the number of points in the sparse grids); • the comparison in performance of the a-priori and the classical dimensionadaptive a-posteriori sparse grid constructions; limiting ourselves to Gauss–Hermite collocation points, which are covered by our theory. The findings indicated that our predicted rates are somewhat conservative. Specifically, the rates of convergence measured in numerical experiments were larger than the theoretical ones by a factor between 0.5 and 1, cf. [14, Table 1]. This is due to some technical estimates applied in the proof of the convergence results which we were so far not able to improve. Concerning the second point, we observed in [14] that the a-priori construction is actually competitive with the aposteriori adaptive variant, especially if one considers the extra PDE solves needed to explore the set of multi-indices. We remark in particular that we observed convergence of the sparse grid approximations even in cases in which the theory predicted no convergence (albeit with a rather poor convergence rate, comparable to that attainable with Monte Carlo or Quasi Monte Carlo methods—see also [35, 38] for possible remedies). In this contribution, our goal is the numerical investigation of additional questions that so far remain unanswered by existing theory, among these: 1. whether using Gaussian Leja or Genz–Keister nodes yields improvement over Gauss–Hermite nodes in our framework, see Sect. 4.1; 2. whether changing the random field representation from Karhunen-Loève (KL) to Lévy-Ciesielski (LC) expansion for the case q = 1 (pure Brownian bridge) improves the efficiency of the numerical computations, see Sect. 4.2. As explained above, this is motivated by the fact that LC expansion of the random field allowed [2] to prove convergence of the best-N-term approximation of the lognormal problem over Hermite polynomials. The tests were performed using the Sparse Grids Matlab Kit.2 We briefly recall the basic approaches of the two heuristics employed for constructing the multi-index sets Λ. We refer to [14] for the full details of the two algorithms. The first is the classical dimension-adaptive algorithm introduced by Gerstner and Griebel in [20] with some suitable modifications to make it work with non-nested quadrature rules and for quadrature/interpolation on unbounded domains. It is driven by a posteriori 2 v.18-10 “Esperanza”, which can be downloaded under the BSD2 license at https://sites.google. com/view/sparse-grids-kit.

20

O. G. Ernst et al.

5

10

4

8

3

6

2

4

1

2

12 10 8 6 4

0

0

0.5

1

0

2

0

0.5

1

0

0

0.5

1

Fig. 5 30 realizations of the random field for different values of q. Left: q = 3; center: q = 1.5; right: q = 1. Note the different scaling of the vertical axis

error indicators computed along the outer margin of the current multi-index set. The mechanism by which new random variables are activated in the multi-index set uses a “buffer” of fixed size containing variables whose error indicators have been computed but not yet selected. The second approach is an a-priori tailored choice of multi-index set Λ, which can be derived from the study of the decay of the spectral coefficients of the solution. We thus consider the problem in Eq. (1) with f = 1. We set the pointwise standard deviation of log a to be σ = 3; note that this constant does not appear explicitly in the expression for log a in Sect. 2, i.e., it has been absorbed in φm . Figure 5 shows 30 realizations of the random field a(ω) for different values of q, obtained by truncating the Karhunen-Loève expansion of a(ω) at M = 1000 random variables. Specifically, we consider a smoothed Brownian bridge as in Example 2, with q = 3, 1.5, 1, cf. Eq. (11); for these values of q a truncation at 1000 random variables covers 100%, 99.99996% and 99.93% of the total variance of log a, respectively. The plot shows how the realizations grow increasingly rough as q decreases. Upon plotting the corresponding PDE solutions (not displayed for brevity) one would observe that, by contrast, solutions are much less rough, even in the case q = 1.

4.1 Gauss–Hermite vs. Gaussian Leja vs. Genz–Keister Nodes We begin the analysis with the comparison of the performance of Gauss–Hermite, Gaussian Leja, and Genz–Keister points. To this end, we consider random fields of varying smoothness, we choose an expansion (KL/LC) for each random field considered, and we compute the sparse grid approximation of u with the a-priori and a-posteriori dimension-adaptive sparse grid algorithm, with Gauss–Hermite, Gaussian Leja and Genz–Keister points (i.e., 6 runs per choice of random field and associated expansion). Specifically, we consider three different random field expansions, i.e., a KL expansion of the smoothed Brownian bridge with q = 3, and

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

21

a standard Brownian bridge (q = 1) expanded with either KL or LC expansion, cf. again Examples 1 and 2. We compute the error in the full L2μ (Γ ; H01(D)) norm again with a Monte Carlo sampling over 1000 samples of the random field, which has been verified to be sufficiently accurate for our purposes. These samples are generated considering a “reference truncation level” of the random field with 1000 random variables, which substantially exceeds the number of random variables active during the execution of the algorithms (which never involve more than a few hundred random variables). In the first set of results, we report the convergence of the error with respect to the number of points in the grid. The manner of counting of the points is a subtle issue and can be done in various ways. Here we consider the following different counting strategies: “incremental”: the number of points in the sparse grid ΞΛ as defined in (17), i.e., the points required to compute the application of UΛ as given in (15), “combitec”: the number of points necessary for the combination technique representation of UΛ in (16); since c(i; Λ) may be zero for some i ∈ Λ, we can omit the corresponding # Ui in (16) and consider the possibly smaller combitec sparse grid ΞΛct := i∈Λ : c(i;Λ)=0 Ξ (i) . These strategies exhaust the counting strategies for the a-priori construction; note that these two counting schemes yield different values for non-nested points (such as Gauss–Hermite), while they are identical for nested points (such as Gaussian Leja and Genz–Keister). For the a-posteriori construction, one should also further decide whether to apply these counting strategies including or excluding the indices in the margin of the current set (“I-set” and “G-set” in the legend, respectively). Note that the “I-set” choice is more representative of the “optimal index-set” computed by the algorithm, while the “G-set” is more representative of the actual computational cost incurred when running the algorithm. Results are reported in Figs. 6 and 7. Throughout this section, we use the following abbreviations in the legend of the convergence plots: GH for Gauss– Hermite, LJ for Gaussian Leja, GK for Genz–Keister. Figure 6 compares the performance of the three choices of points for the three choices of random field expansions and the two sparse grid constructions mentioned earlier (a-posteriori/apriori), in terms of L2μ -error vs. number of collocation points. Different colors identify different combination of grid constructions and counting (light blue for a-priori-incremental; red for a-posteriori-I-set-incremental; gray for a-posteriori-Gset-incremental). The results for Gauss–Hermite points are indicated by solid lines with square filled markers, those for Gaussian Leja points by solid lines with empty triangle markers, and those for Genz–Keister by dashed lines with empty diamond markers. The first and foremost observation to be made is that the Gaussian Leja performance is consistently better than Genz–Keister and Gauss–Hermite across algorithms (a-priori/a-posteriori) and test cases, while Gauss–Hermite and Genz– Keister performance is essentially identical, in agreement with what reported e.g. in [10, 35]. Only the Genz–Keister performance for the a-priori construction in the case q = 3 is surprisingly good; we do not have an explanation for this, and leave it

22

10 0

O. G. Ernst et al. KL expansion, q=3

KL expansion, q=1

LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-post, G set LJ-post, G set GK-post, G set GH-post, I set LJ-post, I set GK-post, I set

10 -2

10 -4

10 -6 0 10 10 0

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb pts KL expansion, q=3

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb pts KL expansion, q=1

10 1

10 2

10 3

10 4

10 5

nb pts LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-prior LJ-prior GK-prior GH-post, I set LJ-post, I set GK-post, I set

10 -2

10 -4

10 -6 0 10 10 0

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb pts KL expansion, q=3

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb pts KL expansion, q=1

10 1

10 2

10 3

10 4

10 5

nb pts LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-post, G set LJ-post, G set GK-post, G set GH-prior LJ-prior GK-prior

10 -2

10 -4

10 -6 0 10

10 1

10 2

10 3

nb pts

10 4

10 5

0.01 10 0

10 1

10 2

10 3

nb pts

10 4

10 5

0.01 10 0

10 1

10 2

10 3

10 4

10 5

nb pts

Fig. 6 Comparison of performance for Gaussian Leja, Genz–Keister, and Gauss–Hermite points for different test cases and different sets of multi-indices. The plots report error versus number of points. To make the visual comparison easier we split the presentation into three parts. The top row shows the two different sets produced by the a-posteriori algorithm (a-posteriori-I-set-incremental and a-posteriori-G-set-incremental). The middle row compares a-priori and a-posteriori algorithms in terms of the optimal sets produced (a-priori-incremental and a-posteriori-I-set-incremental). The bottom row compares a-priori and a-posteriori algorithms in terms of bare computational cost (apriori-incremental and a-posteriori-G-set-incremental)

to future research. Secondly, we observe that the a-priori algorithm performs worse than the a-posteriori for q = 3 (both considering the “G-set” and the “I-set”—left panel in the middle and bottom rows), while for the case q = 1 it performs worse than the a-posteriori “I-set” but better than the a-posteriori “G-set” (regardless of type of expansion—mid and right panels in the central and bottom rows). This means that while there are better choices for the index set than a-priori one (e.g., the a-posteriori “I-set”), these might be hard to derive, so that in practice it might be convenient to use the a-priori algorithm. This is in agreement with the findings reported in [14] and not surprising, given that in the case q = 1 features a larger

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

10 0

KL expansion, q=3

KL expansion, q=1

23

LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-post, I set GH-post, I set combi tec LJ-post, I set

10 -2

10 -4

10 -6 0 10

10 0

10 1

10 2

10 3

10 4

0.01 10 0

nb pts KL expansion, q=3

10 1

10 2

10 3

10 4

0.01 10 0

nb pts KL expansion, q=1

10 1

10 2

10 3

10 4

nb pts LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-post, G set LJ-post, G set GK-post, G set GH-post, I set LJ-post, I set GK-post, I set

10 -2

10 -4

10 -6 0 10 10 0

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb idx KL expansion, q=3

10 1

10 2

10 3

10 4

10 5

0.01 10 0

nb idx KL expansion, q=1

10 1

10 2

10 3

10 4

10 5

nb idx LC expansion

0.1

0.1

0.05

0.05

0.02

0.02

GH-prior LJ-prior GK-prior GH-post, I set LJ-post, I set GK-post, I set

10 -2

10 -4

10 -6 0 10

10 1

10 2

nb idx

10 3

10 4

0.01 10 0

10 1

10 2

nb idx

10 3

10 4

0.01 10 0

10 1

10 2

10 3

10 4

nb idx

Fig. 7 Top row: further analysis of influence of counting strategies in assessing the performance of Gaussian Leja, Genz–Keister, and Gauss–Hermite points. Middle and bottom rows: plot of error versus number of indices in the sparse grid set for different test cases. The plots in these two rows are grouped in the same way as in Fig. 6

number of random variables and therefore is harder to be handled by the a-posteriori algorithm. In Fig. 7 we analyze in more detail the relatively poor performance of Gauss– Hermite points. In the top row we want to investigate whether the “incremental”/“combitec” counting (which we recall produces different results only for Gauss–Hermite points) explains at least partially the gap between the Gauss– Hermite and the Gaussian Leja results in Fig. 6. To this end, we focus on the a-posteriori “I-set”. For such grid and counting, we report the convergence curves from Fig. 6 for both the Gauss–Hermite and the Gaussian Leja collocation points and add in black with filled markers the “combitec” counting, which is more favorable to Gauss–Hermite points. The plots show, however, that the counting method accounts for only a small fraction of the gap.

24

O. G. Ernst et al. 0.2

KL-LJ-post, I set LC-LJ-post, I set KL-LJ-prior LC-LJ-prior

0.1 0.08

0.2

KL-LJ-post-I set LC-LJ-post-buff5-I set LC-LJ-post-buff20-I set

LC-LJ-post, bNt LC-LJ-post-buff5, I set LC-LJ-post-buff20, I set

0.06

0.06

0.04

KL-LJ-post, I set

0.1 0.08

0.1 0.08

0.06

KL-LJ-post, bNt

0.04 0.04

0.02 10

0.02 0

10

2

10

4

0.02 10 0

nb pts

10

2

10

4

nb pts

10 0

10 2

10 4

nb idx

Fig. 8 Comparison of performance for LC and KL expansions

In the middle and bottom rows instead we investigate whether the set of multiindices chosen by the algorithm also has an influence—in other words, could it be that because of the family of points, the algorithms are “tricked” into exploring less effective index sets? To this end, we redo Fig. 6 by showing the convergence with respect to the number of multi-indices in the set Λ, instead of with respect to the number of points. The plots show that in this setting, there is essentially no difference in performance between Gauss–Hermite, Gaussian Leja and Genz– Keister points (again, excluding the case of Genz–Keister points for a-priori construction in the case q = 3), which means that the sets obtained by the apriori/a-posteriori algorithm, while different, are “equally good” in approximating the solution.3 Thus, the consistent difference between Gaussian Leja, Genz–Keister and Gauss–Hermite nodes is really due to the nestedness of the former two choices. Between the two choices of nested points, the Gaussian Leja points are more granular and easier to compute up to an arbitrary number: in conclusion, they appear to be a more suitable choice of collocation points for the lognormal problem in terms of accuracy versus number of points.

4.2 KL vs. LC Expansion The second set of tests aims at assessing whether expanding the random field over the wavelet basis (LC expansion) brings any practical advantage in convergence of the sparse grid algorithm over using the standard KL expansion. Since from the previous discussion we know that Gaussian Leja nodes are more effective than Gauss–Hermite and Genz–Keister points, we only consider Gaussian Leja points in this section. Results are reported in Fig. 8. In the left plot, we compare the convergence of the error versus number of points for the a-priori and a-posteriori “I-set” for LC 3

Incidentally, note that the a-priori algorithm doesn’t take into account the kind of univariate nodes that will be used to build the sparse grids. Also note that of course the convergence of Gaussian Leja with respect to either number of points or number of multi-indices is identical, given that each multi-index adds one point.

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

25

and KL expansion; we employ the same color-coding as in Fig. 6 (blue for prior construction, red for the “I-set” of the a-posteriori construction), using filled markers for LC results and empty markers for KL results. The lines with filled markers are always significantly above the lines with empty markers, i.e., the convergence of the sparse grid adaptive algorithm is significantly faster for the KL expansion than for the LC expansion. This can easily be explained by the implicit ordering introduced by the KL expansion in the importance of the random variables: because the modes of the KL are ordered in descending order according to the percentage of variance of the random field they represent, they are already ordered in a suitable way for the adaptive algorithm, which from the very start can explore informative directions of variance (although the KL expansion is optimized for the representation of the input rather than for the output). The LC expansion instead uses a-priori choices of the expansion basis functions and in particular batches (of increasing cardinality) of those basis functions are equally important (i.e., the wavelets at the same refinement level). On the other hand, the adaptive algorithm explores random variables in the expansion order, which means that sometimes the algorithm has to include “unnecessary” modes of the LC expansion before finding those that really matter. Of course, a careful implementation of the adaptive algorithm can, to a certain extent, mitigate this issue. In particular, increasing the size of the buffer of random variables (cf. the description at the beginning of Sect. 4) improves the performance of the adaptive algorithm. The default number of inactive random variables is 5— the convergence lines in the left plot are obtained in this way. In the middle plot we confirm that, as expected, increasing the buffer from 5 to 20 random variables improves the performance of the sparse grid approximation when applied to the LC case (black line with filled markers instead of red line with filled markers). Note, however, that a significant gap remains between the convergence of the sparse grid approximation for the LC expansion with a buffer of 20 random variables and the convergence of the sparse grid for the KL expansion. This means that not only does the buffer play a role, but the KL expansion is overall a more convenient basis to work with. This aspect is further elaborated in the right plot of Fig. 8. Here we show the convergence of the sparse grid approximation for KL (5-variable buffer) and LC (either 5-variable or 20-variable buffer) against the number of indices in the sparse grids (dashed lines with markers), and compare this convergence against an estimate of the corresponding best-N-term (bNt) expansion of the solution in Hermite polynomials (full lines without markers); different colors identify different expansions. Of course, the convergence of the bNt expansion also depends on the LC/KL basis, therefore we show two bNt convergence curves. The bNt was computed by converting the sparse grid into the equivalent Hermite expansion (see [16, 40] for details) and then rearranging the Hermite coefficients in order of decreasing magnitude. The plot shows that the sparse grid approximation of the solution by KL expansion is quite close to the bNt convergence (blue lines), which means that there is not much room for “compressibility” in the sparse grid approximation. Conversely, the 5-variable-buffer sparse grid approximation of the problem with LC expansion is somehow far from the bNt (red lines) and only the

26

O. G. Ernst et al. 150 * N

10000

6000 4000 2000 0

KL LC, buff=5 LC, buff=20

0

500

1000

100

50

0

KL LC, buff=5 LC, buff=20

0

500

1000

max component of

M explored

nb idx in red-margin

8000

6 4 2

KL

0

500

6 4 2

1000 LC, buff=5

0

500

6 4 2

1000 LC, buff=20

0

500

1000

Fig. 9 Evolution of the multi-index set Λ for LC and KL expansions along iterations of the dimension-adaptive algorithm

20-variable-buffer (black dashed line) gets reasonably close: this means that the 5-variable-buffer is “forced” to add to the approximation “useless” indices merely because the ordering of the variables in the LC expansion is not optimal and the buffer is not large enough. Finally, we report in Fig. 9 some performance indicators for the construction of the index set for the KL and LC cases, which offer further insight towards explaining the superior KL performance. The figure on the left shows the growth of the size of the outer margin of the dimension-adaptive algorithm at each iteration, where we recall that one iteration is defined as the process of selecting one index from the outer margin and evaluating the error indicator for all its forward neighbors; this in particular means that the number of PDE solves per iteration is not fixed. All three algorithms (KL, 5-variable-buffer LC and 20-variable-buffer LC) stop after 10,000 PDE solves. KL displays the fastest growth in the outer margin size, followed by LC20 and then LC5, which is perhaps counter-intuitive; on the other hand, the more indices are considered, the more likely it is to find ones “effective” in reducing the approximation error. The figure in the center shows the growth in the number of explored dimensions: again, KL has the quickest and steadiest growth, which means that the algorithm favors adding new variables over exploring those already active. This might be again counter-intuitive, but there is no contradiction between this observation and the superior performance of KL: the point here is actually precisely the fact that the LC random variables are not conveniently sorted, so the algorithm is obliged to explore those already available rather than adding new ones; this is especially visible for the LC5 case, which displays a significant plateau in the growth in the number of variables in the middle of the algorithm execution. The three plots on the right finally show the largest component of multi-index ν ∗N that has been selected from the reduced margin at iteration N for the three algorithms (from the top: KL, LC5, LC20): a large maximum component means that the algorithm has favored exploring variables already activated, while if the maximum component is equal to 2 the algorithm has activated a new random variable (indices start from 1 in the Sparse Grids Matlab Kit). Most of the values in these plots are between 2 and 3, which again shows that the algorithms favor adding new variables rather than exploring those already available. Finally, we mention (plot omitted for brevity)

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

27

that despite the relatively large number of random variables activated, each tensor grid in the sparse grid construction is at most 4-dimensional,4 which means that interactions between five or more of the random variables appearing in the KL or LC expansion, respectively, are considered negligible by the algorithm.

5 Conclusions In this contribution we have investigated some practical choices related to the numerical approximation of random elliptic PDEs with lognormal diffusion coefficients by sparse grid collocation methods. More specifically, we discussed two issues, namely (a) whether it pays off from a computational point of view to replace the classical Karhunen–Loève expansion of the log-diffusion field with the Lévy– Ciesielski expansion, as advocated in [2] for theoretical purposes and (b) what type of univariate interpolation node sequence should be used in the sparse grid construction, choosing among Gauss—Hermite, Gaussian Leja and Genz—Keister points. Following a brief digression into the issue of convergence of interpolation and quadrature of univariate and multivariate functions based on these three classes of nodes, we compared the performance of sparse grid collocation for the approximate solution of the lognormal random PDEs in a number of different cases. The computational experiments suggest that Gaussian Leja collocation points, due to their approximation properties, granularity and nestedness, are the superior choice for the sparse grid approximation of the random PDE under consideration, and that the Karhunen–Loève expansion offers a computationally more effective parametrization of the input random field than the Lévy–Ciesielski expansion. Acknowledgments The authors would like to thank Markus Bachmayr and Giovanni Migliorati for helpful discussions and Christian Jäh for Proposition 3. Björn Sprungk is supported by the DFG research project 389483880. Lorenzo Tamellini has been supported by the GNCS 2019 project “Metodi numerici non-standard per PDEs: efficienza, robustezza e affidabilità” and by the PRIN 2017 project “Numerical Analysis for Full and Reduced Order Methods for the efficient and accurate solution of complex systems governed by Partial Differential Equations”.

Appendix We show that the Karhunen–Loève expansion of the Brownian bridge discussed in Example 1 does not satisfy the conditions of Theorem 1 for p > 0. To this end, we first state

4 In other words, out of the M random variables considered, only four are simultaneously activated to build the tensor grids—which four of course depends on each tensor grid.

28

O. G. Ernst et al.

Proposition 3 Let (bm )m∈N be a monotonically decreasing sequence of real numbers with limm→∞ bm = 0. Then for any θ ∈ [0, 2π] we have 

bm sin(mθ ) < ∞.

m≥1

Proof Dirichlet’s test for the convergence of series implies the statement if there exists a constant K < ∞ such that $ M $ $ $ $ $ sin(mθ )$ ≤ K ∀M ∈ N. $ $ $ m=1

Now, Lagrange’s trigonometric identity tells us that M 

sin(mθ ) =

m=1

cos ((M + 0.5)θ ) 1 cot(0.5θ ) − , 2 2 sin(0.5θ )

θ ∈ (0, 2π).

Hence, since sin(m0) = sin(m2π) = 0 the statement follows easily.

 

Proposition 4 Given the Karhunen–Loève expansion of the Brownian bridge as in (9), the function kτ (x) :=

∞  m=1

τm

√ 2 sin(mπx), πm

x ∈ D = [0, 1],

is pointwise well-defined for τm = m1/q with q > 1 in which case (τm−1 )m∈N ∈ p (N) for any p > q > 1. However, assuming that kτ : [0, 1] → R is well-defined for a sequence τ = (τm )m∈N with (τm−1 )m∈N ∈ p (N) for a p ≤ 2, then kτ ∈ / L∞ (D). √

Proof The first statement follows by Proposition 3 and πm2 τm = Cm1/q−1 → 0 as m → ∞. The second statement follows by contracdiction. Assume that kτ ∈  τm2 L∞ (D), then also kτ ∈ L2 (D) and via kτ L2 (D) = π12 ∞ m=1 m2 we have that −p

τm2 ≤ cm for a c ≥ 0—otherwise kτ L2 (D) = +∞. Thus, τm ≥ c−p/2 m−p/2 and  since m≥1 m−p/2 < +∞ if and only if p > 2, we end up with (τm−1 )m∈N ∈ / 2 (N).  

For values p > 2 we provide the following numerical evidence: we choose τm = m1/p , i.e., (τm−1 )m∈N ∈ p+ (N),  > 0, and compute the values of the function κτ (x) as given in Proposition 4 in a neighborhood of x = 0 numerically. The reason we are interested in small values of x is the fact that κτ (x), x = 0, can be bounded 1 by 12 cot(0.5πx) + 2 sin(0.5πx) by means of Proposition 3. Thus, we expect a blowup for small values of x. Indeed, we observe numerically that κτ (x) for τm = m1/p

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE p = 2.5 p=5 p = 10 p = 20

101 (x)

Fig. 10 Growth of κτ (x) as given in Proposition 4 for decaying x → 0+ and choices τm = m1/p with various values of p—the observed growth matches x −1/p

29

100 10-4

10-2 x

behaves like x −1/p for small values of x > 0, see Fig. 10. This implies that κτ is unbounded in a neighborhood of x = 0 for any of the above choices of τm and, therefore, does not satisfy the conditions of Theorem 1.

References 1. I. Babuška, F. Nobile, and R. Tempone. A stochastic collocation method for elliptic partial differential equations with random input data. SIAM Review, 52(2):317–355, June 2010. 2. M. Bachmayr, A. Cohen, R. DeVore, and G. Migliorati. Sparse polynomial approximation of parametric elliptic PDEs. part II: lognormal coefficients. ESAIM Math. Model. Numer. Anal., 51(1):321–339, 2016. 3. M. Bachmayr, A. Cohen, and G. Migliorati. Representations of Gaussian random fields and approximation of elliptic PDEs with lognormal coefficients. J. Fourier Anal. Appl., 18:621– 649, 2018. 4. J. Baglama, D. Calvetti, and L. Reichel. Fast Leja points. Electronic Transactions on Numerical Analysis, 7:124–140, 1998. 5. V. Barthelmann, E. Novak, and K. Ritter. High dimensional polynomial interpolation on sparse grids. Advances in Computational Mathematics, 12:273–288, 2000. 6. A. Benassi, S. Jaffard, and R. D. Elliptic Gaussian random processes. Revista Mathemática Iberoamericana, 13:19–90, 1997. 7. R. Bhattacharya and E. C. Waymire. A Basic Course in Probability Theory. Springer, Cham, 2nd edition, 2016. 8. B. Bohn, M. Griebel, and J. Oettershagen. Optimally rotated coordinate systems for adaptive least-squares regression on sparse grids. In Proceedings of the 2019 SIAM International Conference on Data Mining, pp. 163–171, SIAM, 2019. https://doi.org/10.1137/1.9781611975673. 19 9. J. Charrier. Strong and weak error estimates for elliptic partial differential equations with random coefficients. SIAM Journal on Numerical Analysis, 50(1):216–246, 2012. 10. Chen, P. Sparse quadrature for high-dimensional integration with Gaussian measure. ESAIM: M2AN, 52(2):631–657, 2018. 11. Z. Ciesielski. Hölder condition for realization of gaussian processes. Trans. Amer. Math. Soc., 99:403–464, 1961.

30

O. G. Ernst et al.

12. F. E. Elliot, D. J. Horntrop, and A. J. Majda. A Fourier–wavelet Monte Carlo method for fractal random fields. Journal of Computational Physics, 132:384–408, 1994. 13. F. E. Elliot and A. J. Majda. A wavelet Monte Carlo method for turbulent diffusion with many spatial scales. Journal of Computational Physics, 113:82–111, 1994. 14. O. Ernst, B. Sprungk, and L. Tamellini. Convergence of sparse collocation for functions of countably many Gaussian random variables (with application to elliptic PDEs). SIAM J. Numer. Anal., 56(2):877–905, 2018. 15. I.-G. Farcas, J. Latz, E. Ullmann, T. Neckel, and H.-J. Bungartz. Multilevel adaptive sparse Leja approximations for Bayesian inverse problems. SIAM Journal on Scientific Computing, 42(1):A424–A451, 2020. 16. L. Formaggia, A. Guadagnini, I. Imperiali, V. Lever, G. Porta, M. Riva, A. Scotti, and L. Tamellini. Global sensitivity analysis through polynomial chaos expansion of a basin-scale geochemical compaction model. Computational Geosciences, 17(1):25–42, 2013. 17. R. A. Freeze. A stochastic-conceptual analysis of one-dimensional groundwater flow in nonuniform homogeneous media. Water Resources Research, 11(5):725–741, 1975. 18. J. Galvis and M. Sarkis. Approximating infinity-dimensional stochastic Darcy’s equations without uniform ellipticity. SIAM J. Numer. Anal., 47(5):3624–3651, 2009. 19. A. Genz and B. D. Keister. Fully symmetric interpolatory rules for multiple integrals over infinite regions with Gaussian weight. Journal of Computational and Applied Mathematics, 71(2):299–309, 1996. 20. T. Gerstner and M. Griebel. Dimension-adaptive tensor-product quadrature. Computing, 71(1):65–87, 2003. 21. R. Ghanem and P. Spanos. Stochastic Finite Elements: A Spectral Approach. Springer-Verlag, New York, 1991. 22. D. Gilbarg and N. S. Trudinger. Elliptic Partial Differential Equations of Second Order. Springer-Verlag, Berlin Heidelberg, 2001. 23. C. J. Gittelson. Stochastic Galerkin discretization of the log-normal isotropic diffusion problem. Math. Models Methods Appl. Sci., 20(2):237–263, 2010. 24. I. G. Graham, F. Y. Kuo, J. A. Nichols, R. Scheichl, C. Schwab, and I. H. Sloan. Quasi-Monte Carlo finite element methods for elliptic PDEs with lognormal random coefficients. Numer Math., 131:329–368, 2015. 25. H. Harbrecht, M. Peters, and M. Siebenmorgen. Multilevel accelerated quadrature for pdes with log-normally distributed diffusion coefficient. SIAM/ASA Journal on Uncertainty Quantification, 4(1):520–551, 2016. 26. F. Heiss and V. Winschel. Likelihood approximation by numerical integration on sparse grids. J. Econometrics, 144(1):62–80, 2008. 27. P. Jantsch, C. G. Webster, and G. Zhang. On the Lebesgue constant of weighted Leja points for Lagrange interpolation on unbounded domains. IMA Journal of Numerical Analysis, 39(2):1039–1057, 2019. 28. D. Loukrezis and H. De Gersem. Approximation and Uncertainty Quantification of Stochastic Systems with Arbitrary Input Distributions using Weighted Leja Interpolation. Algorithms, 13(3):51, 2020. 29. D. S. Lubinsky. A survey of weighted polynomial approximation with exponential weights. Surveys in Approximation Theory, 3:1–105, 2007. 30. H. Luschgy and G. Pagès. Expansions for Gaussian processes and Parseval frames. Electronic Journal of Probability, 14(42):1198–1221, 2009. 31. S. D. Marchi. On Leja sequences: some results and applications. Applied Mathematics and Computation, 152:621–647, 2004. 32. A. Mugler and H.-J. Starkloff. On the convergence of the stochastic Galerkin method for random elliptic partial differential equations. ESAIM: Mathematical Modelling and Numerical Analysis, 47(5):1237–1263, 2013. 33. A. Narayan and J. D. Jakeman. Adaptive Leja sparse grid constructions for stochastic collocation and high-dimensional approximation. SIAM Journal on Scientific Computing, 36(6):A2952–A2983, 2014.

Expansions and nodes for sparse grid collocation for Lognormal elliptic PDE

31

34. S. Neuman, M. Riva, and A. Guadagnini. On the geostatistical characterization of hierarchical media. Water Resources Research, 44(2), 2008. 35. F. Nobile, L. Tamellini, F. Tesei, and R. Tempone. An adaptive sparse grid algorithm for elliptic PDEs with lognormal diffusion coefficient. In Sparse Grids and Applications – Stuttgart 2014. Springer-Verlag, 2016. 36. F. Nobile, R. Tempone, and C. Webster. An anisotropic sparse grid stochastic collocation method for partial differential equations with random input data. SIAM Journal on Numerical Analysis, 46(5):2411–2442, 2008. 37. F. Nobile, R. Tempone, and C. Webster. A sparse grid stochastic collocation method for partial differential equations with random input data. SIAM Journal on Numerical Analysis, 46(5):2309–2345, 2008. 38. F. Nobile and F. Tesei. A Multi Level Monte Carlo method with control variate for elliptic PDEs with log-normal coefficients. Stochastic Partial Differential Equations: Analysis and Computations, 3(3):398–444, 2015. 39. I. Papaioannou, M. Ehre, and D. Straub. PLS-based adaptation for efficient PCE representation in high dimensions. Journal of Computational Physics, 387:186 – 204, 2019. 40. G. Porta, L. Tamellini, V. Lever, and M. Riva. Inverse modeling of geochemical and mechanical compaction in sedimentary basins through polynomial chaos expansion. Water Resources Research, 50(12):9414–9431, 2014. 41. E. B. Saff and V. Totik. Logarithmic Potentials with External Fields, volume 316 of Grundlehren der mathematischen Wissenschaften. Springer, 1997. 42. C. Schwab and C. Gittelson. Sparse tensor discretizations of high-dimensional parametric and stochastic PDEs. Acta Numerica, 20:291–467, 2011. 43. B. Sprungk. Numerical Methods for Bayesian Inference in Hilbert Spaces. PhD thesis, TU Chemnitz, 2017. 44. L. Tamellini. Polynomial Approximation of PDEs with Stochastic Coefficients. PhD thesis, Politecnico di Milano, 2012. 45. R. Taylor and V. Totik. Lebesgue constants for Leja points. IMA Journal of Numerical Analysis, 30:462–486, 2010. 46. R. Tipireddy and R. Ghanem. Basis adaptation in homogeneous chaos spaces. Journal of Computational Physics, 259:304 – 317, 2014. 47. L. M. M. van den Bos, B. Sanderse, W. A. A. M. Bierbooms, and G. J. W. van Bussel. Bayesian model calibration with interpolating polynomials based on adaptively weighted Leja nodes. Communications in Computational Physics, 27(1):33–69, 2019. 48. D. Xiu and J. Hesthaven. High-order collocation methods differential equations with random inputs. SIAM Journal on Scientific Computing, 37(3):1118–1139, 2005.

Sparse Grids Approximation of Goldstone Diagrams in Electronic Structure Calculations Heinz-Jürgen Flad and Gohar Flad-Harutyunyan

Abstract Coupled cluster theory provides hierarchical many-particle models and is presently considered as the ultimate benchmark in quantum chemistry. Despite its practical significance, a rigorous mathematical analysis of coupled cluster models is still in its infancy. The present work focuses on nonlinear models within the random phase approximation (RPA). Solutions of these models are commonly represented by series of a particular class of Goldstone diagrams so-called RPA diagrams. Using techniques from singular analysis, we provide a connection between RPA diagrams and classical pseudo-differential operators which enables an efficient treatment of the linear and nonlinear interactions in these models. Based on a detailed asymptotic analysis, we discuss the computational complexity of adaptive sparse grids approximation of RPA diagrams in the framework of best N-term approximation theory. In particular, we determine the Besov regularity of RPAdiagrams and provide corresponding convergence rates.

1 Introduction Coupled cluster (CC) theory, cf. [1, 2, 25, 26, 34] and references therein, provides an hierarchical but intrinsically nonlinear approach to many-particle systems which enables systematic truncation schemes reflecting both, the physics of the problem under consideration as well as the computational complexity of the resultant equations. Unfortunately rigorous insights concerning the corresponding errors of

H.-J. Flad () Zentrum Mathematik, Technische Universität München, Garching, Germany e-mail: [email protected] G. Flad-Harutyunyan Mathematisch-Geographische Fakultät, Katholische Universität Eichstätt-Ingolstadt, Eichstätt, Germany Present Address: Institut für Mathematik, RWTH Aachen, Aachen, Germany e-mail: [email protected]; [email protected] · © Springer Nature Switzerland AG 2021 H.-J. Bungartz et al. (eds.), Sparse Grids and Applications - Munich 2018, Lecture Notes in Computational Science and Engineering 144, https://doi.org/10.1007/978-3-030-81362-8_2

33

34

H.-J. Flad and G. Flad-Harutyunyan

approximation are hard to achieve and a detailed understanding of the properties of solutions is still missing, see however recent progress on the subject by Schneider, Rohwedder, Kwaal and Laestadius [27, 35–37]. A key aspect, in order to perform a rigorous analysis of the computational complexity of CC models, is the asymptotic behaviour of solutions near coaslescence points of electrons. The corresponding asymptotic analysis for eigenfunctions of the many-particle Schrödinger equation has a long tradition [15, 16, 21–24], starting with the seminal work of Kato [24] and culminated in the analyticity result of M. and T. Hoffmann-Ostenhof, Fournais and Sørensen [16]. Although CC theory originates from the many-particle Schrödinger equation, it is not simply possible to transfer the results to solutions of specific CC models, because these models represent highly nonlinear approximations to Schrödinger’s equation. The peculiar structure of linear and nonlinear interactions in CC models requires a new approach based on pseudo-differential operator algebras which has been developed in our previous work [9, 10]. We considered these operator algebras in the framework of a commonly employed series expansion of solutions in terms of so-called Goldstone diagrams, cf. [28] for further details. Our main result for a popular CC model, presented in Sect. 2, affords the classification of these Goldstone diagrams in terms of symbol classes of classical pseudo-differential operators and provides their asymptotic expansions near coalescence points of electrons.

1.1 Nonlinear Models of Electron Correlation In the following we want to focus on the SUB-2 approximation [2] and discuss the properties of simplified models which contain the leading contributions to electron correlation. The quantities of interest are so called pair-amplitudes     τij : R3 ⊗ s ⊗ R3 ⊗ s → R, (x1 , x2 ) → τij (x1 , x2 ), where indices i, j, k, l refer to the occupied orbitals 1, 2, . . . N with respect to an underlying mean-field theory like Hartree-Fock, and s := {α, β} denotes spin degrees of freedom. The most general model we want to consider in the present work is given by a system of nonlinear equations for pair-amplitudes, cf. [29], which is of the form   (2) (1) Q f1 + f2 − i − j τij (x1 , x2 ) = −QV[i,j ] (x1 , x2 )  (2) (1) −QV (2) (x1 , x2 )τij (x1 , x2 ) − 14 Q τkl (x1 , x2 )V[k,l] , Ψij  (2) +Pij12 Q



k,l (1)

τik (x1 , x2 )Vkj (x2 )

(3)

k

−Pij12 Q

 k

(2)

τik (x1 , x3 )Vkj (x3 , x2 ) dx3

(4)

Goldstone Diagrams and Sparse Grids

− 12 Pij12 Q

 

35 (2)

τik (x1 , x3 )V[k,l] (x3 , x4 )τlj (x4 , x2 ) dx3 dx4 .

(5)

k,l

In order to simplify our discussion, let us assume in the following smooth electronnuclear potentials, originating e.g. from finite nucleus models. Before we enter into a brief discussion of the underlying physics of the equations let us start with some technical issues. In order to keep formulas short we use in (3), (4) and (5) the permutation operator Pij12 := 1 + (21)(j i) − (12)(j i) − (21)(ij ). Equations for pair-amplitudes are formulated on a subspace of the underlying twoparticle Hilbert space which is characterized by the projection operator Q := (1 − q1 )(1 − q2 ) with q :=

N 

|φi φi |,

(6)

i=1

where φi , with i = 1, 2, . . . , N, represent occupied orbitals. The operator Q is commonly known as strong orthogonality operator [41]. Due to the presence of this operator, pair-amplitudes rely on the constraint Qτij (x1 , x2 ) = τij (x1 , x2 ).

(7)

The physical reason behind Q is Pauli’s principle which excludes the subspace assigned to the remaining N − 2 particles from the Hilbert space of the pair and an orthogonality constraint between the mean field part (1)

Ψij (x1 , x2 ) := φi (x1 )φj (x2 ) − φj (x1 )φi (x2 ) and the corresponding pair-amplitude τij . In order to set the equations for pair-amplitudes into context let us first consider the case of Eq. (1), where the parts (2), (3), (4) and (5) have been neglected. What remains in this case is first-order Møller-Plesset perturbation theory which provides second- and third-order corrections to the energy. Going further to Eq. (2) one recovers the dominant contributions to short-range correlation, so-call particle ladder diagrams. Neglecting interactions between different electron pairs altogether in (2) to (5), one recovers the Bethe-Goldstone equation   (ij ) (2) Q f1 + f2 − i − j τij (x1 , x2 ) = −QV[i,j ] (x1 , x2 ) − QVfluc (x1 , x2 )τij (x1 , x2 ),

36

H.-J. Flad and G. Flad-Harutyunyan

here the fluctuation potential is given by (ij )

Vfluc (x1 , x2 ) :=

1 (i,j ) (i,j ) (2) (1) − vH x (x1 ) − vH x (x2 ) + 12 V[i,j ] , Ψij  |x1 − x2 |

(8)

where (i,j )

(j )

(j )

(i) vH x := v(i) H + vH + vx + vx

(9)

represents the contribution of orbitals i, j to the Hartree and exchange potential, respectively. The asymptotic behaviour of solutions for both models can be directly derived from the structure of asymptotic parametrices of Hamiltonian operators and has been already discussed in Ref. [12] in some detail. The three remaining terms (3), (4) and (5) of the effective pair-equation take part in the so called random phase approximation (RPA) which is essential for the correct description of long-range correlations. In the following we want to consider also some simplified versions of RPA where exchange contributions are neglected. Such models are of particular significance with respect to applications of RPA as a post DFT model, cf. [17]. It has been shown that these models are equivalent to solving particular variants of CC-RPA equations, cf. [39]. The terms (1), (2), (3), (4) and (5) contain various effective interaction potentials Vkj(1) (x2 ) :=



1 φk (x3 )φj (x3 ) dx3 , |x2 − x3 |

(10)

1 , |x1 − x2 |

(11)

V (2) (x1 , x2 ) := (2)

Vij (x1 , x2 ) := φi (x1 )V (2) (x1 − x2 )φj (x2 ),

(12)

(2) (2) (2) V[i,j ] (x1 , x2 ) := Vij (x1 , x2 ) − Vj i (x1 , x2 ).

(13)

With respect to the SUB2 level of approximation, the RPA model considered in the present work is still incomplete insofar as various terms present in the full SUB-2 model are still missing. These missing nonlinear terms, however, do not contribute anything new from the point of view of the following asymptotic singular analysis. Taking them into account would only render our presentation unnecessarily complicated. Therefore our RPA model represents a good choice in order to study the properties of pair-amplitudes in some detail and in particular the effect of non linearity which enters into our model via the coupling term (5). The nonlinear character of our model is not quite of the form familiar from the theory of nonlinear partial differential equations, cf. [18], and we will therefore consider an unconventional approach to tackle the problem. Our approach reflects not only the particular character of the various coupling terms but also the singular structure

Goldstone Diagrams and Sparse Grids

37

of the interactions and pair-amplitudes. The RPA terms (4) and (5) resemble to compositions of kernels of integral operators and it is tempting to consider these terms in the wider context of an appropriate operator algebra. It will be shown in the following, that the algebra of classical pseudo-differential operators provides a convenient setting. As a complementary approach, we have studied pair-amplitudes in the framework of weighted Sobolev spaces with asymptotics which we have already considered in the context of singular analysis in order to determine the asymptotic behaviour near coalescence points of electrons, cf. Ref. [10, 12]. Both seemingly disparate approaches complement one another in the asymptotic singular analysis of RPA models.

1.2 Iteration Schemes and Their Diagrammatic Counter Parts The Bethe-Goldstone equation and various nonlinear RPA models, discussed above, are commonly solved in an iterative manner. Before we delve into the technicalities of our approach let us briefly discuss iteration schemes and their physical interpretation in a rather simple and informal manner in order to outline certain essential features of the present work. To simplify our notation, occupied orbital indices i, j, k, l and spin degrees of freedom which appear in pair-amplitudes and interaction potentials have been dropped because they are not relevant in the following discussion. The focus is in particular on linear terms like fV (2) τ (x1 , x2 ) := V (2) (x1 , x2 ) τ (x1 , x2 ),

(14)

  fV (1) τ (x1 , x2 ) := V (1) (x1 ) + V (1) (x2 ) τ (x1 , x2 ),

(15)

 fV (2) ◦τ (x1 , x2 ) :=

V (2) (x1 , x3 ) τ (x3 , x2 ) dx3

and nonlinear terms of the form  fτ ◦V (2) ◦τ (x1 , x2 ) := τ (x1 , x3 )V (2) (x3 , x4 )τ (x4 , x2 ) dx3 dx4

(16)

(17)

for a certain asymptotic type of the pair-amplitude τ . With these definitions at hand, let us briefly outline a suitable fixed point iteration scheme which illustrates some main issues of our approach. Actually, this ansatz for the solution of nonlinear CC type equations represents a canonical choice in numerical simulations. The basic structure of our problem can be represented by the greatly simplified nonlinear equation Aτ = −V (2) − fV (2) τ + fV (1) τ − fV (2) ◦τ − fτ ◦V (2) ◦τ ,

(18)

38

H.-J. Flad and G. Flad-Harutyunyan

where A is an elliptic second order partial differential operator. A simple iteration scheme for this equation may consist of the following steps. First solve the equation Aτ0 = −V (2) with fixed right hand side. Calculate fV (2) τ0 , fV (1) τ0 , fV (2) ◦τ0 and fτ0 ◦V (2) ◦τ0 and solve in the next iteration step Aτ1 = −V (2) − fV (2) τ0 + fV (1) τ0 − fV (2) ◦τ0 − fτ0 ◦V (2) ◦τ0 .

(19)

The last two steps can be repeated generating a iterative sequence of linear equations Aτn+1 = −V (2) − fV (2) τn + fV (1) τn − fV (2) ◦τn − fτn ◦V (2) ◦τn .

(20)

which can be solved in a consecutive manner until convergence of the sequence τ1 , τ2 , τ3 . . . has been achieved. Such an iteration scheme is rather convenient for an asymptotic analysis of the solutions, see e.g. [14] where such an analysis has been actually performed for the nonlinear Hartree-Fock model. Apparently, the basic problem of such kind of iteration scheme is to state necessary and sufficient conditions for its convergence. Whereas in the case of the Hartree-Fock model convergence has been proven for certain iteration schemes, the situation is actually less satisfactory in CC theory, cf. [36, 37]. In concrete physical and chemical applications, non-convergence of an iteration scheme can be usually addressed to specific properties of the system under consideration. Therefore let us simply assume in the following that our iteration scheme is actually convergent and we focus on the asymptotic properties of solutions of intermediate steps. Within the present work, we are mainly interested in the asymptotic behaviour of iterated pair-amplitudes τi , i = 0, 1, . . ., near coalescence points of electrons. In order to extract these properties we apply methods from singular analysis [13] and solve (20) via an explicitly constructed asymptotic parametrix and corresponding Green operator, cf. [9] for further details. Let us briefly outline the basic idea and and some involved mathematical techniques from singular analysis. We do this in a rather informal manner by just mentioning the essential properties of the applied calculus and refer to the monographs [8, 20, 38] for a detailed exposition. In a nutshell, a parametrix P of an elliptic differential operator A is a pseudo-differential operator which can be considered as a generalized inverse, i.e., when applied from the left or right side it yields P A = I + Gl

AP = I + Gr ,

where the remainders Gl and Gr are left and right Green operators, respectively. In contrast to the standard calculus of pseudo-differential operators on smooth manifolds, our calculus applies to singular spaces with conical, edge and corner type singularities as well. In the smooth case, remainders correspond to compact operators with smooth kernel function, whereas in the singular calculus Green

Goldstone Diagrams and Sparse Grids

39

operators encode important asymptotic information which we want to extract. Acting on (20) with the parametrix from the left yields τn+1 = −P V (2) −PfV (2) τn +PfV (1) τn −PfV (2) ◦τn −Pfτn ◦V (2) ◦τn −Gl τn+1 .

(21)

The parametrix P maps in a controlled manner between functions with certain asymptotic behaviour which means that we can derive from the asymptotic properties of the terms on the right hand side of (20) its effect on the asymptotic behaviour of τn+1 . Furthermore it is an essential property of Gl that the operator maps onto a space of specific asymptotic type. Therefore the asymptotic type of Gl τn+1 is fixed and does not depend on τn+1 . Thus we have full control on the asymptotic properties of the right hand side of (21) and consequently on the asymptotic type of the iterated pair-amplitude τn+1 . At this point of our discussion, it is convenient to introduce a diagrammatic notation indispensable in quantum many-particle theory. The following considerations are based on Goldstone diagrams, cf. [28, 31] for a comprehensive discussion from a physical point of view. For the mathematically inclined reader let us briefly outline the basic idea in the following paragraph. Our discussion is based on the simplified nonlinear equation (18) and is solely of algebraic character. Certain minor ingredients of CC models are missing, e.g., sums over occupied orbitals, which are however integral parts of the diagrammatic representation. Only at the end of the section, we will briefly sketch the graphical elements which are used for actual diagrams. The iteration scheme discussed in the previous paragraphs can be further decomposed by taking into account the linearity of the differential operator A. Instead of solving the first iterated equation (19) as a whole, let us consider the decomposition Aτ1,1 = −fV (2) τ0 ,

Aτ1,2 = fV (1) τ0 ,

Aτ1,3 = −fV (2) ◦τ0 ,

Aτ1,4 = −fτ0 ◦V (2) ◦τ0 ,

from which one recovers the first iterated solution via the sum τ1 = τ0 + τ1,1 + τ1,2 + τ1,3 + τ1,4 ,

(22)

where each term actually corresponds to an individual Goldstone diagram. In the next iteration step, one can further use the decomposition (22) to construct the interaction terms on the right hand side. For each new term on the right hand side obtained in such a manner one can again solve the corresponding equation which leads to the decomposition of the second iterated solution into Goldstone diagrams τ2 = τ0 + τ1,1 + τ1,2 + τ1,3 + τ1,4 + τ2,1 + τ2,2 + · · · . This process can be continued through any number of successive iteration steps. Therefore from a diagrammatic point of view iteration schemes correspond to the

40

H.-J. Flad and G. Flad-Harutyunyan

summation of an infinite series of Goldstone diagrams which represents a pairamplitude. Therefore, if we restrict ourselves in the following to study intermediate solutions τn , n ∈ N, within the iteration scheme, we actually consider asymptotic properties of certain finite sums of Goldstone diagrams. In particular it is possible to consider specific diagrams or appropriate subtotals. A possible choice for such a subtotal of Goldstone diagrams is e.g. the finite sum of diagrams which represents the progression of an iteration process, i.e., Pn := τn − τn−1

with n > 0.

(23)

The regularity and multi-scale features of these subtotals are especially interesting with respect to the numerical analysis of CC theory. It is e.g. possible to study their approximation properties with respect to systematic basis sets in appropriate function spaces. After our informal algebraic discussion of the iteration scheme and its decomposition into contributions corresponding to Goldstone diagrams, let us finally outline the basis diagrammatic elements. Actual Goldstone diagrams are depicted in Figs. 1 and 2. In particular Fig. 1 represents the SUB-2 model outlined in Sect. 1.1 before. In these diagrams lines going up, so called particle lines, correspond to a projection onto the orthogonal complement of the subspace spanned by occupied orbitals, which can be represented by ordinary integrals due to the orthogonality constrained (7), satisfied by pair-amplitudes. On the contrary, lines going down, so called hole lines, correspond to a projection onto the subspace spanned by occupied orbitals. Due to the finite number of occupied orbitals, hole lines can be represented by finite rank operators, cf. (6). The Coulomb interaction is represented by horizontal

τ a)

τ

τ

b)

c)

τ e)

τ

τ f)

d)

τ

τ g)

Fig. 1 Goldstone diagrams of the SUB-2 terms taken into account by our model. (a) Coulomb interaction (1). (b)–(d) Particle, hole and particle-hole ladder diagrams which contribute to (2) and (3). (e) Linear RPA diagram which contributes to (4). (f), (g) Nonlinear RPA diagrams, corresponding to (5), direct and exchange contributions, respectively

Goldstone Diagrams and Sparse Grids

...

41

...

...

...

Fig. 2 Stripping of ladder contributions from RPA Goldstone diagrams. Upper part: cutting-out of particle, hole and particle-hole ladder insertions. Lower part: example of a RPA diagram with multiple ladder insertions. The symbol of the resulting forth-order RPA diagram belongs to the symbol class Scl−16 as well as the symbol corresponding to the original diagram

wavy lines. Finally, the bottom lines in diagrams correspond to partial solutions, see e.g. (22), obtained in the previous iteration steps.

2 Asymptotic Properties of RPA Diagrams This section contains a summary of our results, presented in Ref. [10], concerning the asymptotic properties of iterated pair-amplitudes and certain classes of RPA diagrams. The iterated pair-amplitudes can be decomposed into a finite number of Goldstone diagrams each of them has a characteristic asymptotic behaviour near coalescence points of electrons. In the following, let us denote by τRPA an arbitrary Goldstone diagram which contributes to an iterated pair-amplitude τ (n) . It is convenient to study iterated pair-amplitudes and Goldstone diagrams with respect to the alternative Cartesian coordinates x := x1 ,

z := x1 − x2 ,

(24)

which become our standard Cartesian coordinates in the remaining part of the paper. By abuse of notation, we refer to iterated pair-amplitudes τ (n) and Goldstone diagrams τRPA either with respect to (x1 , x2 ) or (x, z) variables, i.e., τRPA (x1 , x2 ) ≡ τRPA (x, z). As already mentioned before, our focus is on asymptotic expansions of iterated pair-amplitudes and Goldstone diagrams near coalescence points of electrons. i.e., z → 0. This can be achieved by identifying these quantities with kernel functions of classical pseudo-differential operators. These operators provide an algebra which enables an efficient treatment of interaction terms, in particular the

42

H.-J. Flad and G. Flad-Harutyunyan

nonlinear ones as has been demonstrated in Ref. [10]. Let the corresponding symbol of a Goldstone diagram τRPA be given by  σRPA (x, η) :=

e−izη τRPA (x, z) dz.

The symbol belongs to the standard Hörmander class S p (R3 × R3 ) if it belongs to C ∞ (R3 × R3 ) and satisfies the estimate $ $  p−|β| $ α β $ for all x, η ∈ R3 . $∂x ∂η σRPA (x, η)$  1 + |η| Here and in the following a  b means that a ≤ Cb for some constant C which is independent of variables or parameters on which a, b may depend on. Furthermore, p it belongs to the class Scl (R3 × R3 ), p ∈ Z, of classical symbols if a decomposition σRPA (x, η) =

N−1 

σp−j (x, η) + σp−N (x, η)

(25)

j =0

into symbols σp−j ∈ S p−j (R3 × R3 ) and remainder σp−N ∈ S p−N (R3 × R3 ) for any N ∈ N exits, such that for λ ≥ 1 and η greater some constant, we have σp−j (x, λη) = λp−j σp−j (x, η). The asymptotic expansion of a classical Goldstone symbol in Fourier space is related to a corresponding asymptotic expansion of its kernel function. In the following theorem we establish the connection between Goldstone diagrams and classical pseudo-differential operators and give a simple p rule to determine the symbol class Scl (R3 × R3 ) to which they belong. Furthermore, we derived the asymptotic expansion of a Goldstone diagram near coalescence points of electrons. For a proof of the theorem, we refer to Ref. [10]. Let us distinguish in the following between smooth and singular contributions to an asymptotic expansion. Here smooth refers to asymptotic terms which belong to C ∞ (R3 × R3 ). Such terms do not cause any difficulties for approximation schemes applied in numerical simulations. Actually, the asymptotic analysis discussed below, does not provide much information concerning smooth terms, instead it focuses on singular contributions which determine the computational complexity of numerical methods for solving CC equations. Theorem 1 Goldstone diagrams of RPA-CC pair-amplitudes can be considered as kernel functions of classical pseudo-differential operators without logarithmic terms in their asymptotic expansion. Classical symbols (25) corresponding to Goldstone p diagrams belong to the symbol classes Scl with p ≤ −4. The asymptotic expansion p of a Goldstone diagram τRPA with symbol σRPA ∈ Scl , expressed in spherical coordinates (z, θ, φ), is given by τRPA (x, z) ∼

 0≤j

τp−j (x, z, θ, φ) modulo C ∞ (R3 × R3 ),

(26)

Goldstone Diagrams and Sparse Grids

43

with τp−j (x, z, θ, φ) = zj −p−3

j −p−3 

l 

gj,lm (x) Ylm (θ, φ),

l=0 m=−l j −p−l even

where functions gj,lm belong to C ∞ (R3 ). In the following, we refer to (26) as the singular part of the asymptotic expansion of a Goldstone diagram. The symbol class of a diagram τRPA can be determined in the following manner (i) Remove all ladder insertions in the diagram. (ii) Count the number of remaining interaction lines n. The corresponding symbol of the diagram τRPA belongs to the symbol class Scl−4n , cf. Fig. 2. An appropriate measure of the singular behaviour of Goldstone diagrams is the so-called asymptotic smoothness property discussed in the following corollary, to which we refer in Sect. 3 where we discuss approximation properties of these diagrams. Corollary 1 A Goldstone diagram τRPA , with corresponding symbol in the symbol p class Scl , belongs to C ∞ (R3 × R3 \ {0}) and satisfies the asymptotic smoothness property $ $ $ β α $ $∂x ∂z τRPA $  |z|−3−p−|α|−N

for − 3 − p − |α| − N < 0, and any N ∈ N0 , (27)

where for |α| ≤ −3 − p it has bounded partial derivatives. According to Theorem 1, the ladder diagrams (b), (c) and (d) in Fig. 1 do not alter symbol classes within the standard iteration scheme outlined in Sect. 1.2. In the standard RPA models usually considered in the literature, cf. [39], these diagrams are neglected altogether. What remains are the RPA diagrams (e), (f) and (g) in Fig. 1, which represent the driving terms of the iteration scheme. Corollary 2 The symbols of Goldstone diagrams representing the progression Pn , cf. (23), of the n’th iteration step of standard RPA models, i.e. no ladder insertions, can be classified according to the descending filtration of symbol classes Scl−4(n+1) ⊃ Scl−4(n+2) ⊃ Scl−4(n+3) ⊃ · · · ⊃ Scl−4(2

n+1 −1)

.

44

H.-J. Flad and G. Flad-Harutyunyan

3 Adaptive Sparse Grids Approximation of RPA Diagrams In Sect. 2 we have shown that RPA diagrams can be considered within the algebra of classical pseudo-differential operators. The correspondence between classical symbols and kernel functions provides the asymptotic behaviour of RPA diagrams near coalescence points of electrons and enables us to study adaptive approximation schemes like best N-term approximation in hierarchical wavelet bases. Previous results, presented in Ref. [11], on the best N-term approximation of two-particle correlation functions of Jastrow factors can be literally transfered to pair-amplitudes. What remains is a corresponding discussion for general RPA diagrams related to their symbol class. This is of potential interest with respect to the numerical simulation of RPA models. Let us just mention Corollary 2, where it has been shown how the symbol classes of RPA diagrams in iterative remainders vary with respect to the number of iteration steps. Best N-term approximation with respect to a hierarchical tensor product basis can be considered as an adaptive variant of sparse grids, cf. [4]. It can be easily generalized to higher order terms in CC theory, i.e. to three, four, etc. particle amplitudes. In this sense our approach can be also considered as an adaptive variant of related work [19, 42] on sparse grids approximation of solutions of the manyparticle Schrödinger equation.

3.1 Best N -Term Approximation in a Nutshell The concept of best N-term approximation belongs to the realm of nonlinear approximation theory. For a detailed exposition of this subject we refer to Ref. [7]. Loosely speaking, we consider for a given basis {ζi : i ∈ Λ} the best possible approximation of a function f in the nonlinear subset ΣN which consists of all possible linear combinations of at most N basis functions, i.e., % ΣN :=



& ci ζi : Δ ⊂ Λ, #Δ ≤ N, ci ∈ R .

(28)

i∈Δ

Here, the approximation error σN (f ) :=

inf f − fN H

fN ∈ΣN

(29)

Goldstone Diagrams and Sparse Grids

45

is given with respect to the norm of an appropriate Hilbert space H . Best N-term approximation spaces Aαq (H ) for a Hilbert space H can be defined according to Aαq (H ) := {f ∈ H : |f |Aαq (H ) < ∞} with |f |Aαq (H )

(30)

 q 1 q α −1 := . N σN (f ) N N∈N

It follows from the definition that a convergence rate σN (f ) ∼ N −α with respect to the number of basis functions can be achieved. In our application, we consider anisotropic tensor product wavelets of the form (s ,s )

(s )

(s )

1 2 1 2 χj1 ,j (x1 , x2 ) = γj1 ,a (x1 ) γj2 ,a (x2 ). 2 ,a1 ,a2 1 2

(31)

These so called hyperbolic wavelets [6] do not loose their efficiency in higher dimensions. Each multivariate wavelet corresponds to an isotropic tensor product (1) (x) := 2j/2 ψ(2j x − a) and scaling functions of orthogonal univariate wavelets ψj,a (0) (x) := 2j/2 ϕ(2j x − a) on the same level of refinement j , i.e., ψj,a (s)

(s )

(s )

(s )

γj,a (x) = ψj,a11 (x1 ) ψj,a22 (x2 )ψj,a33 (x3 ) with s := (s1 , s2 , s3 ), a := (a1 , a2 , a3 ), (32) (0)

Pure scaling function tensor products γj0 ,a are included on the coarsest level j0 only. For further details concerning wavelets, we refer to the monographs [5, 30].

3.2 Besov Regularity of RPA Diagrams Following Nitsche [32], we consider tensor product Besov spaces

'

  B˜ qα (Ω × Ω) = Bqα+1 (Lq (Ω)) Bqα (Lq (Ω)) Bqα (Lq (Ω)) Bqα+1 (Lq (Ω))

for bounded domains Ω ⊂ R3 . These spaces are norm equivalent to weighted q norms for anisotropic wavelet coefficients q

f 

B˜ qα



=

 2

max{j1 ,j2 }q

j1 ,j2 ≥j0

with

1 ,s2 ) χj(s1 ,j ,f 2 ,a1 ,a2

 $q   $$ (s ,s ) $ 1 2 $χj1 ,j2 ,a1 ,a2 , f $ , if α =

3 q

−

3 2

s1 ,s2 a1 ,a2

(33)

 :=

R3 ×R3

1) 2) γj(s1 ,a (x) f (x, y) γj(s2 ,a (y) dxdy. 1 2

46

H.-J. Flad and G. Flad-Harutyunyan

The norm equivalence requires a univariate wavelet ψ with p > α + 1 vanishing β moments and ψ, ϕ ∈ Bq (Lq ) for some β > α + 1. The corresponding relation between best N-term approximation spaces and Besov spaces is given by α/3 Aq (H 1 (Ω × Ω)) = B˜ qα (Ω × Ω), if α =

3 q

− 32 .

(34)

The following lemma provides the Besov regularity of RPA diagrams depending on their symbol class, and according to (34), anticipated convergence rates of adaptive approximation schemes. It should be mentioned that the following bounds concerning Besov regularities are sharp and cannot be improved. This follows from a simple argument, cf. Ref. [11, Corollary 2.4], which can be easily adapted to the present case. Lemma 1 Let τRPA represent a RPA Goldstone diagram with corresponding symbol p 3 in Scl (R3 × R3 ), p ≤ −4. Then τRPA belongs to B˜ qα (Ω × Ω) for q > − 1+p and α=

3 q

− 32 .

The following proof is closely related to the proof of Lemma 2.1 in Ref. [11], (s) For each isotropic 3d-wavelet γj,a , we define a cube j,a centred at 2−j a with edge length 2−j L, such that supp γj,a ⊂ j,a . In order to estimate the norm (33) for a RPA diagram, we restrict ourselves to wavelet coefficients with j1 ≥ j2 and |s1 | = |s2 | = 1. This combination of 3d-wavelet types corresponds to the worst case where vanishing moments can act in one direction only. We first consider the case dist(j1 ,a1 , j2 ,a2 ) ≤ 2−j2 L. In order to apply the asymptotic smoothness property, c.f. Corollary 1, (s)

$ $ $ $ β α $∂x ∂z τRPA $  1 $ $ $ $ β α $∂x ∂z τRPA $  |z|−3−p−|α|

for |α| ≤ −3 − p,

(35)

for |α| > −3 − p,

(36)

let us decompose the cube j2 ,a2 into non overlapping subcubes i (i ∈ Δ) with edge length 2−j1 L. The subcubes i with i ∈ Δ0 := {i ∈ Δ : dist(j1 ,a1 , i ) ≤ 2−j1 L} are considered separately. Their number is #Δ0 = O(1) independent of the wavelet levels j1 , j2 . For the remaining subcubes i (i ∈ Δ \ Δ0 ) it becomes necessary to control their contributions with respect to dist(j1 ,a1 , i ) because #(Δ \ Δ0 ) = O(23(j1−j2 ) ) depends on the wavelet levels. The wavelet coefficients can be estimated by the separate sums $ $ $ (s1 ,s2 ) $ $χj1 ,j2 ,a1 ,a2 , τRPA $ $ $ $  $$ $ (s1 ) (s2 ) ≤ γj1 ,a1 (x) τRPA (x, x − y) γj2 ,a2 (y)dxdy$ $ $ $ j ,a ×i i∈Δ0

1 1

$  $$ + $ $ j i∈Δ\Δ0

1 ,a1

$ $

×i

$ (s1 ) (s2 ) γj1 ,a (x) τRPA (x, x − y) γj2 ,a (y)dxdy$ . 1 2 $

(37)

Goldstone Diagrams and Sparse Grids

47

For the first sum we can use the following proposition Proposition 1 The RPA diagram τRPA satisfies the estimate $ $ $ $

$

R3

$ (s) τRPA (x, x − y)γj,a (y)dy$$

 2−j (m−3) τRPA (x, ·)H m (R3 ) ,

for dist(x, j,a ) ≤ 2−j L and any integer m such that m < min{− 12 − p, q}. It is an immediate consequence of the symbol estimate, see e.g. [40], $  $ p−|α| $ $ β α , $∂x ∂η σRPA (x, η)$  1 + |η| that τRPA (x, ·) belongs to the Sobolev space H s (R3 ) for s < − 12 − p. Let us decompose the pair-amplitude into a polynomial and a singular remainder τRPA (x, z) = Qp (x, z) + Rpm (x, z), with Qp a polynomial of degree less than m in z := x − y with coefficients which are smooth functions in x. For the remainder we can achieve the following estimate, cf. Prop. 4.3.2 [3], 3

Rpm (x, x − ·)L∞ (j,a )  2−(m− 2 )j Rpm (x, x − ·)H m (B) , where B is a ball centered at x with radius 2−j +3 L. Taking into account the normalization of the wavelet, we obtain the desired estimate. Suppose s1,1 = 1 (s1 ) , cf. (32), we obtain from Proposition 1 the estimate for the wavelet γj1 ,a 1 $  $$ $ $ j

i∈Δ0

1 ,a1

×i

(s1 ) γj1 ,a (y) τRPA (x, x − 1

$ $

$ (s2 ) y) γj2 ,a (x)dxdy$ 2 $

3

 2−(m+3)j1 2 2 j2 τRPA (x, ·)H m (R3 ) % 3 2−(q+3)j1 2 2 j2 , if q < − 12 − p  . 3 2(p−2)j1 2 2 j2 , if q > − 12 − p

(38)

The second sum can be estimated using the next proposition (see, e.g., Ref. [11] for details). Proposition 2 Suppose the function f (x) with x ∈ R3 is smooth on the support of (s) . Then the following estimate holds an isotropic 3d-wavelet γj,a $ $ $ $

R3

$ $ s p s p s p (s) f (x) γj,a (x)dx$$  2−(p|s|+3/2)j ∂x11 ∂x22 ∂x33 f L

(s) ∞ (supp γj,a )

with |s| := s1 +s2 +s3 .

48

H.-J. Flad and G. Flad-Harutyunyan (s )

1 With this and the estimates (35) and (36) for wavelets γj1 ,a with q vanishing 1 moments (i.e., |s1 | = 1), we obtain $ $ $  $$ $ (s1 ) (s2 ) γj1 ,a1 (y) τRPA (x, x − y) γj2 ,a2 (x)dxdy$ $ $ $ j ,a ×i

i∈Δ\Δ0

1 1





3

3

2−(q+ 2 )j1 2−3j1 2 2 j2 ∂y1 τRPA L∞ (j q

1 ,a1

×i )

i∈Δ\Δ0

2

−(q+ 23 )j1

⎧ ⎪ ⎨

2

3 2 j2



2−j2 +2 L 2−j1 L

3

r 2−k dr

3

if k ≤ 2 2−(q+ 2 )j1 2−( 2 −k)j2 , 3 3  (j1 − j2 + 1)2−( 2 −p)j1 2 2 j2 , if k = 3 , ⎪ 3 3 ⎩ 2−( 2 −p)j1 2 2 j2 , if k > 3

(39)

where the parameter k = 0 if q ≤ −3 − p and k = 3 + p + q otherwise. Once we have obtained the estimates (38) and (39), it is straightforward to get an upper bound for the contribution of anisotropic tensor products with translation parameters (a1 , a2 ) ∈ Aj1 ,j2 := {(a1 , a2 ) : dist(j1 ,a1 , j2 ,a2 ) ≤ 2−j2 L} to the norm (33)  j1 ≥j2 ≥j0

˜ 1 2qj

 (a1 ,a2 )∈Aj1 ,j2

$q˜ $ $ $ (s1 ,s2 ) $χj1 ,j2 ,a1 ,a2 , τRPA $ ⎧ ⎪ ⎨

q˜

˜ 2 −3)j1 −( 2 −k)qj2 2−(q q+ 2 , if k ≤ 2 3 −( 21 q−p ˜ q−3)j ˜ ˜ 2 1 2 2 qj  (j − j + 1)2 , if k = 3 2 ⎪ 1 3 −( 21 q−p ˜ q−3)j ˜ qj ˜ j1 ≥j2 ≥j0 ⎩ 1 2 2 22 , if k > 3 % q˜ > 3 1 , for k ≤ 1 q+ 2 < ∞, if , (40) 3 , for k ≥ 2 q˜ > − 1+p



3

where we have used #Aj1 ,j2 = O(23j1 ). In order to get an upper bound for the norm (33) it remains to estimate the contributions of anisotropic wavelet coefficients where the supports of the corresponding 3d-wavelets are well separated. For this we have to consider the parameter set Bj1 ,j2 := {(a1 , a2 ) : 2−j2 L < dist(j1 ,a1 , j2 ,a2 )}. Let us assume q > −p − 3, i.e. k = 3 + p + q, using estimates (38) and (39) and Proposition 2,

Goldstone Diagrams and Sparse Grids

49

the contributions can be estimated by $ $q˜ $ (s1 ,s2 ) $ $χj1 ,j2 ,a1 ,a2 , τRPA $

 (a1 ,a2 )∈Bj1 ,j2





3

p

p

1 ,a1

(a1 ,a2 )∈Bj1 ,j2

2

q˜

˜ 1 +j2 ) 2−(q+ 2 )q(j ∂x1 ∂y1 τRPA L∞ (j

−(q q+ ˜ 32 q−3)j ˜ ˜ 32 q−3)j ˜ 1 −(q q+ 2



2

diam Ω 2−j2 L

×j2 ,a2 )

˜ r 2−q(3+p+2q) dr

˜ 2 q−3)j ˜ ˜ 2 1 (q+p+ 2 )qj  2−(q q+ 2 , 3

3

where we have used p > α + 1 and α =

3 q

− 32 , hence

  5 2 − 2q q˜ − 3q˜ − pq˜ < −1 − q˜ q + p + 2 < −1 since p + q > −3, p, q ∈ Z, follows for the exponent of the integrand. The remaining sum with respect to the wavelet levels yields  j1 ≥j2 ≥j0

˜ 1 2qj

$q˜ $ $ $ (s1 ,s2 ) $χj1 ,j2 ,a1 ,a2 , τRPA $

 (a1 ,a2 )∈Bj1 ,j2





1

3

˜ 2 q−3)j ˜ ˜ 2 1 2(q+p+ 2 )qj 2−(q q+

j1 ≥j2 ≥j0





%

j1 ≥j0

< ∞, if

1

˜ 2 q−3)j ˜ 1 , if p + q = −2 2−(q q+ (3+ qp+ ˜ q)j ˜ 1 , if p + q ≥ −1 2

%

q˜ > q˜ >

3 , q+ 12 3 − 1+p ,

for p + q = −2 for p + q ≥ −1

,

from which we obtain, together with our previous estimate (40), the lower bound on the Besov space parameter q.

4 Conclusions and Outlook The present work can be considered as a first step towards a rigorous numerical analysis of nonlinear models in electronic structure theory. Estimates for particular classes of Goldstone diagrams within the framework of best N-term approximation theory might help to improve the efficiency of numerical algorithms. In particular,

50

H.-J. Flad and G. Flad-Harutyunyan

these estimates provide guidelines for adaptive and sparse representations of multiindex quantities, cf. the SUB-2 model discussed in Sect. 1.1, by tensor product approximation schemes. In our future work, we want to extend the present analysis to Feynman diagrams. By considering all possible time orderings, Feynman diagrams can be represented by sums of Goldstone diagrams, cf. [31]. Understanding Feynman diagrams suggests a possibility to study the asymptotic behaviour of Green’s functions and derived quantities like density response functions and density matrices. These quantities provide interesting links between perturbation theory and density functional theory, see e.g. [33], the two main pillars of electronic structure theory.

References 1. R. J. Bartlett and M. Musial, Coupled-cluster theory in quantum chemistry, Rev. Mod. Phys. 79 (2007) 291–352. 2. R. F. Bishop, An overview of coupled cluster theory and its applications in physics, Theor Chim Acta 80 (1991) 95–148. 3. S. C. Brenner and L. R. Scott, The Mathematical Theory of Finite Element Methods, 2nd ed. (Springer, New York, 2002). 4. H.-J. Bungartz and M. Griebel, Sparse grids, Acta Numerica 13 (2004) 1–123. 5. I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional Conference Series in Applied Mathematics 61 (1992). 6. R.A. DeVore, S.V. Konyagin, and V.N. Temlyakov, Hyperbolic wavelet approximation, Constr. Approx. 14 (1998) 1–26. 7. R.A. DeVore, Nonlinear approximation, Acta Numerica 7 (1998) 51–150. 8. Y. V. Egorov, B.-W. Schulze, Pseudo-Differential Operators, Singularities, Applications. Birkhäuser: Basel; 1997. 9. H.-J. Flad, G. Flad-Harutyunyan, B.-W. Schulze, Explicit Green operators for quantum mechanical Hamiltonians. II. Edge singularities of the helium atom, Asian-European Journal of Mathematics 13 (2020) 2050122, (64 pages). 10. H.-J. Flad and G. Flad-Harutyunyan, Singular analysis of RPA diagrams in coupled cluster theory, Preprint: arXiv:1801.07573 [math-ph] 11. H.-J. Flad, W. Hackbusch and R. Schneider, Best N-term approximation in electronic structure calculations. II. Jastrow factors, ESAIM: M2AN 41 (2007) 261–279. 12. H.-J. Flad, G. Harutyunyan and B.-W. Schulze, Singular analysis and coupled cluster theory, PCCP 17 (2015) 31530–31541. 13. H.-J. Flad, G. Harutyunyan and B.-W. Schulze, Asymptotic parametrices of elliptic edge operators, J. Pseudo-Differ. Oper. Appl. 7 (2016) 321–363. 14. H.-J. Flad, R. Schneider and B.-W. Schulze, Regularity of solutions of Hartree-Fock equations with Coulomb potential, Math. Methods Appl. Sci. 31 (2016), 2172–2201. 15. S. Fournais, M. Hoffmann-Ostenhof, T. Hoffmann-Ostenhof, and T. Østergaard Sørensen, Sharp regularity results for Coulombic many-electron wave functions, Commun. Math. Phys. 255 (2005) 183–227. 16. S. Fournais, M. Hoffmann-Ostenhof, T. Hoffmann-Ostenhof, T. Østergaard Sørensen, Analytic structure of many-body Coulombic wave functions, Comm. Math. Phys. 289 (2009) 291–310. 17. F. Furche Developing the random phase approximation into a practical post-Kohn-Sham correlation model, J. Chem. Phys. 129 (2008) 114105 (8 pages). 18. D. Gilbarg and N. S. Trudinger, Elliptic Partial Differential Equations of Second Order (Springer, Berlin, 1998).

Goldstone Diagrams and Sparse Grids

51

19. M. Griebel and J. Hamaekers, Sparse grids for the Schrödinger equation, ESAIM: M2AN 41 (2007) 215–247. 20. G. Harutyunyan, B.-W. Schulze, Elliptic Mixed, Transmission and Singular Crack Problems. EMS Tracts in Mathematics Vol. 4 (European Math. Soc., Zürich, 2008). 21. M. Hoffmann-Ostenhof, and R. Seiler, Cusp conditions for eigenfunctions of n-electron systems, Phys. Rev. A 23 (1981) 21–23. 22. M. Hoffmann-Ostenhof and T. Hoffmann-Ostenhof, Local properties of solutions of Schrödinger equations, Commun. Partial Diff. Eq. 17 (1992) 491–522. 23. M. Hoffmann-Ostenhof, T. Hoffmann-Ostenhof, and H. Stremnitzer, Local properties of Coulombic wave functions, Commun. Math. Phys. 163 (1994) 185–215. 24. T. Kato, On the eigenfunctions of many-particle systems in quantum mechanics, Commun. Pure Appl. Math. 10 (1957) 151–177. 25. H. Kümmel, Origins of the coupled cluster method, Theor Chim Acta 80 (1991) 81–89. 26. H. Kümmel, K. H. Lührmann, and J. G. Zabolitzky, Many-fermion theory in exp S- (or coupled cluster) form, Physics Reports 36 (1978) 1–63. 27. A. Laestadius and S. Kvaal, Analysis of the extended coupled-cluster method in quantum chemistry, SIAM J. on Numerical Analysis 56 (2018) 660–683. 28. I. Lindgren and J. Morrison, Atomic Many-Body Theory (Springer, Berlin, 1986). 29. K. H. Lührmann, Equations for subsystems, Ann. Phys. 103 (1977) 253–288. 30. S. Mallat, A Wavelet Tour of Signal Processing (Academic Press, San Diego, 1998). 31. J. W. Negele and H. Orland, Quantum Many-Particle Systems (Addison-Wesley, Reading MA, 1988). 32. P.-A. Nitsche, Best N-term approximation spaces for tensor product wavelet bases, Constr. Approx. 24 (2006) 49–70. 33. G. Onida, L. Reining and A. Rubio, Electronic excitations: density-functional versus manybody Green’s function approaches, Rev. Mod. Phys. 74 (2002) 601–659. 34. J. Paldus and X. Li, A critical assessment of coupled cluster method in quantum chemistry, in Advances in Chemical Physics, Eds. I. Prigogine and S. A. Rice, 110 (1999) 1–175. 35. T. Rohwedder, The continuous coupled cluster formulation for the electronic Schrödinger equation, ESAIM: M2AN 47 (2013) 421–447. 36. T. Rohwedder and R. Schneider, Error estimates for the coupled cluster method, ESAIM: M2AN, 47 (2013) 1553–1582. 37. R. Schneider, Analysis of the projected coupled cluster method in electronic structure calculation Num. Math. 113 (2009) 433–471. 38. B.-W. Schulze, Boundary Value Problems and Singular Pseudo-Differential Operators (Wiley, New York, 1998). 39. G. E. Scuseria, T. M. Henderson and D. C. Sorensen, The ground state correlation energy of the random phase approximation from a ring coupled cluster doubles approach, J. Chem. Phys. 129 (2008) 231101 (4 pages). 40. E. M. Stein, Harmonic Analysis: Real-Variable Methods, Orthogonality, and Oscillatory Integrals, (Princeton University Press, Princeton, 1993). 41. K. Szalewicz, B. Jeziorski, H. J. Monkhorst and J. G. Zabolitzky, Atomic and molecular correlation energies with explicitly correlated Gaussian geminals. I. Second-order perturbation treatment for He, Be, H 2 , and LiH, J. Chem. Phys. 78 (1983) 1420–1430. 42. H. Yserentant, Regularity and Aproximability of Electronic Wave Functions, Lecture Notes in Mathematics 2000, (Springer, Berlin, 2010).

Generalized Sparse Grid Interpolation Based on the Fast Discrete Fourier Transform Michael Griebel and Jan Hamaekers

Abstract In (Griebel and Hamaekers, Fast discrete Fourier transform on generalized sparse grids, Sparse grids and Applications, volume 97 of Lecture Notes in Computational Science and Engineering, pages 75–108, Springer, 2014), an algorithm for trigonometric interpolation involving only so-called standard information of multivariate functions on generalized sparse grids has been suggested and a study on its application for the interpolation of functions in periodic Sobolev spaces of dominating mixed smoothness has been presented. In this complementary paper, we now give a slight modification of the proofs, which yields an extension from the t ) to the more general pairing (H s , H t,r ) and which in addition pairing (H s , Hmix mix results in an improved estimate for the interpolation error. The improved (constructive) upper bound is in particular consistent with the lower bound for sampling on regular sparse grids with r = 0 and s = 0 given in (D˜ung, Acta Mathematica Vietnamica, 43(1):83–110, 2018; D˜ung et al., Hyperbolic Cross Approximation, Advanced Courses in Mathematics - CRM Barcelona, Birkhäuser/Springer, 2018).

1 Introduction This is an addendum to our previous paper [9]. Throughout this article we will use the definitions and notation given therein. As noted in [9], so-called sparse grid based approaches [2, 12] have emerged as useful techniques to tackle higherdimensional problems, since they allow to break the curse of dimensionality under

M. Griebel Institute for Numerical Simulation, University of Bonn, Bonn, Germany Fraunhofer Institute for Algorithms and Scientific Computing SCAI, Schloss Birlinghoven, Sankt Augustin, Germany J. Hamaekers () Fraunhofer Institute for Algorithms and Scientific Computing SCAI, Schloss Birlinghoven, Sankt Augustin, Germany e-mail: [email protected] © Springer Nature Switzerland AG 2021 H.-J. Bungartz et al. (eds.), Sparse Grids and Applications - Munich 2018, Lecture Notes in Computational Science and Engineering 144, https://doi.org/10.1007/978-3-030-81362-8_3

53

54

M. Griebel and J. Hamaekers

certain conditions. For example, for the periodic Sobolev spaces , " n -  t,r n 2t 2r 2 . ˆ Hmix (T ) := f : (1 + |kd |) (1 + |k|∞ ) |fk | < ∞ !

k∈Zn d=1

of bounded mixed smoothness on the n-dimensional torus Tn which, e.g. indeed appear with t = 34 − , r = 1 for the solution of electronic Schrödinger equation [20], a specific generalization of the regular sparse grid spaces based on T Fourier polynomials eik x and associated Fourier coefficients fˆk with frequencies k from the generalized hyperbolic cross % Γ2TL

:= k ∈ Z : n

n 

& (1 + kd ) · (1 + |k|∞ ) ≤ 2

L(1−T )

d=1

were introduced in [18] and further discussed in [9, 11, 12, 14, 19]. Here, T ∈ [−∞, 1) is an additional parameter that controls the mixture of isotropic and mixed smoothness: The case T = 0 corresponds to the conventional hyperbolic cross (or regular sparse grid) discretization space where O(2L Ln−1 ) frequencies are involved. Furthermore, the case T = −∞ corresponds to the full tensor grid. The case T → 1 corresponds to a latin hypercube and the case 0 < T < 1 resembles certain energy-norm based sparse grids where the order of the amount of included frequencies does not depend on the number of dimensions n, i.e. it is O(2L ) only. Usually, these generalized sparse grid spaces are based on linear information, i.e. on the Fourier coefficients of f , which involve an explicit integration of f against the respective Fourier basis function. In contrast to that, so called standard information involves only function values, which is of interest in many practical applications. In this case, it is in general not clear if the approximation error of an associated interpolant exhibits the same order of the convergence rate as that of the best linear approximation [15, 25]. t In [9], we have shown that, if L ∈ N0 , T < 1, s < t, t > 12 and f ∈ Hmix := t,0 Hmix with a pointwise convergent Fourier series, it holds ⎧ ⎨

  n−1 L n−1 − (t −s)+(T t −s) n−T 2 L f 

for T ≥ st ,

mix

for T < st ,

f − IJ T f H s  L ⎩2−(t −s)Lf  t H

t Hmix

(1)

where IJ T denotes the general sparse grid interpolation operator, as defined via (13) L with JLT := {l : |l|1 − T |l|∞ ≤ (1 − T )L},

T < 1.

(2)

Generalized Sparse Grid Interpolation Based on the Fast Discrete Fourier Transform

55

Analogous estimates for the specific case of regular sparse grids, i.e. T = 0, can be found e.g. in [23, 24, 27]. Moreover, for T = 0, estimates with an improved n−1 logarithmic term, i.e. L 2 , are given in [3, 6, 22, 26, 28]. In this addendum to our previous article [9] we now show the following: Let t,r L ∈ N0 , T < 1, s − r < t, t + nr > 12 , t ≥ 0, r ≥ 0 and f ∈ Hmix with a pointwise convergent Fourier series. Then it holds  ⎧  n−1 L n−1 ⎨2− (t −(s−r))+(T t −(s−r)) n−T L 2 f H t,r mix f − IJ T f H s  L ⎩2−(t −(s−r))Lf  t,r

Hmix

for T ≥ for T
0, in contrast to the previous estimate (1). At the same time we prove for the first clause a logarithmic term of type L(n−1)/2 only, which improves on the Ln−1 term contained in the first clause of (1). Moreover, for the specific case T = 0, s = 0, r = 0, this upper bound is in particular consistent with the lower bound given in case of sampling on the Smolyak grid (regular sparse grid) in [4, 5, 17]. The remainder of this paper is organized as follows: In Sect. 2, we summarize necessary definitions. In Sect. 3, we present our improved error estimate. In Sect. 4, we discuss our new result and give some concluding remarks.

2 Fourier-Based Approximation for General Sparse Grids In this section we will shortly recall necessary definitions given in our previous article [9]. Let Tn be the n-torus, which is the n-dimensional cube Tn ⊂ Rn , T = [0, 2π], where opposite sides are identified. We then have n-dimensional coordinates x := (x1 , . . . , xn ), where xd ∈ T. We define the basis function associated to a multi-index k = (k1 , . . . , kn ) ∈ Zn by ωk (x) :=

 n 

 ωkd

(x) =

d=1

n 

ωkd (xd ),

d=1

Every f ∈ L2 (Tn ) has the unique expansion f (x) = Fourier coefficients are given by fˆk :=

ωk (x) := eikx .

1 (2π)n

 Tn



ωk∗ (x)f (x) dx.

k∈Zn

fˆk ωk (x), where the

(3)

56

M. Griebel and J. Hamaekers

In the following, let us now define finite-dimensional subspaces of the space L2 (Tn ) ∼ = H 0 (Tn ) and associated appropriate interpolation operators. To this end, we first consider the one-dimensional case, i.e. n = 1, and we set σ : N0 → Z : j →

% −j/2

if j is even,

(4)

(j + 1)/2 if j is odd.

For l ∈ N0 we introduce the one-dimensional nodal basis Bl := {φj }0≤j ≤2l −1

with

φj := ωσ (j )

(5)

and the corresponding spaces V l := span{Bl }. Now, let the Fourier series k∈Z fˆk ωk be pointwise convergent to f (x). Then, : m = 0, . . . , 2l − 1} of level l ∈ N0 , the for interpolation points Sl := {m 2π 2l  (l) interpolation operator can be defined by Il : V → Vl : f → Il f := fˆ φj j ∈Gl

j

with indices Gl := {0, . . . , 2l − 1} and discrete nodal Fourier coefficients 

fˆj(l) := 2−l

f (x)φj∗ (x),

(6)

x∈Sl

which only involve point evaluations f (x) at points x ∈ Sl . This way, the 2l interpolation conditions f (x) = Il f (x) for all x ∈ Sl are fulfilled. In particular, from (6) and (3), one can deduce the well-known aliasing formula (l) fˆj =



fˆk 2−l

k∈Z



ωσ∗ (j ) (x)ωk (x) =

x∈Sl



fˆσ (j )+m2l .

(7)

m∈Z

Next, we introduce an univariate Fourier hierarchical basis function for j ∈ N0 by ψj :=

% φ0

for j = 0,

φj − φ2l −1−j

for 2l−1 ≤ j ≤ 2l − 1, l ≥ 1,

(8)

and we define the one-dimensional hierarchical Fourier basis including basis functions up to level l ∈ N0 by Blh := {ψj }0≤j ≤2l −1 . Let us further introduce the difference spaces Wl :=

% span{B0h } span{Blh

h } \ Bl−1

for l = 0, for l > 0.

Note that there holds the relation Vl = span{Bl } = span{Blh } for all l ∈ N0 . / Thus, we have the decomposition of Vl into the direct sum Vl = lv=0 Wl . Now, let

Generalized Sparse Grid Interpolation Based on the Fast Discrete Fourier Transform

57

l ∈ N0 and u ∈ Vl . Then,  for any u ∈ Vl , one can easily switch from its hierarchical representation u = 0≤j ≤2l −1 uhj ψj , where uhj ∈ R, to the nodal representation  u = 0≤j ≤2l −1 uj φj by a linear transform. Next, we define the difference operator Δl := (Il − Il−1 ) : V → Wl , for l ≥ 0, where we set I−1 = 0. Note that the image of Δl is a subspace of Wl . Hence, we define the corresponding hierarchical Fourier coefficients f˘j by the unique representation   (l)  (l)  Δl f = (fˆj − fˆj(l−1) )φj + (9) fˆj φj =: f˘j ψj 0≤v 0, i = 1, . . . , d, and let Vim be the approximation to v(mk, i1 hx1 , . . . , id hxd ), m = 1, . . . , N, ij ∈ 1 ,...,id Z, where i0,j := [x0,j / hxj ], the closest integers to x0,j / hxj . We approximate v(0, x1 , . . . , xd ) by Vi01 ,...,id

=

d 

h−1 xj δ(i0,1 ,...,i0,d )

% d =

j =1

−1 j =1 hxj ,

0,

ij = i0,j ,

∀j ∈ {1, . . . , d},

otherwise.

In analogy with (5), we define a linear functional by  PT =

Rd+

v(T , x) dx.

(35)

Similar as before, we use an implicit finite difference scheme to approximate (34), and we use the trapezoidal rule for P with a truncation of the domain. We have the following conjectures and results. Conjecture 1 Assume the implicit finite difference scheme is stable,2 and the timestep k and mesh size hxi satisfy k ≤ λ min{h2x1 , h2x2 , · · · , h2xd },

(36)

for arbitrary fixed λ > 0. Then we have the error expansion ;   E |ViN − v(T , x1 , . . . , xd )|2 = O(h2x1 ) + O(h2x2 ) + . . . + O(h2xd ), 1 ,...,id where N = T /k is the number of time steps. Next we apply the sparse combination method to (35) with fixed k satisfying (36). Similar to Sect. 2.2, let Δi be the first-order difference operator along directions i = 1, . . . , d, defined as in (14), and Δ = Δ1 ⊗ · · · ⊗ Δd .

2

We expect this to be true for ‘small enough’ correlations as in the two-dimensional case for (9), but it is not obvious what the conditions will be for specific d > 2 without performing the analysis.

Combination Technique and MLMC for SPDEs

225

Conjecture 2 Assume the implicit finite difference scheme is stable. Let hxi = h0 · 2−li , i = 1, · · · , d, and k be the timestep such that for an arbitrary fixed λ > 0, k ≤ λ min{h2x1 , h2x2 , · · · , h2xd }, where N = T /k the number of time steps. Then the first and second moments of ΔPlN satisfy $  $ $ $ $E ΔPlN $ = O(h2x1 · · · h2xd ),

$2  $ $ $ E $ΔPlN $ = O(h4x1 · · · h4xd ).

(37)

Given a sequence of index sets Il = {l ∈ Nd0 : l1 + · · · + ld ≤ l + 1}, the approximation on level l is defined (similar to (17)) as PlN =

 l∈Il

ΔPlN .

(38)

Note that we use the same k for all ΔPlN , (l1 , · · · , ld ) ∈ Il . Then we have: Proposition 2 Assume Conjectures 1 and 2 to be true. Then, for P given by (35) and PlN by (38), we have ;   E |PlN − P |2 = O(l d−1 2−2l ).   Moreover, by choosing l = 12 (− log2 ε + (d − 1) log2 | log ε|) , the computational cost W to achieve a RMSE ε using standard Monte Carlo estimation is   7 5 W = O ε− 2 | log ε| 2 (d−1) .

(39)

Next, we combine MLMC with the sparse combination scheme. Similar to Sect. 2.3, let Pl := PlNl be an approximation to P as in (17) using a discretisation with timestep kl and index set Il defined above. Then we define δPl = Pl − Pl−1 ,

l ≥ 0,

(40)

where we denote P−1 := 0. Thus the approximation to P at level l ∗ has the form ∗

E[Pl ∗ ] =

l  l=0

= > l∗   N N E[δPl ] = E ΔPl l − ΔPl l−1 . l=0

l∈Il

l∈Il−1

In this way, we simulate δPl , l = 0, 1, . . . , l ∗ instead of directly simulating Pl ∗ , such that the variance of δPl = Pl −Pl−1 is considerably reduced by using the same Brownian path for Pl and Pl−1 .

226

C. Reisinger and Z. Wang

  8l be an estimator for E δPl using Ml samples. Each estimator is an average Let Y of Ml independent samples, Ml  8l = 1 δ P8l(m) , Y Ml

l = 0, . . . , l ∗ ,

m=1

8(m) is the m-th sample arising from a single SPDE approximation. The where δ P l MLMC estimator is defined as ∗

P8l ∗ :=

l  l=0

Ml =    (m) >    N N (m) 8l = 1 ΔP8l l ΔP8l l−1 − , Y Ml m=1

l∈Il

(41)

l∈Il−1

(m)  is the m-th sample for the difference on spatial grid level where ΔP8lNl l = (l1 , . . . , ld ) of the SPDE approximation using Nl time steps. Following [7], through optimising Ml to minimise the computational cost for a fixed variance, we can achieve the optimal complexity O(ε−2 ) in this case. Proposition 3 Assume Conjectures 1 and 2 to be true. Given δPl from (40), there exist C1 , C2 , C3 > 0, such that E[δPl ] ≤ C1 · l d−1 2−2l , Var[δPl ] ≤ C2 · l 2d−2 2−4l , Cost[δPl ] ≤ C3 · l d−1 23l . Then, given a RMSE ε, the MLMC estimator (41) leads to a total complexity O(ε−2 ). Proof The first inequalities follow directly from (37). The complexity then is a small modification of [7, Theorem 3.1].   If we use MLMC without sparse combination to estimate E[Pt ], by letting hxj = h0 · 2−l , k = k0 · 2−2l , then for constants C1 , C2 , C3 > 0, independent of h and k, El ≤ C1 · 2−2l ,

Vl ≤ C2 · 2−4l ,

Wl ≤ C3 · 2(d+2)l .

Similarly, we have the following result. Proposition 4 Given a RMSE ε, the total computational cost W for P in (35) using MLMC satisfies  d W = O ε −1− 2 ,

d ≥ 3.

(42)

Comparing (42) with (39), we can see that when d > 5, the sparse combination scheme with standard MC performs better than MLMC on regular grids.

Combination Technique and MLMC for SPDEs

227

6 Conclusion We considered a two-dimensional parabolic SPDE and a functional of the solution. We analysed the accuracy and complexity of a sparse combination multilevel Monte Carlo estimator, and showed that, by using a semi-implicit Milstein finite difference discretisation (7), we achieved the order of complexity O(ε−2 ) for a RMSE ε, whereas the cost using standard Monte Carlo is O(ε−4 ), and that using MLMC is O(ε−2 (log ε)2 ). When generalising to higher-dimensional problems, sparse combination with MLMC maintains the optimal complexity, whereas MLMC has an increasing total cost as the dimension increases. Further research will apply this method to a Zakai type SPDE with nonconstant coefficients. Another open question is a complete analysis of the numerical approximation of initial-boundary value problems for the considered SPDE.

References 1. 2. 3. 4.

A. Bain and D. Crisan. Fundamentals of Stochastic Filtering, volume 3. Springer, 2009. H. Bungartz. Finite elements of higher order on sparse grids. Shaker, 1998. H. Bungartz and M. Griebel. Sparse grids. Acta Numer., 13:147–269, 2004. N. Bush, B. M. Hambly, H. Haworth, L. Jin, and C. Reisinger. Stochastic evolution equations in portfolio credit modelling. SIAM J. Financ. Math., 2(1):627–664, 2011. 5. R. Carter and M. B. Giles. Sharp error estimates for discretizations of the 1d convection– diffusion equation with Dirac initial data. IMA J. Numer. Anal., 27(2):406–425, 2007. 6. T. Gerstner and M. Griebel. Dimension–adaptive tensor–product quadrature. Comput., 71(1):65–87, 2003. 7. M. B. Giles. Multilevel Monte Carlo path simulation. Operat. Res., 56(3):607–617, 2008. 8. M. B. Giles, F. Y. Kuo, and I. H. Sloan. Combining sparse grids, multilevel MC and QMC for elliptic PDEs with random coefficients. In International Conference on Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing, pages 265–281. Springer, 2016. 9. M. B. Giles and C. Reisinger. Stochastic finite differences and multilevel Monte Carlo for a class of SPDEs in finance. SIAM J. Financ. Math., 3(1):572–592, 2012. 10. E. Gobet, G. Pages, H. Pham, and J. Printems. Discretization and simulation of the Zakai equation. SIAM J. Numer. Anal., 44(6):2505–2538, 2006. 11. M. Griebel, M. Schneider, and C. Zenger. A combination technique for the solution of sparse grid problems. In P. de Groen and R. Beauwens, editors, Iterative Methods in Linear Algebra, pages 263–281. Elsevier, 1990. 12. A. L. Haji-Ali, F. Nobile, and R. Tempone. Multi-index Monte Carlo: when sparsity meets sampling. Numer. Math., 132(4):767–806, 2016. 13. C. Hendricks, M. Ehrhardt, and M. Günther. High-order ADI schemes for diffusion equations with mixed derivatives in the combination technique. Appl. Numer. Math., 101:36–52, 2016. 14. N. V. Krylov and B. L. Rozovskii. Stochastic evolution equations. J. Sov. Math., 16(4):1233– 1277, 1981. 15. D. W. Peaceman and H. H. Rachford, Jr. The numerical solution of parabolic and elliptic differential equations. J. Soc. Ind. Appl. Math., 3(1):28–41, 1955. 16. C. Pflaum and A. Zhou. Error analysis of the combination technique. Numer. Math., 84(2):327– 350, 1999. 17. C. Reisinger. Analysis of linear difference schemes in the sparse grid combination technique. IMA J. Numer. Anal., 33(2):544–581, 2012.

228

C. Reisinger and Z. Wang

18. C. Reisinger. Mean-square stability and error analysis of implicit time-stepping schemes for linear parabolic SPDEs with multiplicative Wiener noise in the first derivative. Int. J. Comput. Math., 89(18):2562–2575, 2012. 19. C. Reisinger and Z. Wang. Analysis of multi-index Monte Carlo estimators for a Zakai SPDE. J. Comput. Math., 36(2):202–236, 2018. 20. C. Reisinger and Z. Wang. Analysis of sparse grid multilevel estimators for multi-dimensional Zakai equations. arXiv preprint arXiv:1904.08334, 2019. 21. C. Reisinger and Z. Wang. Stability and error analysis of an implicit Milstein finite difference scheme for a two-dimensional Zakai SPDE. BIT Numer. Math., June 2019. First online. 22. C. Zenger. Sparse grids. In Parallel Algorithms for Partial Differential Equations: Proceedings of the 6th GAMM-Seminar, Kiel, January 1990, Notes on Numerical Fluid Mechanics 31, W. Hackbusch, ed., Vieweg, Braunschweig, pages 241–251, 1991.

Efficiently Transforming from Values of a Function on a Sparse Grid to Basis Coefficients Robert Wodraszka and Tucker Carrington Jr.

Abstract In many contexts it is necessary to determine coefficients of a basis expansion of a function f (x1 , . . . , xD ) from values of the function at points on a sparse grid. Knowing the coefficients, one has an interpolant or a surrogate. For example, such coefficients are used in uncertainty quantification. In this chapter, we present an efficient method for computing the coefficients. It uses basis functions that, like the familiar piecewise linear hierarchical functions, are zero at points in previous levels. They are linear combinations of any, e.g. global, nested basis func(k) tions ϕik (xk ). Most importantly, the transformation from function values to basis coefficients is done, exploiting the nesting, by evaluating sums sequentially. When the number of functions in level k equals k (i.e. when the level index is increased by one, only one point (function) is added) and the basis function indices    satisfy  b i − 11 ≤ b, the cost of the transformation scales as O D D+1 + 1 Nsparse , where Nsparse is the number of points on the sparse grid. We compare the cost of doing the transformation with sequential sums to the cost of other methods in the literature.

1 Introduction Sparse grids are often used to interpolate multi-dimensional functions [9, 18, 23]. An interpolant for a function f (x1 , x2 , . . . , xD ) is usually built from a set of basis functions and a set of interpolation points. In this chapter, the basis functions are (k) products of 1-D functions which are denoted ϕik (xk ), where ik = 1, 2, . . . , nk and k = 1, 2, . . . , D. D is the number of dimensions and nk is the number of basis

R. Wodraszka · T. Carrington Jr. () Chemistry Department, Queen’s University, Kingston, ON, Canada e-mail: [email protected] © Springer Nature Switzerland AG 2021 H.-J. Bungartz et al. (eds.), Sparse Grids and Applications - Munich 2018, Lecture Notes in Computational Science and Engineering 144, https://doi.org/10.1007/978-3-030-81362-8_10

229

230

R. Wodraszka and T. Carrington Jr.

functions for dimension k. All interpolants in this chapter can be written in the general form f¯(x1 , . . . , xD ) =



Ci

i

D 

(k)

ϕik (xk ),

(1)

k=1

where the sum is over a set of i = (i1 , . . . , iD ) indices. Throughout this chapter: (k) a superscript in brackets indicates a particular dimension; ϕjk is the jk -th basis function for dimension k; vectors, matrices, and tensors are in bold type, level indices are script ; the lower case letter a labels a point; and basis set functions are labelled by i, j, i  , j  . The coefficients Ci are determined so that the interpolant  (1) (2) (D) is equal to known function values at the interpolation points, ra1 , ra2 , . . . , raD . Note that ra(k) k is a point in dimension k and that ak labels a point. We shall assume throughout the chapter that the number of interpolation points is equal to the number of basis functions. A simple and general interpolation method is obtained by using special 1-D (k) basis functions, Gak (xk ), called Lagrange type functions, that are equal to zero at all points except one, a tensor product basis, and a tensor product grid of interpolation points. In this case, the coefficients Ci are simply the known values of the function at the points. The most common Lagrange type functions are the Lagrange polynomials

that span the same polynomial space as the monomial basis n−1 0 1 xk , xk , . . . , xk . However, it is possible to make Lagrange type functions that span the space spanned by any 1-D basis, see [3, 21], G(k) ak (xk )

=

nk  jk =1

[(B(k) )−1 ]jk ,ak ϕjk (xk ), ak = 1, 2, . . . , nk . (k)

(2)

nk is the number of basis functions for dimension k. A Lagrange type function is labelled by the point at which it is centred. In this chapter,   points for dimension k (k) (k)  (k) are labelled by ak or ak . In Eq. (2), (B )ak ,jk = ϕjk rak . Making these Lagrange type functions requires inverting a small nk × nk matrix. Instead of Lagrange type functions, it is also common to use a basis composed of piecewise linear functions (hat functions), called a “nodal” basis. [11] In this nodal basis, the coefficients Ci are also the known values of the function at the points. Any tensor product basis is afflicted by the curse of dimensionality. A better multi-dimensional basis can be made by using sparse grid or Smolyak type ideas and 1-D basis functions that are importance-ordered [11]. An importance-ordered 1-D basis, ϕi(k) (xk ), is one in which a basis function is more important if its value of k ik is smaller. An important basis function is one whose coefficient is large. This is the famous idea of Archimedes [1]. For the purpose of interpolating with the sparse-grid

Basis Coefficients from Function Values

231

Ansatz, it is best to use ZAPPL functions (see Sect. 2), made from an importanceordered basis. To interpolate with Smolyak’s idea one frequently uses [9]  Δ 1 ⊗ Δ 2 . . . ⊗ Δ D , (3) I (D, b) = −11 ≤b

where k labels a level. Δ k = U k − U k −1

(4)

interpolation rule; U 0 = 0 [9]. In the restriction on the and U k is a 1-D  D sum,  − 11 = k=1 |lk − 1|, where lk = 1, . . . , b + 1 ∀k = 1, . . . , D. The importance-ordered basis is divided into levels and for each level there is a corresponding set of points. In level k , for coordinate k, there are mk ( k ) points and basis functions. In this chapter, for simplicity, we shall set mk ( k ) = k , but the same ideas can be implemented when mk ( k ) ≥ k [3]. Everywhere in this chapter, we shall assume that the sequences of points are nested, i.e., the set of points with mk ( k ) points includes all the points in the set with mk ( k − 1) In this chapter, the grids included in the sum in Eq. (3) are those that satisfy the condition  − 11 ≤ b, but other choices are possible [3]. The space spanned by the pruned (restricted) basis is smaller than the space spanned by the tensor product basis. If both the function being interpolated and the basis functions are smooth, the pruning is effective. In general, formulations that obviate the sum over levels are less costly than Eq. (3) which requires a sum over levels. The interpolant made from Eq. (3) can be written in terms of Lagrange type (k) functions or in terms of the functions ϕik (xk ), from which the Lagrange type (k)

functions are made. If written in terms of ϕik (xk ), the interpolant is [I (D, b) f ](x1 , . . . , xD ) =

 i−11 ≤b

(1)

(2)

(D)

Ci1 ,i2 ,...,iD ϕi1 (x1 )ϕi2 (x2 ) . . . ϕiD (xD ) . (5)

Re-writing the sum over levels as a sum over basis indices is only this simple if mk ( k ) = k . When mk ( k ) = k , the restriction on the basis indices is not the same as the restriction on the levels [3, 6]. By equating Eq. (5) and the Lagrange type function form of the interpolant, one obtains an expression for Ci1 ,i2 ,...,iD in terms of values of the function on the sparse grid [3]. In Sect. 3, we present simpler ideas for obtaining Ci1 ,i2 ,...,iD . They work only if the sets of points are nested. It is common to use a basis of piecewise linear functions, divided into levels, that are defined so that the space spanned by the functions in the first levels is the same as the space spanned by the -th nodal basis. [11] These functions are called “hierarchical”. They are importance-ordered. In addition, they have the property that functions in level are equal to zero at points in levels 1, 2, . . . , − 1. We call this the zero-at-points-in-previous-levels (ZAPPL) property. When basis functions with the ZAPPL property are used, it is not necessary to sum over levels to determine

232

R. Wodraszka and T. Carrington Jr.

an interpolant [3, 11, 16]. In [24], the ZAPPL property is called the fundamental property. Functions that satisfy the ZAPPL property are also called incremental hierarchical functions. However, in the sparse grid literature they are very often piecewise linear [11]. Ref. [16] is an important exception to this rule. Using the ideas of section 2, it is straightforward to make ZAPPL functions from any global 1-D basis functions. The efficiency of the approach of Sect. 3 relies on using basis functions with the ZAPPL property.

2 1-D ZAPPL Basis Functions ZAPPL functions can be made from any 1-D basis set which is divided into levels and nested sets of points associated with the levels. A general recipe for making ZAPPL functions is given in Refs. [3, 24]. In those papers the ZAPPL functions (k) are In this chapter, in level 1, we have ϕ1 , in level 2 we have    called hierarchical. (k)

(k)

(k)

(k)

(k)

, etc. Correspondingly, in level 1,   (k) (k) (k) we have the point r1 , in level 2 we have the points r1 , r2 , in level 3 we have   (k) (k) (k) r1 , r2 , r3 , etc. The ZAPPL functions are: ϕ1 , ϕ2

, in level 3 we have ϕ1 , ϕ2 , ϕ3

ϕ˜i(k) (xk ) = k

ik  jk =1

(k) A˜ (k) ik ,jk ϕjk (xk ),

  (k) (k) (k) where A˜ ik ,jk is chosen so that ϕ˜ ik rak = 0 (k)

(6)

∀ ak < ik

(k) and A˜ ik ,ik = 1.

(k)

Note that ϕ˜ ik (xk ) depend on the interpolation points and ϕik (xk ) do not. The 1-D (k) points must be chosen so that B˜ (k) is not singular, or near-singular, where B˜ ak ,ik =   (k) (k) (k) ϕ˜ ik rak . In the rest of this chapter we, for simplicity, omit tildes. Bak ,ik always (k) means B˜ ak ,ik and ϕik always means ϕ˜ ik . The ZAPPL functions defined in Eq. (6) have the advantageous ZAPPL property, but they may be smooth and are not the common piecewise linear hierarchical functions [11]. To interpolate smooth functions it is often better to use smooth basis functions. Equation (6) can be used to make ZAPPL functions from any importanceordered basis. For example, a set of importance-ordered B splines could be used [24]. The prescription of Eq. (6) can be used regardless of the choice of mk ( k ), the number of functions in level k . In many cases, choosing mk ( k ) so that it does not increase exponentially with k reduces the cost of calculations. Of course, it must be possible to choose nested sets of points with mk ( k ) points in level k . In this chapter, our cost estimates are computed using mk ( k ) = k , which means that when k is increased by one, we must add a single new point. One way to do this is to use Leja points [8, 19, 22].

Basis Coefficients from Function Values

233

3 Transforming from Function Values to Basis Coefficients In this section, we present our efficient scheme for transforming a vector whose elements are values of a function at points on the sparse grid to a vector whose elements are the coefficients of a ZAPPL basis expansion of the function. Everything (k) in this section is valid for any choice of the ϕik (xk ) functions and any choice of the (nested) points. We compare our scheme to other transformation methods in the literature [13, 15, 24]. The method of this section was presented at the Sparse Grids and Applications conference in Munich. After the conference, David Holzmueller showed that the ideas can be formulated in terms of LU decompositions [17]. Note that if one wishes basis expansion coefficients in a (nested) basis that is not a ZAPPL basis, one can use the method of this section to transform from the grid to the ZAPPL coefficients and then efficiently (evaluating sums sequentially) transform from the ZAPPL coefficients to the coefficients in the desired basis, see Eq. (41) in Ref. [3].

3.1 It Appears One Needs to Invert B Let f : RD → R be a multivariate function. Its Smolyak interpolant can be written as in Eq. (5). Equation (5) is similar to the generalised polynomial chaos expansion (GPCE) employed when solving stochastic differential equations [15, 27]. Note, however, that our basis functions are not necessarily (weighted) polynomials; they can be chosen to reduce the size of the basis required for the interpolation. The goal is to obtain the expansion coefficients Ci given the values of the function f at the (1) (D) sparse grid points, i.e, f ra1 , . . . , raD . Explicit equations for the matrix-vector product required to compute Ci , Eq. (19), and for the cost, Eq. (25), are simple if the basis function indices in Eq. (5) are restricted by i − 11 ≤ b and the grid indices are restricted by a − 11 ≤ b. Both these restrictions are inherited from the level restriction in Eq. (3). The most straightforward approach for obtaining Ci from Eq. (1) is to solve a system of linear equations 

f¯(ra ) = f (ra ) = ⇐⇒ Ci =

 a−11 ≤b

Ci

i−11 ≤b



B−1



i,a

D 

(k)

ϕik

  ra(k) ∀ a − 11 ≤ b, k

(7)

k=1

f (ra )

∀ i − 11 ≤ b.

(8)

234

R. Wodraszka and T. Carrington Jr.

By solving Eq. (8), one obtains all of the coefficients Ci from one calculation. The elements of the matrix B are values of the basis functions at the sparse grid points, i.e., Ba1 ,...,aD = i1 ,...,iD

D 

(k)

ϕik

  ra(k) ∀ i − 11 ≤ b and a − 11 ≤ b. k

(9)

k=1

With the chosen restrictions of the indices, the number of sparse grid points (and product basis functions) is 

Nsparse

 D+b = . D

(10)

  3 floating point In general, to solve the linear system of equations, O Nsparse operations are required. Directly solving the linear equations in Eq. (8) or inverting B will therefore, especially for high-dimensional problems, require considerable computer time and computer memory. Smolyak interpolation thus has the advantage that Nsparse % (b + 1)D , but the disadvantage that it is not simple to determine Ci . If both the basis and the point set are tensor products, i.e., the restrictions imposed on the indices are i − 1∞ ≤ b and a − 1∞ ≤ b, then the entire set of coefficients Ci can be easily found because one can exploit the fact that B = B(kron) =

D 

B(k) ,

(11)

k=1 (k)

(k)

a Kronecker product of small (b + 1)×(b + 1) matrices Bak ,ik = ϕik

  (k) rak . Hence,

D  −1  −1  B−1 = B(kron) B(k) = .

(12)

k=1

This has two advantages. First, it is not necessary to invert a large matrix and second, it is possible to evaluate the sums in Eq. (8) sequentially, Ci =

b+1   aD =1

B(D)

−1  iD ,aD

...

b+1  −1   B(2) a2 =1

b+1   i2 ,a2 a1 =1

B(1)

−1 

f (ra ),

i1 ,a1

(13) as is frequently done in chemical physics to transform between a grid and a basis [20]. Not exploiting the Kronecker product structure of Eq. (11) and solving the

Basis Coefficients from Function Values

235

  3   operations, whereas linear system directly would require O (b + 1)3D = O Nfull the sequential summation approach of Eq. (13) requires only   O D(b + 1)D+1 = O(D(b + 1)Nfull )

(14)

operations, where Nfull = (b + 1)D . Each sum in Eq. (13) can be thought of as calculating (b + 1)D−1 MVPs for a matrix of size (b + 1). As each of these MVPs requires (b + 1)2 multiplications and there are (b + 1)D−1 of them, the total cost scales as (b + 1)D+1 . Equation (14) is the cost of computing the entire set of Ci coefficients. Although B in Eq. (8) is not a Kronecker product, we show in the next subsection that the sparse grid to basis transformation can nevertheless be done sequentially, when ZAPPL basis functions are employed. The numerical cost then scales as  = O D

>  b + 1 Nsparse . D+1

(15)

Again, this is the cost of computing the entire set of Ci coefficients. Because in both cases sums are evaluated sequentially, Eq. (15) and the right side of Eq. (14) have the same structure and there is a factor of D in both, but in Eq. (15), Nfull is replaced b with Nsparse and (b + 1) is replaced with D+1 + 1.

3.2 ZAPPL Functions and Sequential Summation Obviate the Need to Invert B We begin by sorting the products of ZAPPL functions that form the tensor product basis into two groups: the Nsparse functions that are included in the sparse basis go into a group labelled “retained”; and the Nfull − Nsparse functions that are excluded from the sparse basis go into a group labelled “discarded”. Let C ∈ RNfull ×Nsparse be a chopping matrix which is an Nfull × Nfull identity matrix from which the columns for the excluded functions have been deleted. In terms of C, B can be written as B = CT B(kron) C  −1 ⇐⇒ B−1 = CT B(kron) C .

(16) (17)

It is far from obvious that one can calculate elements of the matrix on the right side of Eq. (17) by inverting small matrices for each coordinate and then do the sums in an equation like Eq. (8) sequentially so that one obtains an equation similar to Eq. (13). Both are necessary, if one is to find an inexpensive method for computing

236

R. Wodraszka and T. Carrington Jr.

Ci on the LHS of Eq. (8). However, if excluded functions are zero at retained points then 

CT B(kron) C

−1

 −1 = CT B(kron) C.

(18)

In words, we can interchange the operations of chopping and inverting. The matrix in the matrix-vector product in Eq. (8) is the inverse of the retained block, but because the inverse of the retained block can be replaced by a block of the inverse of B(kron) (due to Eq. (18)) it is possible do sums sequentially; see Eq. (19). Equation (18) will be satisfied for any order of the tensor product basis functions if it is satisfied for one order. By block Gaussian elimination, [37] it is simple to prove that Eq. (18) is correct if the tensor product are ordered so that the  basis functions  top left block of the reordered B(kron) is CT B(kron) C and the top right block of the reordered B(kron) is zero. We have done calculations with the i − 1 ≤ b pruning condition and in this case it is easy to show that if the 1-D functions are ZAPPL functions then functions excluded from the tensor product basis are zero at retained points and thus Eq. (18) is satisfied. We are now able to evaluate the sum in Eq. (8) sequentially, and when using a simple pruning condition obtain explicit equations for the upper limits of the sums. For example, for the a − 11 ≤ b pruning condition, one has, 



Ci =

a−11 ≤b



=

 −1  (kron) C f (ra ) C B T

i,a



B(kron)

−1



(max)  

aD

Ci =

aD =1

B(D)

∀ i − 11 ≤ b

f (ra ) i,a

a−11 ≤b

−1 

(max) −1    (2) ... B

a2

iD ,aD

(max) −1    (1) B

a1

i2 ,a2 a1 =1

a2 =1

f (ra ) .

i1 ,a1

(19) If the [B(k) ]−1 matrices were not lower triangular, then the range of possible ik values would be limited by (max)

ik

=b−

k−1 

(ik  − 1) + 1 ,

(20)

k  =1

and the upper limits on the sums over ak would be (max)

ak

=b−

D  k  =k+1

(ak  − 1) + 1 .

(21)

Basis Coefficients from Function Values

237

However, because B(k) is lower triangular, [B(k) ]−1 is also lower triangular and therefore the upper limits of the sums are (max)

ak

= ik .

(22)

and (max) ik

=b−

k−1 

D 

(i − 1) − k

k  =1

(ak  − 1) + 1.

(23)

k  =k+1

Related sequential summation techniques were previously used: with a basis restricted by j − 11 ≤ b in Ref. [25]; for Smolyak quadrature in Refs. [6, 7]; for collocation in Ref. [2]; and for collocation with a hierarchical (ZAPPL) basis in Ref. [3–5, 26, 30]. Note that in Refs. [3–6, 30] a similar sequential summation idea is used with sparse grids made with either mk ( k ) = or a pruning condition more general than i − 11 ≤ b. The unidirectional principle also exploits the sequential evaluation of sums [31–33, 36]. The numerical cost associated with the prescription in Eq. (19) can be estimated by counting the number of multiplications required for each of the D sequential summations. The number of multiplications required for the sum over a1 is b+1  1 Nmult = D

aD =1

=

b 

b−(a D −1)+1

b 

...

aD−1 =1 aD 

...

aD =0 aD−1 =0

=

 D b− D k  =3 (ak  −1)+1 b− k  =2 (ak  −1)+1

aD 

aD =0 aD−1 =0





i1 

a2 =1

i1 =1

a1 =1

a2  i1 a3  

1

1

a2 =0 i1 =0 a1 =0

...

a3  a2 

(i1 + 1).

(24)

a2 =0 i1 =0

The expression in Eq. (24) can be evaluated analytically [12] yielding  Nmult = D

 b + 1 Nsparse , D+1

(25)

which is the scaling given in Eq. (15). Equation (15) is the cost of computing all the Ci coefficients. The transformation of Eq. (19) was used in Ref. [26, 30]. Its cost depends directly on the number of points on the sparse grid.

238

R. Wodraszka and T. Carrington Jr.

3.3 Comparison with Other Methods In the field of uncertainty quantification, [28, 29] it is often desirable to transform from a vector of values of a function on a sparse grid to a vector of coefficients of a so-called generalised polynomial chaos expansion (GPCE), for example, to facilitate obtaining stochastic quantities of interest. The GPCE is a special case of the sparse (k) grid interpolation of Eq. (3), where in the GPCE case the ϕik (xk ) are (weighted) polynomials. Several different methods are used to obtain the coefficients. Some methods for computing basis expansion coefficients build a B matrix whose rows and columns each have a level label and a label identifying a basis (k) function within that level [24]. Although ϕik (xk ) can be chosen to make it possible to exploit the sparsity of the corresponding B, the sums required to compute the coefficients are costly. The key advantage of our algorithm is the sequential evaluation of the sums. Algorithm 1 of Ref. [24] scales as the square of the number of functions in the basis. The scaling of Eq. (25) is much better. Other methods for computing basis expansion coefficients manipulate separately the grids that together compose the sparse grid. An algorithm of this kind is used in Refs. [13, 15]. They solve linear systems for all the tensor product grids associated with the levels  that satisfy  − 11 ≤ b. For one of the tensor product grids, they write  βp Lp (ra˜ ) , f (ra˜ ) = (26) pk ≤ k ∀k

where Lp is a basis function for the tensor product grid labelled by , and solve for βp for each tensor product grid separately. Coefficients with the same p for different grids must be combined to determine Ci . ra˜ is a point on the tensor product grid associated with ; the union of the grids whose points are denoted ra˜ is the sparse grid on which the points are denoted by ra . The total cost of this separate-grids method is the sum of the costs of matrix(sep) vector products for the grids, Nmult,2 , and the cost of inverting matrices for the grids, (sep)

Nmult,3 , (sep)

(sep)

cost = Nmult,2 + Nmult,3 ,

(27)

where (sep)

Nmult,n =



D 

nk .

(28)

−11 ≤b k=1

In this equation, we assume that the number of points for coordinate xk in level k is equal to k and that the cost of the matrix-vector product required to obtain

Basis Coefficients from Function Values

βp is (

D

k=1 k )

2.

239

If the matrix whose elements are Lp (ra˜ ) is orthogonal, and there (sep)

is therefore no need to invert it, Nmult,2 is the total cost. On the other hand, when (sep)

Lp (ra˜ ) is not orthogonal, then Nmult,3 must be included in Eq. (27). It is also possible to exploit the tensor product character of the grids. After some algebraic transformations, Eq. (28) can be cast into a form similar to Eq. (24). However, it does not seem to be possible to find an equation in closed form (sep) for Nmult,n for a general D [12]. We thus evaluate Eq. (28) numerically, to compare the cost of this separate-grids method and the approach in Sect. 3.2. We plot the numbers of operations, in Eq. (25) and Eq. (28), as a function of D for the threshold parameters b = 4, b = 9, and b = 14 in Figs. 1, 2, and 3, respectively. For Figs. 1, 2, and 3, the columns that are deleted are those for which i − 11 > b. When b = 9 and b = 14 the sequential summation method of Sect. 3.2 is orders of magnitude less costly than the separate-grids method.

b=4

10 8 N

(sep)

mult

(sep)

Nmult,2

Nmult,3

10 7

operations

10 6

10 5

10 4

10 3

10 2 5

10

15

20

25

D

Fig. 1 Scaling of floating point operations as a function of D, b = 4 (i.e. 5 points per coordinate)

240

R. Wodraszka and T. Carrington Jr. b=9

10 16 N

(sep)

(sep)

N mult,2

mult

Nmult,3

10 14

operations

10 12

10 10

10 8

10 6

10 4

10 2 5

10

15

20

25

D

Fig. 2 Scaling of floating point operations as a function of D, b = 9 (i.e. 10 points per coordinate) b = 14

10 20 N(sep)

Nmult

N(sep)

mult,2

mult,3

operations

10 15

10 10

10 5 2

4

6

8

10

12

14

16

18

D

Fig. 3 Scaling of floating point operations as a function of D, b = 14 (i.e. 15 points per coordinate)

Basis Coefficients from Function Values

241

4 Conclusion In this chapter, we explain that by doing sums sequentially it is possible to efficiently obtain expansion coefficients in a basis of products of 1-D global functions from values of a function on a sparse grid. Such coefficients are needed in many contexts. They are needed whenever one wishes to make an interpolant from a sparse grid and global 1-D functions. For example, in Sect. 2.6 of Ref. [14] the coefficients, called polynomial chaos expansion coefficients, are determined by evaluating integrals with Smolyak quadratures. In Sect. 2.7 of Ref. [14], the coefficients are obtained by solving a system of linear equations whose size is equal to the number of sparse grid points. If both the basis and the grid are tensor products, it is well established that basis coefficients can be computed from function values by doing sums sequentially. See for example, Table 10.1 of Ref. [10]. By doing sums sequentially, one obviates the need to loop simultaneously over all the indices. It is not well established that a similar sequential summation idea can be used when the grid and the basis are built from a general sparse grid recipe and nested sets of 1-D grids and global 1D bases. In Ref. [3], the original sparse grid interpolation, in terms of products of differences of 1-D interpolation operators, was formulated using sequential sums. In this chapter, we have shown that it is also possible to interpolate, i.e., to compute basis expansion coefficients, using sequential sums, without writing the interpolant in terms of products of differences of 1-D interpolants. The sequential summation approach is considerably less costly than established methods.

5 Relation with the Chapter of David Holzmueller and Dirk Pflueger We have read a preliminary version of the chapter by Holzmueller and Pflueger and wish to clarify the relationship between their contribution and ours. First, in many of our equations we use a special and simple restriction to determine which functions (points) are included in the sparse basis (grid): i − 1 ≤ b. This enables us to write out the sums in Eq. (19) explicitly and to impose the restriction by using the appropriate upper limits on the sums. Holzmueller and Pflueger do not explain in detail how they evaluate matrix vector products, but appear to use index lists and their formalism allows one to use much more general restrictions. Ideas similar to those in this paper can be used with the restriction g1 (i1 − 1) + g2 (i2 − 1) + · · · + gD (iD − 1) ≤ b, where gc (ic − 1), c = 1, · · · D, is a monotonically increasing function [3, 30]. In this case it is also possible to derive equations for the upper limits on the sums in Eq. (19). Much more general restrictions can be used if one is willing to forgo deriving equations for the upper limits and instead uses index lists [34, 35]. Second, in this paper we have one function (point) per level. Holzmueller and Pflueger have no such constraint. However, this restriction can be lifted [3]. Third, we have not given a recipe for choosing 1-D basis functions and

242

R. Wodraszka and T. Carrington Jr.

dividing the tensor product functions obtained from them into two groups, labelled “retained” and “discarded”, so that all the discarded functions are zero at the retained points. This is simple if the 1-D functions are ZAPPL functions and all the functions that satisfy g1 (i1 − 1) + g2 (i2 − 1) + · · · + gD (iD − 1) ≤ b are retained (gc (ic −1) is a monotonically increasing function). It is in general not simple if some of the functions that satisfy g1 (i1 − 1) + g2 (i2 − 1) + · · · + gD (iD − 1) ≤ b are excluded, i.e., if there are holes. An easy, and often inexpensive, way to avoid holes is to plug them by adding functions to the retained basis. When using spatially localised basis functions, it is sometimes advantageous to discard functions in regions in which the function being interpolated is smooth, i.e. to introduce holes. In the sparse grid literature this is known as spatial adaptivity. When using global basis functions, often best for representing a smooth function, spatial adaptivity is less useful. Acknowledgments Research reported in this article was funded by The Natural Sciences and Engineering Research Council of Canada. We thank David Holzmueller for sending us reference [13] and for emails about his LU perspective. We are grateful to both David Holzmueller and Dirk Pflueger for discussions.

References 1. The quadrature of the parabola. http://en.wikipedia.org/wiki/The_Quadrature_of_the_ Parabola. 2. Gustavo Avila and Tucker Carrington, Solving the Schroedinger equation using Smolyak interpolants. J. Chem. Phys. 139, 134114 (2013) 3. Gustavo Avila and Tucker Carrington, A multi-dimensional Smolyak collocation method in curvilinear coordinates for computing vibrational spectra. J. Chem. Phys. 143, 214108 (2015) 4. Gustavo Avila and Tucker Carrington, Computing vibrational energy levels of CH4 with a Smolyak collocation method. J. Chem. Phys. 147, 144102 (2017) 5. Gustavo Avila and Tucker Carrington, Reducing the cost of using collocation to compute vibrational energy levels: results for CH2 NH. J. Chem. Phys. 147, 064103 (2017) 6. Gustavo Avila and Tucker Carrington, Nonproduct quadrature grids for solving the vibrational Schroedinger equation. J. Chem. Phys. 131, 174103 (2009) 7. Gustavo Avila and Tucker Carrington, Using a pruned basis, a non-product quadrature grid, and the exact Watson normal-coordinate kinetic energy operator to solve the vibrational Schroedinger equation for C2 H4 . J. Chem. Phys. 135, 064101 (2011) 8. Gustavo Avila, Jens Oettershagen, and Tucker Carrington, Comparing nested sequences of Leja and Pseudogauss points to interpolate in 1-D and solve the Schroedinger equation in 9-D. In: Sparse Grids and Applications (Springer International Publishing, Miami, 2016), pp. 1–17 9. Volker Barthelmann, Erich Novak, and Klaus Ritter, High dimensional polynomial interpolation on sparse grids. Adv. Comput. Math. 12, 273 (2000) 10. John P. Boyd, Chebyshev & Fourier Spectral Methods, 2nd edn. (Dover Publications, New York, 2001) 11. H.-J. Bungartz and M. Griebel, Sparse grids. Acta Numerica 13, 147 (2004) 12. S. Butler and P. Karasik, A note on nested sums. J. of Int. Seq. 13, 1 (2010) 13. G. T. Buzzard, Efficient basis change for sparse-grid interpolating polynomials with application to t-cell sensitivity analysis. Comput. Bio. J. 2013, 1 (2013)

Basis Coefficients from Function Values

243

14. Michael S. Eldred, S. L. Brown, Brian M. Adams, Daniel M. Dunlavy, David M. Gay, Laura Painton Swiler, William E. Hart, J. D. Griffin, P. D. Hough, Tammy G. Kolda, and Monica Martinez-Canales, Dakota , a multilevel parallel object-oriented framework for design optimization, parameter estimation, uncertainty quantification, and sensitivity analysis version 4.0 user’s manual. (2002) https://dakota.sandia.gov/content/manuals 15. L. Formaggia, A. Guadagnini, I. Imperiali, V. Lever, G. Porta, M. Riva, A. Scotti, and L. Tamellini, Global sensitivity analysis through polynomial chaos expansion of a basin-scale geochemical compaction model. Comput. Geosci. 17, 25 (2013) 16. Klaus Hallatschek, Fouriertransformation auf dünnen Gittern mit hierarchischen Basen. Numerische Mathematik 63, 83 (1992) 17. D. Holzmueller, private communication. (2018) 18. Andreas Klimke and Barbara Wohlmuth, Algorithm 847: Spinterp: Piecewise multilinear hierarchical sparse grid interpolation in matlab. ACM Trans. Math. Softw. 31, 561 (2005) 19. F. Leja, Sur certaines suites liées aux ensembles plans et leur application à la représentation conforme. Ann. Polonici Math. 4, 8 (1957) 20. J. C. Light and Tucker Carrington, Discrete variable representations and their utilization. Adv. Chem. Phys. 114, 263 (2000) 21. G. Mastroianni and D. Occorsio, Lagrange interpolation at Laguerre zeros in some weighted uniform spaces. Acta Math. Hung. 91, 27 (2001) 22. A. Narayan and J. D. Jakeman, Adaptive Leja sparse grid construction for stochastic collocation and high-dimensional approximation. SIAM J. Sci. Comput. 36, 2952 (2014) 23. Dirk Pflüger, Spatially Adaptive Sparse Grids for High-Dimensional Problems (Verlag Dr. Hut, München, 2010) 24. Julian Valentin and Dirk Pflüger, Fundamental splines on sparse grids and their application to gradient-based optimization. In: Sparse Grids and Applications (Springer International Publishing, Miami, 2016), pp. 229–251 25. X.-G. Wang and Tucker Carrington, The utility of constraining basis function indices when using the Lanczos algorithm to calculate vibrational energy levels. J. Phys. Chem. A 105, 2575 (2001) 26. Robert Wodraszka and Tucker Carrington, A pruned collocation-based multi-configuration time-dependent Hartree approach using a Smolyak grid for solving the Schrödinger equation with a general potential energy surface. J. Chem. Phys. 150, 154108 (2019) 27. D. Xiu and G. Karniadakis, The Wiener-Askey polynomial chaos for stochastic differential equations. SIAM Journal on Scientific Computing 24, 619 (2002) 28. Ivo Babuška, Fabio Nobile, Raúl Tempone, A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data, SIAM REVIEW Vol. 52, No. 2, pp. 317–355, (2010) 29. D. Xiu and J. S. Hesthaven, High-order collocation methods for differential equations with random inputs, SIAM J. Sci. Comput., 27 1118–1139 (2005) 30. Emil J. Zak and Tucker Carrington, Using collocation and a hierarchical basis to solve the vibrational Schrödinger equation J. Chem. Phys. 150, 204108 (2019) 31. R. Balder, Adaptive Verfahren für elliptische und parabolische Differentialgleichungen auf dünnen Gittern. Dissertation, Technische Universitét Dissertation, Technische Universitét München, 1994 32. C. Feuersaenger, Sparse Grid Methods for Higher Dimensional Approximation. Dissertation, Institut für Numerische Simulation, Universität Bonn, 2010 33. D. Pflueger, Spatially adaptive sparse grids for high-dimensional problems. Dissertation, Technische Unversität München, 2010 34. R. Wodraszka and T. Carrington, Using a pruned, nondirect product basis in conjunction with the multi-configuration time-dependent Hartree (MCTDH) method J. Chem. Phys. 145, 044110 (2016)

244

R. Wodraszka and T. Carrington Jr.

35. R. Wodraszka and T. Carrington, Systematically expanding nondirect product bases within the pruned MCTDH method: A comparison with multi-layer MCTDH J. Chem. Phys. 146, 194105 (2017) 36. A. Zeiser, Fast matrix-vector multiplication in the sparse-grid Galerkin method J. Sci. Comput. 47, 328 (2011) 37. Tzon-Tzer Lu and Sheng-Hua Shiou, Computers and Mathematics with Applications 43, 119– 129 (2002)

A Sparse-Grid Probabilistic Scheme for Approximation of the Runaway Probability of Electrons in Fusion Tokamak Simulation Minglei Yang, Guannan Zhang, Diego del-Castillo-Negrete, Miroslav Stoyanov, and Matthew Beidler

Abstract Runaway electrons (RE) generated during magnetic disruptions present a major threat to the safe operation of plasma nuclear fusion reactors. A critical aspect of understanding RE dynamics is to calculate the runaway probability, i.e., the probability that an electron in the phase space will runaway on, or before, a prescribed time. Such probability can be obtained by solving the adjoint equation of the underlying Fokker-Planck equation that controls the electron dynamics. In this effort, we present a sparse-grid probabilistic scheme for computing the runaway probability. The key ingredient of our approach is to represent the solution of the adjoint equation as a conditional expectation, such that discretizing the differential operator reduces to the approximation of a set of integrals. Adaptive sparse grid interpolation is utilized to approximate the map from the phase space to the runaway probability. The main novelties of this effort are the integration of the sparse-grid method into the probabilistic numerical scheme for computing escape probability, and the application of the proposed method in computing RE probabilities. Two numerical examples are given to illustrate that the proposed method can achieve O (Δt) convergence, and that the local anisotropic adaptive refinement strategy (M. Stoyanov, Adaptive sparse grid construction in a context of local anisotropy and multiple hierarchical parents. In: Sparse Grids and Applications-Miami 2016, Springer, Berlin, 2018, pp. 175–199) can effectively handle the sharp transition layer between the runaway and non-runaway regions.

M. Yang Department of Mathematics and Statistics, Auburn University, Auburn, AL, USA G. Zhang () · M. Stoyanov Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN, USA e-mail: [email protected] D. del-Castillo-Negrete · M. Beidler Fusion Energy Division, Oak Ridge National Laboratory, Oak Ridge, TN, USA © Springer Nature Switzerland AG 2021 H.-J. Bungartz et al. (eds.), Sparse Grids and Applications - Munich 2018, Lecture Notes in Computational Science and Engineering 144, https://doi.org/10.1007/978-3-030-81362-8_11

245

246

M. Yang et al.

1 Introduction In magnetically confined fusion plasmas, runaway electrons (RE) can be generated during magnetic disruptions due to the strong electric field resulting from the rapid cooling of the plasma, see for example [2] and references therein. At high enough velocities, the drag force on an electron due to Coulomb collisions in a plasma decreases as the particle velocity increases. As a result, in the presence of a strong enough parallel electric field, fast electrons can “runaway” and be continuously accelerated, see for example the review in [12] and references therein. Understanding this phenomena has been an area of significant interest because of the potential impact that RE can have to the safe operation of the international test nuclear fusion reactor ITER.1 In particular, if not avoided or mitigated, RE can severely damage plasma facing components [5, 10, 16]. In this work, we propose a sparse-grid probabilistic scheme to study RE dynamics in phase space. Although the full RE model is defined in a six-dimensional phase space, here we focus on the RE dynamics in a three-dimensional space with coordinates (p, ξ, r), where p denotes the magnitude of the relativistic momentum, ξ the cosine of the pitch angle θ , i.e. the angle between the electron’s velocity and the local magnetic field, and r the minor radius. In this case, the dynamics of the distribution function of electrons is determined by the Fokker-Planck (FP) equation describing the competition between the electric field acceleration, Coulomb collisions, synchrotron radiation damping, and sources describing the second generation of RE due to head-on collisions [25]. In the study of RE, a set of important questions involve statistical observables different from the electron distribution function. Examples of particular interest to the present paper are the runaway probability, PRE (t, p, ξ, r), that an electron with phase space coordinates (p, ξ, r) will runaway on, or before, a time t > 0. Mathematically, PRE (t, p, ξ, r) is the solution of the adjoint of the FP equation, which is a backward parabolic equation in a non-divergence form. It is known that the non-divergence structure of the adjoint equation prevents the use of integration by parts on it to define weak solutions, which is a pre-requisite for formulating finite element methods for this problem. The second challenge is that the coefficients of the adjoint equation are usually very complicated, such that it is hard to convert the non-divergence operator to a divergence operator. Thus, the most widely used approach to approximate PRE (t, p, ξ, r) is “brute-force” Monte Carlo, which is robust and parallelizable, but features very slow convergence. The method we are proposing is different from those based on the solution of the Fokker-Planck equation, e.g., [14, 15], and also different from the direct MonteCarlo simulations. Instead, our approach is based on the Feynman-Kac formula, which establishes a link between the adjoint of the FP equation and the system of

1

ITER (originally the International Thermonuclear Experimental Reactor) is an international nuclear fusion research and engineering mega project, which will be the world’s largest magnetic confinement plasma physics experiment. See https://www.iter.org/ for details.

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

247

stochastic differential equations (SDEs). Specifically, we first represent the solution of the adjoint equation as a conditional expectation with respect to the underlying SDEs that describe the dynamics of the electrons. As such, the task of discretizing the differential operator becomes approximating the conditional expectation, which includes a quadrature rule for numerical integration and an interpolation strategy for evaluating the integrand at quadrature points. In this work, we use local hierarchical sparse grid methods[4, 7, 8, 11, 21] to handle the interpolation for two reasons. First, the terminal condition of the adjoint equation is discontinuous, and the adaptive refinement strategy of sparse grids can effectively capture such irregularity as well as well control the growth the total number of grid points. Second, the threedimensional RE model is a simplification of the full six-dimensional model, and the use of sparse grids can make it easy to extend to the full RE model in the future work. In the literature, sparse grid methods have been applied to various plasma physics problems to approximate physical quantities in the high-dimensional phase space. For instance, sparse grids were combined with PDE solvers, e.g., discontinuous Galerkin approaches, to solve gyrokinetic equations, e.g., Vlasov-Maxwell equations [17, 31], and The Vlasov-Poisson equations [13]. Not surprisingly, sparse grids were also integrated into particle-in-cell schemes [24] to dramatically increase the size of spatial cells and reduce the statistical noise without increasing the number of particles. In addition, scalable and resilient sparse grid techniques were applied to large-scale gyrokinetic problems [1, 9, 17] to significantly accelerate existing gyro-kinetics simulators, e.g., Gyrokinetic Electromagnetic Numerical Experiment (GENE).2 In [6], Leja sequence based sparse interpolation has been used to analyzing gyrokinetic micro-instabilities. This effort brings another important application of the sparse grid methods to the plasma physics community. Compared to existing works in the literature, the main contribution of this effort lies in two aspects, i.e., • Integration of the sparse-grid method into the probabilistic numerical scheme for approximating multi-dimensional escape probability with O (Δt) convergence. • Demonstration of the proposed scheme in computing the probability that electrons will runaway from magnetic confinement in nuclear fusion reactors. The remainder of the paper is organized as follows. In Sect. 2, we present the three-dimensional phase space runaway electron model in the particle-based Langevin formulation, as well as its connection with the adjoint equation. Section 3 discusses the mathematical foundation and the numerical algorithm of the proposed method. Section 4 represents the numerical tests of the proposed method for a two-dimensional Brownian motion problem, as well as the three-dimensional RE problem. A summary and concluding remarks are presented in Sect. 5.

2

http://genecode.org/.

248

M. Yang et al.

2 Problem Setting We consider a three-dimensional runaway electron model describing the dynamics of the magnitude of the relativistic momentum, denoted by p, the cosine of the pitch angle θ , denoted by ξ = cos θ , and the minor radius, denoted by r. The relativistic momentum p is normalized using the thermal momentum and the time is normalized using the thermal collisional frequency. That is, if pˆ and tˆ denote the dimensional √ variables, then p = p/(mv ˆ 2T /m is the thermal T ) and t = νee tˆ, where vT = speed with T the plasma temperature and m the electron mass, and the thermal collision frequency is νee = e4 n ln nΛ/(4πε0 m2 vT3 ) with e the absolute value of the electron charge, ε0 is the vacuum permittivity and Λ the Coulomb logarithm. The electric field is normalized using the Dreicer electric field ED . Specifically, the dynamics are modeled by the following stochastic differential equations (SDEs) = > ⎧ γp 1 ∂  2  2 ⎪ ⎪ dp = Eξ − (1 − ξ ) − CF + 2 p CA dt + 2CA dWp , ⎪ ⎪ τ p ∂p ⎪ ⎪ ⎪ @ ?  ⎨  √ ; E 1 − ξ2 ξ(1 − ξ 2 ) CB 2CB dt + − − 2ξ dξ = 1 − ξ 2 dWξ , ⎪ 2 ⎪ p τ γ p p ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ dr = 2Dr dWr , (2.1) where Wp , Wξ and Wr are the standard Brownian motions, E is the electric field, and the functions CA , CB , CF and Dr are defined by ψ(y) , y = > y2 4 1 2 1 Z + φ(y) − ψ(y) + δ , CB (p) = ν¯ ee v¯T 2 y 2 CA (p) = ν¯ ee v¯T2

CF (p) = 2 ν¯ ee v¯T ψ(y), Dr (p) = D0 exp(−(p/Δp)2 ), 2 φ(y) = √ π p y= , γ



y

e

−s 2

ds ,

0

γ =

;

> = dφ 1 , ψ(y) = 2 φ(y) − y 2y dy

1 + (δp)2 ,

vT δ= = c


0. Mathematically, the runaway probability can be described as the escape probability of a stochastic dynamical systems. For notational simplicity, we define Xt := (p, ξ, r), and rewrite the SDE in (2.1) using Xt , i.e., 

t

Xt = X 0 + 0



t

b(Xs )ds +

σ (Xs )dW s with X 0 ∈ D ⊂ R3 ,

(2.2)

0

where D = [pmin , pmax ] × [−1, 1] × [0, 1], and the drift b and the diffusivity σ can be easily defined based on Eq. (2.1). In the following sections, we will use (2.3) to introduce our probabilistic scheme and will come back to Eq. (2.1) in the section of numerical examples. We divide the boundary of D into three parts ∂ D1 , ∂ D2 and ∂ D3 , defined by ∂ D1 := {p = pmax } ∩ ∂ D , ∂ D2 := ({p = pmin } ∪ {r = 1}) ∩ ∂ D , ∂ D3 := ({ξ = −1} ∪ {ξ = 1} ∪ {r = 0}) ∩ ∂ D , such that ∂ D1 ∪ ∂ D2 ∪ ∂ D3 = ∂ D . The boundary ∂ D1 represents the runaway boundary. To give a formal definition of the runaway probability, we denote the runaway time of Xt by  τ := inf t > 0 | Xt ∈ ∂ D1 , which represents the earliest escape time of the process X t that initially starts from X0 = x ∈ D . Then, the runaway probability can be formally defined by PRE (t, x) = P {τ ≤ t | X0 = x ∈ D } .

(2.3)

250

M. Yang et al.

For a fixed T ∈ [0, Tmax ], the probability PRE (T , x) can be represented by the solution of the adjoint equation of the Fokker-Planck equation based on (2.3). Such adjoint equation is a backward parabolic terminal boundary value problem, i.e., ∂u(t, x) + L∗ (t, x)[u(t, x)] = 0 ∂t

for x ∈ D , t < T ,

u(t, x) = 1

for x ∈ ∂ D1 , t ≤ T ,

u(t, x) = 0

for x ∈ ∂ D2 , t ≤ T ,

∇u(t, x) = 0

for x ∈ ∂ D3 , t ≤ T ,

u(T , x) = 0

(2.4)

for x ∈ D ,

where the operator L∗ (t, x) is the adjoint of the Fokker-Planck operator, defined by ∗

L (x)[u] :=

d  i=1

d ∂u ∂ 2u 1  bi i + (σ σ 0 )i,j i j , ∂x 2 ∂x x i,j =1

where bi is the i-th component of the drift b(x), (σ σ 0 )i,j is the (i, j )-th entry of σ σ 0 and x i is the i-th component of x. It is easy to see that PRE (T , x) can be represented by PRE (T , x) = u(0, x).

(2.5)

It should be noted that the runaway probability at each time T requires a solution of the adjoint equation in (2.4), such that recovering the entire dynamics of PRE in [0, Tmax ] requires a sequence of PDE solutions. However, due to the time independence of b and σ in (2.3) considered in this work, the dynamics of PRE (t, x) for (t, x) ∈ [0, Tmax ] × D can be represented by PRE (t, x) = u(Tmax − t, x)

for t ∈ [0, Tmax ],

(2.6)

where u is the solution of (2.4) with T = Tmax .

3 A Sparse-Grid Probabilistic Method for the Adjoint Equation The theoretical foundation of our probabilistic scheme is the Feynman-Kac theory that links the SDE in Eq. (2.3) to the adjoint problem in Eq. (2.4). This section focuses on solving the adjoint equation in Eq. (2.4). The probabilistic representation of v(t, x) and the temporal discretization is given in Sect. 3.1; spatial discretization

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

251

including a special treatment of the involved random escape time τ is provided in Sects. 3.3 and 3.2.

3.1 Temporal Discretization To write out the probabilistic representation of u(t, x) in Eq. (2.4), we need to rewrite the SDE in Eq. (2.3) in a conditional form, i.e.,  X t,x s =x+

s t

b(X t,x )d t¯ + t¯



s t

σ (Xt,x )dW t¯ for s ≥ t, t¯

(3.1)

where the superscript t,x indicates the condition that Xt,x s starts from (t, x) ∈ [0, Tmax ] × D . Accordingly, we can define the conditional escape time [18] 1 2 , τt,x ) τt,x := min(τt,x

(3.2)

with 1 2 t,x τt,x := inf{s > t | Xt,x s ∈ ∂ D1 }, τt,x := inf{s > t | X s ∈ ∂ D2 },

(3.3)

such that the probabilistic representation of the solution u(t, x) of the adjoint equation in (2.4) is given in [19, 20], i.e.,    , u(t, x) = E u s ∧ τt,x , X t,x s∧τt,x

(3.4)

where s ∧ τt,x denotes the minimum of τt,x and s, τt,x is given in Eq. (3.2), and Xt,x s∧τt,x is defined based on Eq. (3.1). We then discretize the probabilistic representation of u in Eq. (3.4). To proceed, we introduce a uniform time partition for [0, Tmax ]:

T := {0 = t0 < t1 < · · · < tN = Tmax } with Δt = tn+1 − tn and ΔW := W tn+1 − W tn for n = 0, 1, . . . , N. The SDE in Eq. (2.3) can be discretized in the interval [tn , tn+1 ] using the forward Euler scheme: n ,x = x + b(x)Δt + σ (x)ΔW , X tn+1

(3.5)

such that the Eq. (3.4) can be discretized (see [32]) as      tn ,x  un (x) = E un+1 Xn+1 1{τtn ,x >tn+1 } + P τt1n ,x ≤ tn+1 ,

(3.6)

252

M. Yang et al.

where un (x) ≈ u(tn , x), τtn ,x is defined in Eq. (3.2),3 and 1{τtn ,x >tn+1 } is the characteristic function of the event that Xstn ,x does not escape the domain D via ∂ D1 ∪ ∂ D2 before tn+1 .

3.2 Sparse-Grid Interpolation for Spatial Discretization To extend the time-stepping scheme in Eq. (3.6) to a fully-discrete scheme, we need to a spatial discretization scheme to approximate un as well as a quadrature rule to estimate the conditional expectation E[·]. In this work, we intend to use piecewise sparse grid interpolation to approximate un (x) in D . Specifically, since the terminal condition of the adjoint equation in Eq. (2.4) is discontinuous, we used hierarchical sparse grids with piecewise polynomials[4, 21], which is easy to incorporate adaptivity to handle the discontinuity.

3.2.1 Hierarchical Sparse Grid Interpolation We briefly recall the standard hierarchical sparse grid interpolation by borrowing the main notations and results from [3, 4, 21, 22]. The one-dimensional hat function having support [−1, 1] is defined by ψ(x) = max{0 , 1 − |x|} from which an arbitrary hat function with support (xL,i − Δx L , xL,i + Δx L ) can be generated by dilation and translation, i.e., ψL,i (x) := ψ

 x + 1 − iΔx  L , Δx L

where L denotes the resolution level, Δx L = 2−L+1 for L = 0, 1, . . . denotes the grid size of the level L grid for the interval [−1, 1], and xL,i = i Δx L − 1 for i = 0, 1, . . . , 2L denotes the grid points of that grid. The basis function ψL,i (x) has local support and is centered at the grid point xL,i ; the number of grid points in the level L grid is 2L + 1. One can generalize the piecewise linear hierarchical polynomials to high-order hierarchical polynomials, as shown in [3, 4]. As shown in Fig. 1, for L ≥ 0, a piecewise linear polynomial ψL,i (x) is defined based on 3 supporting points, i.e., xL,i and its two ancestors that are also the endpoints of the support [xL,i −Δx L , xL,i +Δx L ]. For q-th order polynomials, q + 1 supporting points are needed to define a Lagrange interpolating polynomial. To do this, at each grid point xL,i , additional ancestors outside of [xL,i − Δx L , xL,i + Δx L ] are borrowed to help build a higher-

The escape time τtn ,x in Eq. (3.6) should be defined by replacing Xt,x with the Euler s discretization, i.e., Xtsn ,x = x + b(x)(s − tn ) + σ (x)(W s − W tn ) for s ≥ tn in Eq. (3.2). We use the same notation without creating confusion.

3

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

253

Fig. 1 Left: linear hierarchical basis; Middle: quartic hierarchical basis where the quadratic polynomials appear since level 2; Right: cubic hierarchical basis where the cubic polynomials appear since level 3

order Lagrange polynomial, then, the desired high-order polynomial is defined by restricting the resulting polynomial to the support [xL,i −Δx L , xL,i +Δx L ]. Figure 1 illustrates the linear, quadratic and cubic hierarchical bases, respectively. With Z = L2 (D ), a sequence of subspaces {ZL }∞ L=0 of Z of increasing dimension 2L + 1 can be defined as  for L = 0, 1, . . . . ZL = span ψL,i (x) | i = 0, 1, . . . , 2L Due to the nesting propertyof {ZL }∞ l=0 , we can define a sequence $  of hierarchical subspaces as WL = span ψL,i (x) | i ∈ BL where BL = i ∈ N $ i = 1, 3, 5, . . . , 2L − 1 for L = 1, 2, . . ., such that ZL = ZL−1 ⊕ WL and WL =  ZL / ⊕L−1 L =0 ZL for L = 1, 2, . . .. Then, the hierarchical subspace splitting of ZL is given by ZL = Z0 ⊕ W1 ⊕ · · · ⊕ WL

for L = 1, 2, . . . .

The one-dimensional hierarchical polynomial basis can be extended to the Ndimensional domain using sparse tensorization. Specifically, the N-variate basis function ψl,i (x) associated with the point x l,i = (xL1 ,i1 , . . . , xLN ,iN ) is defined  N using tensor products, i.e., ψl,i (x) := N n=1 ψLn ,in (xn ), where {ψLn ,in (xn )}n=1 are the one-dimensional hierarchical polynomials associated with the point xLn ,in = in Δx Ln − 1 with Δx Ln = 2−Ln +1 and l = (L1 , . . . , LN ) is a multi-index indicating the resolution level of the basis function. The N-dimensional hierarchical

254

M. Yang et al.

incremental subspace Wl is defined by Wl =

N 

$  WLn = span ψl,i (x) $ i ∈ Bl ,

n=1

where the multi-index set Bl is given by %

$ $ in ∈ {1, 3, 5, . . . , 2Ln − 1} for n = 1, . . . , N Bl := i ∈ N $$ for n = 1, . . . , N in ∈ {0, 1} N

& if Ln > 0 if Ln = 0

.

Similar to the one-dimensional case, a sequence of subspaces, again denoted by 2 {ZL }∞ L=0 , of the space Z := L (D ) can be constructed as ZL =

L 0

WL =

L =0

L 0 0

Wl ,

L =0 α(l )=L

where the key is how the mapping α(l) is defined because it defines the incremental subspaces WL = ⊕α(l )=L Wl . For example, α(l) = |l| = L1 + . . . + LN leads to a standard isotropic sparse polynomial space. The level L hierarchal sparse grid interpolant of the approximation un (x) in Eq. (3.6) is defined by unL (x) :=

L   L =0 |l |=L

(ΔL ⊗ · · · ⊗ ΔLN )un (x)

= unL−1 (x) +

1

 |l |=L

=

unL−1 (x) +

(ΔL1 ⊗ · · · ⊗ ΔLN )un (x)

 

|l |=L i∈Bl

= unL−1 (x) +

 

u

n

(x l ,i ) − unL −1 (x l ,i )



(3.7) ψl ,i (x)

cl ,i ψl ,i (x),

|l |=L i∈Bl

where cl ,i = un (x l ,i ) − unL −1 (x l ,i ) is the multi-dimensional hierarchical surplus [11]. This interpolant is a direct extension, via the Smolyak algorithm [27], of the one-dimensional hierarchical interpolant.

3.2.2 A Strategy for Handling the Boundary Condition After the sparse grid, denoted by S , is constructed, the task becomes to estimate the right-hand side of Eq. (3.6) at all the interior sparse grid points x i ∈ S ∩ D . The

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

255

accuracy of such estimation also depends on how to deal with P(τt1n ,x ≤ tn+1 ). It is known that P(τt1n ,x ≤ tn+1 ) → 1 as x → ∂ D1 . In our previous work [32], we proved, in the one-dimensional case, that if b and σ are bounded functions, i.e., |b(t, x)| ≤ b and |σ (t, x)| ≤ σ for (t, x) ∈ [0, T ] × D , with 0 ≤ b, σ ≤ +∞, and the starting point x in Eq. (3.5) is sufficiently far from the boundary ∂ D satisfying dist (x, ∂ D ) ∼ O ((Δt)1/2−ε ) for any given constant ε > 0, then for sufficiently small Δt, it holds that  P(τt1n ,x ≤ tn+1 ) ≤ C(Δt)ε exp −

1 (Δt)2ε

 ,

(3.8)

where the constant C > 0 is independent of Δt. Even though the estimate in Eq. (3.8) was proved for the one-dimensional case, we exploited the estimate in the three-dimensional to design our numerical scheme. The key idea is to eliminate the destructive effect of P(τt1n ,x ≤ tn+1 ) in the construction of the temporal-spatial discretization scheme by exploiting the estimate in Eq. (3.8). Specifically, we define the spatial mesh size Δx of the sparse grid is on the order of   1 Δx ∼ O (Δt) 2 −ε , such that, for each interior grid point x i , un (x i ) in Eq. (3.6) can be approximated by    n ,xi un (xi ) ≈ E un+1 X tn+1 , L

(3.9)

with the error on the order of O ((Δt)ε exp(−1/(Δt)2ε )). The specific choice of Δx will be given in Sect. 3.3. Such strategy can avoid the approximation of the escape probability P(τt1n ,x ≤ tn+1 ), but the trade-off is that we need to use higher order sparse grid interpolation to balance the total error.

3.3 Quadrature for the Conditional Expectation The last piece of thepuzzleis a quadrature rule for estimating the conditional n ,x i X tn+1 for x i ∈ S ∩ D . Such expectation can be written as expectations E un+1 L    tn ,x i (X ) = E un+1 n+1 L

Rd

  √ x i + b(x i )Δt + σ (x i ) 2Δt η ρ(η)dη, un+1 L

(3.10)

256

M. Yang et al.

where η := (η1 , . . . , ηd  ) follows the d-dimensional standard normal distribution, 1 i.e., ρ(η) := π d/2 exp(− d =1 η 2 ). Thus, we utilized tensor-product Gauss-Hermite quadrature rule to approximate the expectation. Specifically, we denote by {wj }Jj=1 and {aj }Jj=1 the weights and abscissae of the J -point tensor-product Gauss-Hermite tn ,x i rule, respectively. Then the approximation, denoted by 8 E[un+1 L (X n+1 )] is defined by J    tn ,x i uni = 8 E un+1 (X ) = wj un+1 n+1 L L (q ij ),

(3.11)

j =1

with √ q ij := x i + b(x i )Δt + σ (x i ) 2Δt aj

(3.12)

where ωj is a product of the weights of the one-dimensional rule and aj is a ddimensional vector consisting of one-dimensional abscissae, respectively. When 2J ∗ un+1 /∂η2J ∗ is bounded for = 1, . . . , d un+1 L (·) is sufficiently smooth, i.e., ∂ with J ∗ = J 1/d , then the quadrature error can be bounded by [23] $ $ $8 n+1 tn ,x i tn ,x i $ $E[uL (X n+1 )] − E[un+1 L (X n+1 )]$ ≤ C

J ∗! ∗ (Δt)J , 2 (2J ∗ )! J∗

∗

where the constant C is independent of J ∗ and Δt. Note that the factor (Δt)J comes from the 2J ∗ -th order differentiation of the function un+1 with respect to η for = 1, . . . , d. Thus, to achieve first order global convergence rate O (Δt), we only need to use a total of J ∗ = 27 quadrature points. Sparse-grid Gauss-Hermite rule could be used to replace the tensor product rule when the dimension d is higher than 3. For the 3D runaway electron problem under consideration, we found that a level 1 sparse Gauss-Hermite rule with 7 quadrature points cannot provide sufficient accuracy, and a level 2 rule with 37 points is more expensive than the tensor product rule. Thus, we chose to use the tensor-product rule in this work. By putting together all the components introduced in Sect. 3, we summarize our probabilistic scheme as follows: Scheme 1 (The Fully-Discrete Probabilistic Scheme) Given the temporal-spatial partition T × S , the terminal condition uN (x i ) for x i ∈ S , and the boundary condition un (x i ) for x i ∈ S ∩ ∂ D . For n = N − 1, . . . , 0, the approximation of u(tn , x) is constructed via the following steps: • Step 1: generate quadrature abscissae {q ij }Jj=1 , in Eq. (3.12), for x i ∈ S ∩ D ;

n+1 J • Step 2: interpolate un+1 L (x) at the quadrature abscissae to obtain {uL (qij )}j =1 ; n • Step 3: compute the coefficients ui using the quadrature rule in Eq. (3.11); • Step 4: construct the interpolant unL (x) by substituting uin into Eq. (3.7).

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

257

A major novelty of the proposed method is that Scheme 1 is the first numerical method, which integrates sparse grids into a probabilistic scheme, for computing escape probabilities of stochastic dynamical systems. Even though the escape probability can be computed by solving the adjoint equation in Eq. (2.4) with sparsegrid-based PDE solvers, there are several significant advantages of combining sparse grids with the probabilistic scheme. First, the time-stepping scheme is fully explicit but absolutely stable, which has been rigorously proved in our previous work, e.g., [33, 34]. Second, the Feynman-Kac formula makes it natural to incorporate any sparse grid interpolation strategies to approximate the solution u without worrying about the discretization of the differential operator on the sparse grid. Third, it is easy to incorporate legacy codes for Monte Carlo based RE simulation into our scheme to compute runaway probability. This is a valuable feature because realworld RE models usually involve complex multiscale dynamics that is challenging to solve using PDE approaches.

4 Numerical Examples We tested our probabilistic scheme with two examples. In the first example, we compute the escape probability of the standard Brownian motion. Since we know the analytical expression of the escape probability, this example is used to demonstrate the accuracy of our approach. In the second example, we to compute the runaway probability of the three-dimensional RE model given in Sect. 2. The sparse grid interpolation and adaptive refinement are implemented using the TASMANIAN toolbox [28].

4.1 Example 1: Escape Probability of a Brownian Motion We consider the escape probability of a two-dimensional Brownian motion [26]. The spatial domain D is set to [0, 5] × [0, 5] and the temporal domain is set to t ∈ [0, 2] with Tmax = 2. The escape probability P (t, x) can be obtained by solving the standard heat equation ∂u 1 + Δu = 0, ∂t 2 u(t, x) = 1, u(Tmax , x) = 0,

(t, x) ∈ [0, Tmax ] × D , (t, x) ∈ [0, Tmax ] × ∂ D , x ∈ D.

(4.1)

258

M. Yang et al.

The exact solution is given by u(t, x) = 1 +

∞ ∞  

Amn sin (μm x1 ) sin (νn x2 )e−λmn t , 2

m=1 n=1

where μm =

mπ 5 , νn

=

nπ 5 ,

λ=

;

1 . 2(μ2m +νn2 )

The escape probability P (t, x) can be

obtained by substituting u into Eq. (2.6), i.e., P (t, x) = u(Tmax − t, x). We intend to demonstrate that our scheme can achieve first-order convergence O (Δt) when properly choosing the sparse grid resolution, i.e., the level L. To this end, we compare three cases, i.e., √ (a) Hierarchical cubic basis with Δx ∼ O ( √Δt), (b) Hierarchical linear basis with Δx ∼ O ( Δt), (c) Hierarchical cubic basis with Δx ∼ O (Δt), where Δx denotes the mesh size of the one-dimensional rule for building the sparse grids. The error of the three √ cases are shown in Figs. 2 and  3. As expected,  in case (a), i.e., Δx ∼ O ( Δt), the escape probability P τtn ,x ≤ tn+1 for any interior grid point is on the order of O ((Δt)ε exp(−1/(Δt)2ε )), such that

10

0

10 -1

10 -2

Slope = -1 10

-3

10

-4

10 1

10 2

10 3

10 4

Fig. 2 The relative error of the approximate escape probability √ of the standard Brownian motion for √ the three test cases, i.e., (a) cubic basis with Δx ∼ O ( Δt), (b) linear basis with Δx ∼ O ( Δt), (c) cubic basis with Δx ∼ O (Δt)

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

259

Fig. 3 The error distribution in the spatial domain [0, 5] × [0, 5] for t = 0.5, 1.0 and 2.0. The first row corresponds to the case (a), the second row corresponds to the case (b), and the third row corresponds to the case (c) in figure

  neglecting P τtn ,x ≤ tn+1 will asymptotically not affect the first-order convergence w.r.t. Δt. On the other hand, we need to use high-order hierarchical basis to achieve comparable accuracy in spatial approximation. It is shown in Fig. 3 that the use of the hierarchical cubic polynomials, introduced in [4], provides sufficient accuracy to achieve a global convergence √ rate O (Δt). In comparison, in case (b), i.e., using linear basis with Δx ∼ O ( Δt), the linear sparse-grid interpolation only provides O ((Δx)2 ) = O (Δt) local convergence, such that our scheme dose not converge globally. From the second row of Fig. 3, we can see that large errors are generated around the boundary of the spatial domain and gradually propagate to the middle

260

M. Yang et al.

region of the domain. Similar phenomenon appears in the case (c) when setting Δx ∼ O (Δt). In this case, the interior grid points near theboundary are  so close to the boundary that neglecting the escape probability P τtn ,x ≤ tn+1 leads to significant additional error. This is the reason why big errors are firstly generated near the boundary (i.e., t = 0.5), and then propagate to the center.

4.2 The Runaway Probability of the Three-Dimensional RE Model Here we test our method using the 3D runaway electron model given in Eq. (2.1) with the following parameters: Tmax = 120, pmin = 2, pmax = 50, Z = 1, τ = 105, δ = 0.042, E = 0.3, v¯ee = 1, v¯T = 1, D0 = 0.003, Δp = 20. Unlike the example about Brownian motion, where the discontinuous terminal condition is smoothed out very fast, the evolution of the runaway probability PRE is more convection-dominated. As such, we utilized adaptive sparse grids to capture the movement of the sharp transition layer. The standard refinement approach is to construct an initial grid using all points up to some coarse level, then consider the hierarchical surplus coefficients, e.g., the coefficients of the basis functions, which are estimates of the local approximation error in the neighborhood of the associated nodes. The coarse grid is refined by adding the children of nodes with large coefficients ignoring all other points. Such refinement process is repeated until all coefficients fall below some desired tolerance. However, the standard refinement process may stagnate when dealing with functions with localized sharp behavior which results in non-monotonic decay of the coefficients (in the pre-asymptotic regime). In such scenario, a node located in the sharp region could have parent nodes with small surpluses, such that a necessary refinement will be missed in the local sharp region. Even if descendants of the node converge in the sharp region (following paths through other parents), the children have restricted support such that they cannot compensate for the missing parent. A common remedy for this problem is to recursively add all parents of all nodes, but this not desirable as it includes many nodes with small coefficients which would have been ignored in the classic refinement. Therefore, we utilized a more flexible refinement procedure that considers both parents and children of nodes with large coefficients, so as to improve stability and avoid oversampling. Specifically, for each node on the current sparse grid, we first build a set that include both parents and children of the node. Then, we add the children nodes to the sparse grid only if all the parent nodes are already included in the current grid. The parents selective refinement procedure is described in details in [29] and it is implemented in the TASMANIAN open source library [28, 30].

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

261

Fig. 4 Cross sections of the runaway probability PRE as well as the corresponding adaptive sparse grids at three instants of time t = 24, 60 and 120

The evolution of the runaway probability PRE as well as the corresponding adaptive sparse grids are shown in Fig. 4. The runaway boundary is at p = pmax = 50. The main reason of an electron running away is the electric field acceleration, i.e., the term Eξ in the drift of the momentum dynamics. The factor ξ = cos(θ ) in Eξ determines that the electrons with small pitch angles will runaway sooner than the electrons with large pitch angles, which is consistent with the simulation results in Fig. 4. There are two sharp transition layers in this simulation, i.e., the transition between the runaway and the non-runaway regions, and the boundary layer around r = 1 due to small diffusion effect in the minor radius direction. In our simulation, we used the 6-level sparse grid as the initial grid and gradually refine it with the tolerance being 0.001. As expected, the adaptive refinement accurately captured the irregular behaviors. In addition, since the analytical expression of PRE is unknown, we tested the accuracy of our approach by comparing with the direct Monte Carlo

262

M. Yang et al.

Fig. 5 Comparison between the our approach and the direct MC for pitch angle θ = 15◦ and minor radius r = 0.5

method for computing PRE at a few locations in the phase space, and the result is shown in Fig. 5. We can see that the RE probability obtained by our approach is consistent with the MC simulations with 10,000 particles, which numerically demonstrate the accuracy of our method. Another observation is that the MC method can only compute PRE at one location in the phase space at a time, such that PRE at the five locations shown in Fig. 5 requires five repeat simulations of 10,000 particles with different initialization. In comparison, our method can compute PRE at all locations by solving the adjoint equation only once.

5 Concluding Remarks We proposed a sparse-grid probabilistic scheme for the accurate and efficient computation of the time-dependent probability of runaway. The method is based on the direct numerical solution of the Feynman-Kac formula. At each time step the algorithm reduces to the computation of an integral involving the previously computed probability of runaway and the Gaussian propagator. Sparse-grid interpolation is utilized to recover the runaway probability function as well as evaluate the

A Sparse-Grid Probabilistic Scheme for Runaway Electron Simulation

263

quadrature points for estimating the conditional expectation in the Feynman-Kac formulation. The integration of sparse grid into the probabilistic scheme provides a fully explicit and stable algorithm to compute the escape probabilities of stochastic dynamics with O (Δt) convergence. Moreover, the adaptive refinement strategy is demonstrated to be effective in capturing the movement of the sharp transition layer of the runaway probability function. In our future work, we intend to extend our approach to higher dimensional RE problems involving more complicated dynamics. For example, an important RE model to be resolved is to incorporate the relativistic guiding center equations of electron motion into the RE scenario. In this case, the deterministic dynamics of the guiding center motion is six order of magnitudes smaller than the collisional dynamics, which presents significant challenge to the design of numerical schemes. Acknowledgments This material is based upon work supported in part by the U.S. Department of Energy, Office of Science, Offices of Advanced Scientific Computing Research and Fusion Energy Science, and by the Laboratory Directed Research and Development program at the Oak Ridge National Laboratory, which is operated by UT-Battelle, LLC, for the U.S. Department of Energy under Contract DE-AC05-00OR22725.

References 1. M. M. Ali, P. E. Strazdins, B. Harding, and M. Hegland. Complex scientific applications made fault-tolerant with the sparse grid combination technique. The International Journal of High Performance Computing Applications, 30(3):335–359, 2016. 2. A. H. Boozer. Theory of runaway electrons in ITER: Equations, important parameters, and implications for mitigation. Physics of Plasmas, 22(3):032504, 2015. 3. H.-J. Bungartz. Concepts for higher order finite elements on sparse grids. Houston Journal of Mathematics, pages 159–170, 1996. 4. H.-J. Bungartz and M. Griebel. Sparse grids. Acta Numerica, 13:1–123, June 2004. 5. N. W. Eidietis, N. Commaux, E. M. Hollmann, D. A. Humphreys, T. C. Jernigan, R. A. Moyer, E. J. Strait, M. A. VanZeeland, J. C. Wesley, and J. H. Yu. Control of post-disruption runaway electron beams in diii-d. Physics of Plasmas, 19(5):056109, 2012. 6. I.-G. Farcas, T. Görler, H.-J. Bungartz, F. Jenko, and T. Neckel. Sensitivity-driven adaptive sparse stochastic approximations in plasma microinstability analysis. arXiv e-prints, page arXiv:1812.00080, Nov 2018. 7. T. Gerstner and M. Griebel. Dimension–Adaptive Tensor–Product Quadrature. Computing, 71(1):65–87, Aug. 2003. 8. M. Griebel. Adaptive sparse grid multilevel methods for elliptic PDEs based on finite differences. Computing, 61(2):151–179, June 1998. 9. M. Heene, A. P. Hinojosa, M. Obersteiner, H.-J. Bungartz, and D. Pflüger. EXAHD: An exascalable two-level sparse grid approach for higher-dimensional problems in plasma physics and beyond. In W. E. Nagel, D. H. Kröner, and M. M. Resch, editors, High Performance Computing in Science and Engineering ’ 17, pages 513–529, Cham, 2018. Springer International Publishing. 10. E. M. Hollmann, P. B. Aleynikov, T. Fülöp, D. A. Humphreys, V. A. Izzo, M. Lehnen, V. E. Lukash, G. Papp, G. Pautasso, F. Saint-Laurent, and J. A. Snipes. Status of research toward the ITER disruption mitigation system. Physics of Plasmas, 22(2):021802, 2015.

264

M. Yang et al.

11. A. Klimke and B. Wohlmuth. Algorithm 847: Spinterp: piecewise multilinear hierarchical sparse grid interpolation in matlab. ACM Transactions on Mathematical Software (TOMS), 31(4):561–579, 2005. 12. H. Knoepfel and D. A. Spong. Runaway electrons in toroidal discharges. Nuclear Fusion, 19:785–829, June 1979. 13. K. Kormann and E. Sonnendrücker. Sparse grids for the Vlasov–Poisson equation. In J. Garcke and D. Pflüger, editors, Sparse Grids and Applications - Stuttgart 2014, pages 163–190, Cham, 2016. Springer International Publishing. 14. C. Liu, D. P. Brennan, A. Bhattacharjee, and A. H. Boozer. Adjoint Fokker-Planck equation and runaway electron dynamics. Physics of Plasmas, 23(1):010702, 2016. 15. E. Marsch. Kinetic physics of the solar corona and solar wind. Living Reviews in Solar Physics, 3(1):1, Jul 2006. 16. J. R. Martín-Solís, A. Loarte, and M. Lehnen. Runaway electron dynamics in tokamak plasmas with high impurity content. Physics of Plasmas, 22(9):092512, 2015. 17. M. Obersteiner, A. P. Hinojosa, M. Heene, H.-J. Bungartz, and D. Pflüger. A highly scalable, algorithm-based fault-tolerant solver for gyrokinetic plasma simulations. In Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems, ScalA’17, New York, NY, USA, 2017. Association for Computing Machinery. 18. B. K. Øksendal. Stochastic Differential Equations. An Introduction with Applications. Springer Science & Business, Berlin, Heidelberg, Nov. 2010. 19. É. Pardoux. Backward stochastic differential equations and viscosity solutions of systems of semilinear parabolic and elliptic PDEs of second order. In Stochastic Analysis and Related Topics VI, pages 79–127, Boston, MA, 1998. Birkhäuser Boston. 20. S. Peng. A general stochastic maximum principle for optimal control problems. SIAM Journal on control and optimization, 28(4):966–979, 1990. 21. D. Pflüger, B. Peherstorfer, and H.-J. Bungartz. Spatially adaptive sparse grids for highdimensional data-driven problems. Journal of Complexity, 26(5):508–522, 2010. 22. D. M. Pflüger. Spatially Adaptive Sparse Grids for High-Dimensional Problems. Dissertation, Technische Universität München, München, 2010. 23. A. Quarteroni, R. Sacco, and F. Saleri. Numerical Mathematics, volume 332. Springer Science Business Media &, 2007. 24. L. F. Ricketson and A. J. Cerfon. Sparse grid techniques for particle-in-cell schemes. Plasma Physics and Controlled Fusion, 59(2):024002, dec 2016. 25. M. Rosenbluth and S. Putvinski. Theory for avalanche of runaway electrons in tokamaks. Nuclear Fusion, 37(10):1355–1362, oct 1997. 26. Z. Schuss. Brownian Dynamics at Boundaries and Interfaces. In Physics, Chemistry, and Biology. Springer Science & Business Media, Aug. 2013. 27. S. Smolyak. Quadrature and interpolation formulas for tensor products of certain classes of functions. Doklady Akademii Nauk SSSR, 4:240–243, 1963. 28. M. Stoyanov. User manual: Tasmanian sparse grids. Technical Report ORNL/TM-2015/596, Oak Ridge National Laboratory, One Bethel Valley Road, Oak Ridge, TN, 2015. 29. M. Stoyanov. Adaptive sparse grid construction in a context of local anisotropy and multiple hierarchical parents. In Sparse Grids and Applications-Miami 2016, pages 175–199. Springer, 2018. 30. M. Stoyanov, D. Lebrun-Grandie, J. Burkardt, and D. Munster. Tasmanian, November 2013. 31. Z. Tao, W. Guo, and Y. Cheng. Sparse grid discontinuous Galerkin methods for the VlasovMaxwell system. Journal of Computational Physics: X, 3:100022, 2019. 32. J. Yang, G. Zhang, and W. Zhao. A First-Order Numerical Scheme for Forward-Backward Stochastic Differential Equations in Bounded Domains. Journal of Computational Mathematics, 36(2):237–258, 2018. 33. G. Zhang and D. del Castillo-Negrete. A backward Monte-Carlo method for time-dependent runaway electron simulations. Physics of Plasmas, 24(9):092511, Sept. 2017. 34. W. Zhao, G. Zhang, and L. Ju. A stable multistep scheme for solving backward stochastic differential equations. SIAM Journal on Numerical Analysis, 48(4):1369–1394, 2010.