Submit to Entropy Review for Entropy Propose a Special Issue

Journal Browser

► Journal Browser

Information Theory and Stochastics for Multiscale Nonlinear Systems

Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Entropy (ISSN 1099-4300).

Deadline for manuscript submissions: closed (30 September 2019) | Viewed by 40652

Share This Special Issue

Special Issue Editors

Prof. Andrew J. Majda

E-Mail Website
Guest Editor

Department of Mathematics and Climate, Atmosphere, Ocean Science (CAOS),Courant Institute of Mathematical Sciences, New York University, New York, NY 10012, USA
Interests: applied mathematics; stochastic nonlinear modeling; prediction; data assimilation; information theory; uncertainty quantification; climate atmosphere and ocean; statistical physics

Dr. Nan Chen

E-Mail Website
Guest Editor

Department of Mathematics, University of Wisconsin-Madison, Madison, WI 53706, USA
Interests: data assimilation; information theory; uncertainty quantification; climate atmosphere and ocean modeling; machine learning
Special Issues, Collections and Topics in MDPI journals

Prof. Markos A. Katsoulakis

E-Mail Website
Guest Editor

Department of Mathematics and Statistics, University of Massachusetts, Amherst, MA 01003, USA
Interests: uncertainty quantification; information theory; model reduction; statistical mechanics and nonlinear PDEs; multi-scale systems and applications in material science and energy research

Special Issue Information

Dear Colleagues,

Complex multiscale nonlinear stochastic dynamical systems are ubiquitous complex systems in geoscience, engineering, neural and material sciences. They are a grand challenge in contemporary science and engineering. Key issues are their basic mathematical structural properties and qualitative features, their statistical prediction, uncertainty quantification (UQ) and sensitivity, their data assimilation (also known as state estimation or filtering), and coping with the inevitable model errors that arise in approximating such complex systems. These model errors arise through both the curse of small ensemble size for large systems and the lack of physical understanding. Effective reduced nonlinear stochastic models in recent years often blended ideas from information theory, Bayesian statistics, and statistical physics in an emerging paradigm for these grand challenges, including extreme events prediction. In addition to multiscale nonlinear stochastic differential equations, multiscale Markov jump process and Markov chains are also important in applications using this paradigm.

This Special Issue focuses on original and new results concerning information theory, stochastic modeling and complex multiscale nonlinear systems in the paradigm as described above. Contributions for this Special Issue can involve one or several disciplines including mathematics, Bayesian statistics, statistical physics and applications.

Prof. Andrew J. Majda
Dr. Nan Chen
Prof. Markos A. Katsoulakis
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Entropy is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

complex multiscale nonlinear stochastic dynamical systems
uncertainty quantification
prediction
data assimilation
stochastic models
extreme events
Bayesian statistics
model error

Published Papers (11 papers)

Download All Papers

Research

18 pages, 6499 KiB

Open AccessArticle

Machine Learning Predictors of Extreme Events Occurring in Complex Dynamical Systems

by Stephen Guth and Themistoklis P. Sapsis

Entropy 2019, 21(10), 925; https://doi.org/10.3390/e21100925 - 23 Sep 2019

Cited by 30 | Viewed by 3751

Abstract

The ability to characterize and predict extreme events is a vital topic in fields ranging from finance to ocean engineering. Typically, the most-extreme events are also the most-rare, and it is this property that makes data collection and direct simulation challenging. We consider the problem of deriving optimal predictors of extremes directly from data characterizing a complex system, by formulating the problem in the context of binary classification. Specifically, we assume that a training dataset consists of: (i) indicator time series specifying on whether or not an extreme event occurs; and (ii) observables time series, which are employed to formulate efficient predictors. We employ and assess standard binary classification criteria for the selection of optimal predictors, such as total and balanced error and area under the curve, in the context of extreme event prediction. For physical systems for which there is sufficient separation between the extreme and regular events, i.e., extremes are distinguishably larger compared with regular events, we prove the existence of optimal extreme event thresholds that lead to efficient predictors. Moreover, motivated by the special character of extreme events, i.e., the very low rate of occurrence, we formulate a new objective function for the selection of predictors. This objective is constructed from the same principles as receiver operating characteristic curves, and exhibits a geometric connection to the regime separation property. We demonstrate the application of the new selection criterion to the advance prediction of intermittent extreme events in two challenging complex systems: the Majda–McLaughlin–Tabak model, a 1D nonlinear, dispersive wave model, and the 2D Kolmogorov flow model, which exhibits extreme dissipation events. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

32 pages, 1673 KiB

Open AccessArticle

Parameter Estimation with Data-Driven Nonparametric Likelihood Functions

by Shixiao W. Jiang and John Harlim

Entropy 2019, 21(6), 559; https://doi.org/10.3390/e21060559 - 03 Jun 2019

Cited by 5 | Viewed by 2657

Abstract

In this paper, we consider a surrogate modeling approach using a data-driven nonparametric likelihood function constructed on a manifold on which the data lie (or to which they are close). The proposed method represents the likelihood function using a spectral expansion formulation known as the kernel embedding of the conditional distribution. To respect the geometry of the data, we employ this spectral expansion using a set of data-driven basis functions obtained from the diffusion maps algorithm. The theoretical error estimate suggests that the error bound of the approximate data-driven likelihood function is independent of the variance of the basis functions, which allows us to determine the amount of training data for accurate likelihood function estimations. Supporting numerical results to demonstrate the robustness of the data-driven likelihood functions for parameter estimation are given on instructive examples involving stochastic and deterministic differential equations. When the dimension of the data manifold is strictly less than the dimension of the ambient space, we found that the proposed approach (which does not require the knowledge of the data manifold) is superior compared to likelihood functions constructed using standard parametric basis functions defined on the ambient coordinates. In an example where the data manifold is not smooth and unknown, the proposed method is more robust compared to an existing polynomial chaos surrogate model which assumes a parametric likelihood, the non-intrusive spectral projection. In fact, the estimation accuracy is comparable to direct MCMC estimates with only eight likelihood function evaluations that can be done offline as opposed to 4000 sequential function evaluations, whenever direct MCMC can be performed. A robust accurate estimation is also found using a likelihood function trained on statistical averages of the chaotic 40-dimensional Lorenz-96 model on a wide parameter domain. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

23 pages, 1614 KiB

Open AccessArticle

State and Parameter Estimation from Observed Signal Increments

by Nikolas Nüsken, Sebastian Reich and Paul J. Rozdeba

Entropy 2019, 21(5), 505; https://doi.org/10.3390/e21050505 - 17 May 2019

Cited by 9 | Viewed by 3366

Abstract

The success of the ensemble Kalman filter has triggered a strong interest in expanding its scope beyond classical state estimation problems. In this paper, we focus on continuous-time data assimilation where the model and measurement errors are correlated and both states and parameters need to be identified. Such scenarios arise from noisy and partial observations of Lagrangian particles which move under a stochastic velocity field involving unknown parameters. We take an appropriate class of McKean–Vlasov equations as the starting point to derive ensemble Kalman–Bucy filter algorithms for combined state and parameter estimation. We demonstrate their performance through a series of increasingly complex multi-scale model systems. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

25 pages, 878 KiB

Open AccessArticle

Diffusion Equation-Assisted Markov Chain Monte Carlo Methods for the Inverse Radiative Transfer Equation

by Qin Li and Kit Newton

Entropy 2019, 21(3), 291; https://doi.org/10.3390/e21030291 - 18 Mar 2019

Cited by 8 | Viewed by 3231

Abstract

Optical tomography is the process of reconstructing the optical properties of biological tissue using measurements of incoming and outgoing light intensity at the tissue boundary. Mathematically, light propagation is modeled by the radiative transfer equation (RTE), and optical tomography amounts to reconstructing the scattering coefficient in the RTE using the boundary measurements. In the strong scattering regime, the RTE is asymptotically equivalent to the diffusion equation (DE), and the inverse problem becomes reconstructing the diffusion coefficient using Dirichlet and Neumann data on the boundary. We study this problem in the Bayesian framework, meaning that we examine the posterior distribution of the scattering coefficient after the measurements have been taken. However, sampling from this distribution is computationally expensive, since to evaluate each Markov Chain Monte Carlo (MCMC) sample, one needs to run the RTE solvers multiple times. We therefore propose the DE-assisted two-level MCMC technique, in which bad samples are filtered out using DE solvers that are significantly cheaper than RTE solvers. This allows us to make sampling from the RTE posterior distribution computationally feasible. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

12 pages, 350 KiB

Open AccessArticle

Steady-State Analysis of a Flexible Markovian Queue with Server Breakdowns

by Messaoud Bounkhel, Lotfi Tadj and Ramdane Hedjar

Entropy 2019, 21(3), 259; https://doi.org/10.3390/e21030259 - 07 Mar 2019

Cited by 6 | Viewed by 2956

Abstract

A flexible single-server queueing system is considered in this paper. The server adapts to the system size by using a strategy where the service provided can be either single or bulk depending on some threshold level c. If the number of customers in the system is less than c, then the server provides service to one customer at a time. If the number of customers in the system is greater than or equal to c, then the server provides service to a group of c customers. The service times are exponential and the service rates of single and bulk service are different. While providing service to either a single or a group of customers, the server may break down and goes through a repair phase. The breakdowns follow a Poisson distribution and the breakdown rates during single and bulk service are different. Also, repair times are exponential and repair rates during single and bulk service are different. The probability generating function and linear operator approaches are used to derive the system size steady-state probabilities. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

21 pages, 801 KiB

Open AccessArticle

A Study of the Cross-Scale Causation and Information Flow in a Stormy Model Mid-Latitude Atmosphere

by X. San Liang

Entropy 2019, 21(2), 149; https://doi.org/10.3390/e21020149 - 05 Feb 2019

Cited by 5 | Viewed by 2963

Abstract

A fundamental problem regarding the storm–jet stream interaction in the extratropical atmosphere is how energy and information are exchanged between scales. While energy transfer has been extensively investigated, the latter has been mostly overlooked, mainly due to a lack of appropriate theory and methodology. Using a recently established rigorous formalism of information flow, this study attempts to examine the problem in the setting of a three-dimensional quasi-geostrophic zonal jet, with storms excited by a set of optimal perturbation modes. We choose for this study a period when the self-sustained oscillation is in quasi-equilibrium, and when the energetics mimick the mid-latitude atmospheric circulation where available potential energy is cascaded downward to smaller scales, and kinetic energy is inversely transferred upward toward larger scales. By inverting a three-dimensional elliptic differential operator, the model is first converted into a low-dimensional dynamical system, where the components correspond to different time scales. The information exchange between the scales is then computed through ensemble prediction. For this particular problem, the resulting cross-scale information flow is mostly from smaller scales to larger scales. That is to say, during this period, this model extratropical atmosphere is dominated by a bottom-up causation, as collective patterns emerge out of independent entities and macroscopic thermodynamic properties evolve from random molecular motions. This study makes a first step toward an important field in understanding the eddy–mean flow interaction in weather and climate phenomena such as atmospheric blocking, storm track, North Atlantic Oscillation, to name a few. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

21 pages, 2800 KiB

Open AccessArticle

Reduced Models of Point Vortex Systems

by Jonathan Maack and Bruce Turkington

Entropy 2018, 20(12), 914; https://doi.org/10.3390/e20120914 - 30 Nov 2018

Cited by 5 | Viewed by 2508

Abstract

Nonequilibrium statistical models of point vortex systems are constructed using an optimal closure method, and these models are employed to approximate the relaxation toward equilibrium of systems governed by the two-dimensional Euler equations, as well as the quasi-geostrophic equations for either single-layer or two-layer flows. Optimal closure refers to a general method of reduction for Hamiltonian systems, in which macroscopic states are required to belong to a parametric family of distributions on phase space. In the case of point vortex ensembles, the macroscopic variables describe the spatially coarse-grained vorticity. Dynamical closure in terms of those macrostates is obtained by optimizing over paths in the parameter space of the reduced model subject to the constraints imposed by conserved quantities. This optimization minimizes a cost functional that quantifies the rate of information loss due to model reduction, meaning that an optimal path represents a macroscopic evolution that is most compatible with the microscopic dynamics in an information-theoretic sense. A near-equilibrium linearization of this method is used to derive dissipative equations for the low-order spatial moments of ensembles of point vortices in the plane. These severely reduced models describe the late-stage evolution of isolated coherent structures in two-dimensional and geostrophic turbulence. For single-layer dynamics, they approximate the relaxation of initially distorted structures toward axisymmetric equilibrium states. For two-layer dynamics, they predict the rate of energy transfer in baroclinically perturbed structures returning to stable barotropic states. Comparisons against direct numerical simulations of the fully-resolved many-vortex dynamics validate the predictive capacity of these reduced models. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

14 pages, 1520 KiB

Open AccessArticle

On Differences between Deterministic and Stochastic Models of Chemical Reactions: Schlögl Solved with ZI-Closure

by Michail Vlysidis and Yiannis N. Kaznessis

Entropy 2018, 20(9), 678; https://doi.org/10.3390/e20090678 - 06 Sep 2018

Cited by 7 | Viewed by 5419

Abstract

Deterministic and stochastic models of chemical reaction kinetics can give starkly different results when the deterministic model exhibits more than one stable solution. For example, in the stochastic Schlögl model, the bimodal stationary probability distribution collapses to a unimodal distribution when the system size increases, even for kinetic constant values that result in two distinct stable solutions in the deterministic Schlögl model. Using zero-information (ZI) closure scheme, an algorithm for solving chemical master equations, we compute stationary probability distributions for varying system sizes of the Schlögl model. With ZI-closure, system sizes can be studied that have been previously unattainable by stochastic simulation algorithms. We observe and quantify paradoxical discrepancies between stochastic and deterministic models and explain this behavior by postulating that the entropy of non-equilibrium steady states (NESS) is maximum. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

99 pages, 6860 KiB

Open AccessArticle

Model Error, Information Barriers, State Estimation and Prediction in Complex Multiscale Systems

by Andrew J. Majda and Nan Chen

Entropy 2018, 20(9), 644; https://doi.org/10.3390/e20090644 - 28 Aug 2018

Cited by 42 | Viewed by 4042

Abstract

Complex multiscale systems are ubiquitous in many areas. This research expository article discusses the development and applications of a recent information-theoretic framework as well as novel reduced-order nonlinear modeling strategies for understanding and predicting complex multiscale systems. The topics include the basic mathematical properties and qualitative features of complex multiscale systems, statistical prediction and uncertainty quantification, state estimation or data assimilation, and coping with the inevitable model errors in approximating such complex systems. Here, the information-theoretic framework is applied to rigorously quantify the model fidelity, model sensitivity and information barriers arising from different approximation strategies. It also succeeds in assessing the skill of filtering and predicting complex dynamical systems and overcomes the shortcomings in traditional path-wise measurements such as the failure in measuring extreme events. In addition, information theory is incorporated into a systematic data-driven nonlinear stochastic modeling framework that allows effective predictions of nonlinear intermittent time series. Finally, new efficient reduced-order nonlinear modeling strategies combined with information theory for model calibration provide skillful predictions of intermittent extreme events in spatially-extended complex dynamical systems. The contents here include the general mathematical theories, effective numerical procedures, instructive qualitative models, and concrete models from climate, atmosphere and ocean science. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

80 pages, 11447 KiB

Open AccessEditor’s ChoiceArticle

Conditional Gaussian Systems for Multiscale Nonlinear Stochastic Systems: Prediction, State Estimation and Uncertainty Quantification

by Nan Chen and Andrew J. Majda

Entropy 2018, 20(7), 509; https://doi.org/10.3390/e20070509 - 04 Jul 2018

Cited by 39 | Viewed by 5436

Abstract

A conditional Gaussian framework for understanding and predicting complex multiscale nonlinear stochastic systems is developed. Despite the conditional Gaussianity, such systems are nevertheless highly nonlinear and are able to capture the non-Gaussian features of nature. The special structure of the system allows closed analytical formulae for solving the conditional statistics and is thus computationally efficient. A rich gallery of examples of conditional Gaussian systems are illustrated here, which includes data-driven physics-constrained nonlinear stochastic models, stochastically coupled reaction–diffusion models in neuroscience and ecology, and large-scale dynamical models in turbulence, fluids and geophysical flows. Making use of the conditional Gaussian structure, efficient statistically accurate algorithms involving a novel hybrid strategy for different subspaces, a judicious block decomposition and statistical symmetry are developed for solving the Fokker–Planck equation in large dimensions. The conditional Gaussian framework is also applied to develop extremely cheap multiscale data assimilation schemes, such as the stochastic superparameterization, which use particle filters to capture the non-Gaussian statistics on the large-scale part whose dimension is small whereas the statistics of the small-scale part are conditional Gaussian given the large-scale part. Other topics of the conditional Gaussian systems studied here include designing new parameter estimation schemes and understanding model errors. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

20 pages, 683 KiB

Open AccessArticle

Dynamics Analysis of a Nonlinear Stochastic SEIR Epidemic System with Varying Population Size

by Xiaofeng Han, Fei Li and Xinzhu Meng

Entropy 2018, 20(5), 376; https://doi.org/10.3390/e20050376 - 17 May 2018

Cited by 12 | Viewed by 3396

Abstract

This paper considers a stochastic susceptible exposed infectious recovered (SEIR) epidemic model with varying population size and vaccination. We aim to study the global dynamics of the reduced nonlinear stochastic proportional differential system. We first investigate the existence and uniqueness of global positive solution of the stochastic system. Then the sufficient conditions for the extinction and permanence in mean of the infectious disease are obtained. Furthermore, we prove that the solution of the stochastic system has a unique ergodic stationary distribution under appropriate conditions. Finally, the discussion and numerical simulation are given to demonstrate the obtained results. Full article

(This article belongs to the Special Issue Information Theory and Stochastics for Multiscale Nonlinear Systems)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Information Theory and Stochastics for Multiscale Nonlinear Systems

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (11 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI