Howard, Amanda A.; Dong, Justin; Patel, Ravi G.; Elia, Martin R.'.; Yeo, Kyongmin; Maxey, Martin; Stinis, Panos
Numerical simulations are used to study the dynamics of a developing suspension Poiseuille flow with monodispersed and bidispersed neutrally buoyant particles in a planar channel, and machine learning is applied to learn the evolving stresses of the developing suspension. The particle stresses and pressure develop on a slower time scale than the volume fraction, indicating that once the particles reach a steady volume fraction profile, they rearrange to minimize the contact pressure on each particle. Here we consider how the stress development leads to particle migration, time scales for stress development, and present a new physics-informed Galerkin neural network that allows for learning the particle stresses when direct measurements are not possible. The particle fluxes are compared with the Suspension Balance Model with good agreement. We show that when stress measurements are possible, the MOR-physics operator learning method can also capture the particle stresses.
This project applies methods in Bayesian inference and modern statistical methods to quantify the value of new experimental data, in the form of new or modified diagnostic configurations and/or experiment designs. We demonstrate experiment design methods that can be used to identify the highest priority diagnostic improvements or experimental data to obtain in order to reduce uncertainties on critical inferred experimental quantities and select the best course of action to distinguish between competing physical models. Bayesian statistics and information theory provide the foundation for developing the necessary metrics, using two high impact experimental platforms on Z as exemplars to develop and illustrate the technique. We emphasize that the general methodology is extensible to new diagnostics (provided synthetic models are available), as well as additional platforms. We also discuss initial scoping of additional applications that began development in the last year of this LDRD.
Deep operator learning has emerged as a promising tool for reduced-order modelling and PDE model discovery. Leveraging the expressive power of deep neural networks, especially in high dimensions, such methods learn the mapping between functional state variables. While proposed methods have assumed noise only in the dependent variables, experimental and numerical data for operator learning typically exhibit noise in the independent variables as well, since both variables represent signals that are subject to measurement error. In regression on scalar data, failure to account for noisy independent variables can lead to biased parameter estimates. With noisy independent variables, linear models fitted via ordinary least squares (OLS) will show attenuation bias, wherein the slope will be underestimated. In this work, we derive an analogue of attenuation bias for linear operator regression with white noise in both the independent and dependent variables, showing that the norm upper bound of the operator learned via OLS decreases with increasing noise in the independent variable. In the nonlinear setting, we computationally demonstrate underprediction of the action of the Burgers operator in the presence of noise in the independent variable. We propose error-in-variables (EiV) models for two operator regression methods, MOR-Physics and DeepONet, and demonstrate that these new models reduce bias in the presence of noisy independent variables for a variety of operator learning problems. Considering the Burgers operator in 1D and 2D, we demonstrate that EiV operator learning robustly recovers operators in high-noise regimes that defeat OLS operator learning. We also introduce an EiV model for time-evolving PDE discovery and show that OLS and EiV perform similarly in learning the Kuramoto-Sivashinsky evolution operator from corrupted data, suggesting that the effect of bias in OLS operator learning depends on the regularity of the target operator.
Second-order optimizers hold intriguing potential for deep learning, but suffer from increased cost and sensitivity to the non-convexity of the loss surface as compared to gradient-based approaches. We introduce a coordinate descent method to train deep neural networks for classification tasks that exploits global convexity of the cross-entropy loss in the weights of the linear layer. Our hybrid Newton/Gradient Descent (NGD) method is consistent with the interpretation of hidden layers as providing an adaptive basis and the linear layer as providing an optimal fit of the basis to data. By alternating between a second-order method to find globally optimal parameters for the linear layer and gradient descent to train the hidden layers, we ensure an optimal fit of the adaptive basis to data throughout training. The size of the Hessian in the second-order step scales only with the number weights in the linear layer and not the depth and width of the hidden layers; furthermore, the approach is applicable to arbitrary hidden layer architecture. Previous work applying this adaptive basis perspective to regression problems demonstrated significant improvements in accuracy at reduced training cost, and this work can be viewed as an extension of this approach to classification problems. We first prove that the resulting Hessian matrix is symmetric semi-definite, and that the Newton step realizes a global minimizer. By studying classification of manufactured two-dimensional point cloud data, we demonstrate both an improvement in validation error and a striking qualitative difference in the basis functions encoded in the hidden layer when trained using NGD. Application to image classification benchmarks for both dense and convolutional architectures reveals improved training accuracy, suggesting gains of second-order methods over gradient descent. A Tensorflow implementation of the algorithm is available at github.com/rgp62/.