Practicum Abstracts

I continued my previous practicum work using lidar data of wind velocities in the NY Bight to understand the atmospheric mechanisms that cause low level jets in this region. This involved frequency analysis of the raw data, and synthesizing various conceptual models to explain the jets. I am currently wrapping up a small follow-on study to understand impacts of an idealized jet from this conceptual model on a wind turbine.

Coarse-grained models typically rely on closure models to account for unresolved scales. For instance, large eddy simulation for modeling turbulent fluid flows requires modeling closure terms to account for the unresolved sub-filter scales. With the vast amount of data available from high-fidelity simulations, there are unique opportunities to leverage data-driven modeling techniques for closure models. Despite their flexibility, data-driven models struggle in domain shift settings, i.e. when deployed in configurations not captured in the training dataset. In particular, the efficacy of neural network surrogates is difficult to assess a priori due to the deterministic, point-estimate nature of predictions. In high-consequence applications, such models require reliable uncertainty estimates in the data-informed and out-of-distribution regimes. To quantify uncertainties in both regimes, this project employed Bayesian neural networks which are able to capture both epistemic and aleatoric uncertainties. The long term goal of this project is to propagate these uncertainties through a large eddy simulation to characterize the distribution of relevant quantities of interest, e.g. ignition time.

I investigated unstable vibration behavior of wind turbine blades subject to high wind speeds outside of normal operating conditions. I developed a framework to recreate the vibration behavior with 2D cross section simulations enabling a reduction in required computational resources compared to full 3D coupled fluid-structure interaction simulations.

The project was to develop a kinetic Monte Carlo (kMC) model for modeling surface growth of inorganic perovskites for solar cell applications. It was part of an overall multiscale effort called AMAR (Adaptive mesh and algorithmic refinement), where we were trying to model a chemical vapor deposition process using communication between the kMC model and a continuum model. Developing a kMC model from bottom-up required: 1) careful selection and benchmarking of force fields against ab-initio calculations, 2) enumeration of surface events (adsorption, desorption, diffusion, etc), 3) calculation of event rates using nudged elastic band, and 4) implementation of kMC algorithm with SPPARKS.

The recursive randomized butterfly (RBT) matrix represents a well-studied invertible linear transform and has computational properties similar to the fast Fourier transform (FFT). When applied to a matrix, the RBT has a "mixing effect" that can eliminate the presence of zero-elements along the diagonal; consequently, a RBT applied to both sides of a linear equation Ax=b can eliminate the need for pivoting when performing the linear solve with high probability. Unfortunately, the RBT introduces fill-in when applied to a sparse matrix. Prior works have shown that partial application of the RBT leads to a controlled amount of fill-in, allowing us to reap the computational benefit of avoiding pivoting. The goal of this project is to implement the partial RBT transform as as the initial step of an ILDL preconditioner before applying FGMRES on a large sparse linear system. To achieve good performance, the goal was to implement the SRBT and the preconditioner in CUDA, with LLNL HYPRE FGMRES as the backend solver.

I worked on a dimension reduction method for chemical combustion simulations that utilized manifold learning and artificial neural networks. I helped to implement this method in the PeleLMeX code (built on the AMReX framework), and explored the accuracy and performance of the method and the general impact of using it in PeleLMeX simulations.

This project focused on improving routines within the BerkeleyGW code base, which is a tool to compute electron excited-state properties based on many-body perturbation theory. Work was done specifically on the Sigma executable to improve non blocking communication routines for MPI and transfer select subroutines to run on GPU.

We applied decomposition methods, such as the alternative direction method of multipliers and progressive hedging, towards solving long duration energy storage problems on high-performance computers.

Visualizing combustion models in 3D is a difficult problem for may reasons. One of which is that the spacial and time scales of the problem are very large; fluid flow into the domain is slow relative to the diffusion of oxidizers and fuel around the flame surface, creating the chemical reaction that produces the heat, which also diffuses at a different time scale. Therefore, it is difficult to parse the causality relationships between the shape of a flame, the temperature field around a flame, the concentrations of fuel (H2 gas) and oxidizer (O2 gas), and the rate of combustion of a given region of flame. The goal of the project it to understand why certain regions in the modeled combustion that had a relatively "flat" local curvature combusted faster than a 1D model of a perfectly flat flame (the basis of comparison). It was of interest to understand why factors just outside the local region of a flame would affect the flame speed if, theoretically, the flat flame is the same as the theoretical 1D model. The models were all run already, and so I worked in the analysis/visualization pipeline. My project aim was to figure out a way to display the dynamics of flame in terms of local fragments and their semi-local environmental factors:nearby curvature, temperature gradient, and concentrations of various chemical components.

I worked on an in silico analysis of the proteins related to halotolerance in algae strains, using the publicly available proteomes hosted by JGI. The purpose of this project was to identify targets for bioengineering algal strains with increased halotolerance for outdoor cultivation. After completing the initial exploratory analysis and identifying a transcription factor protein of interest, I created a pipeline for orthology inference across diverse algal species (91 in total). This pipeline takes proteins identified, either through empirical molecular studies or in silico analysis, and finds their orthologs across as many algal species as desired.

Large eddy simulation (LES) offers a path towards reducing the computational burden of direct numerical simulations (DNS) by resolving larger length scales and modeling the smallest scales. However, this filtering approach requires modeling closure terms to account for the effects at the sub-filter scale (SFS). Many closure model forms have been posited based on effects such as the ratio of time or length scales, however with the vast amounts of data available from DNS, there are unique opportunities to leverage data-driven modeling techniques. Albeit flexible, data-driven models still depend on the dataset and the functional form of the model chosen. Disseminating such models requires reliable uncertainty estimates in the data-informed and out-of-distribution regimes. To quantify uncertainties in both regimes, I employed Bayesian neural networks and demonstrated their ability to capture both epistemic and aleatoric uncertainties. Here, the focus is on modeling the progress variable scalar dissipation rate which plays a key role in the modeling of turbulent premixed flames. I developed and demonstrated various methods for incorporating prior knowledge into the resultant model. A pathway towards uncertainty propagation through the LES using this class of models was also outlined.

(1) We aimed to uncover the underlying atmospheric physics mechanism by which low-level jets in the US Mid-Atlantic offshore environment form. Our findings, which involved analysis of lidar data and running the Weather Research and Forecasting Model, indicate that the low-level windspeed maxima off the coast of New Jersey form due to a combination of frictional decoupling of warmer air over a cold sea surface, as well as strong thermal gradients induced at the start of fronts. (2) As a second project I am looking at the impact of low level jets on turbine performance. In particular I am studying the impact of a time-varying jet nose height on the turbine loading and fatigue, using the NREL OpenFAST code as well as large-eddy simulations.

Model Predictive Control (MPC) is a popular approach to determine the optimal building control for objectives, such as minimizing cost and greenhouse gas emissions. How effectively MPC achieves these objectives is limited by the computational cost of using large prediction horizons. We use an approach inspired by Q-learning to learn a terminal penalty function, Q, for MPC, that corrects for the limited prediction horizon. By implementing MPC as a differentiable convex neural network layer and choosing a convex Q, we can optimize the weights of Q by repeatedly solving the control problem and applying gradient descent. We demonstrate MPC with Q-learning (MPC-Q) on a five-zone building model, verifying that the learned Q provides good performance for new scenarios and showing that we can achieve accurate control for MPC-Q with a significantly shorter horizon than for MPC alone.

Deconstruction of polyethylene terephthalate is crucial for addressing global-scale plastics pollution. A recently-discovered two-enzyme system, composed of PETase and MHETase, synergistically decomposes the synthetic polymer into its constituent building blocks, offering a promising route to recycling plastic materials that continue to accumulate in landfills. Yet, the low thermostability of MHETase poses an inefficiency in the recycling process, and the search for more thermostable variants of the enzyme continues. In this work, we use molecular dynamics simulations to understand the dynamical processes that lead to an overall thermostabilization of the MHETase enzyme. We analyze variants of MHETase that are predicted by unsupervised learning models to be more thermostable than the parent enzyme, resulting in the identification of 13 variants in total. Further analyses of these variants suggest that mutations which maximize the polar surface area lead to an overall increase in the thermostability of the enzyme, offering a design route to improve the stability of other enzymes.

The GW approximation is a highly accurate ab-initio quantum mechanical technique for understanding the electronic structure of solids and molecules. The evaluation of the polarizability within the GW approximation requires a double summation over occupied/virtual state pairs. Thus, information regarding which individual quantum states contribute to the polarizability is encoded in the computation of the quantity. The relative importance of a particular state to the polarizability can be ascertained by examining how the presence or absence of the state affects the computed value of the polarizability. Implementing a mechanism to examine the relative importance of any arbitrary set of states in the calculation of the inverse dielectric and polarizability matrices in the BerkeleyGW package was the purpose of the practicum project. These matrices can be used to obtain properties such as macroscopic dielectric functions and electron energy loss spectra (EELS), and can be used as input for GW calculations of electronic properties and GW-BSE calculations of optical properties. For example, we were able to compute the simulated EELS spectrum of GaN while including all valence bands, including only Gallium d-bands, and including no Gallium d-bands in the calculation to study how the plasmon peak in the spectrum evolves under these changes. This allowed us to systematically examine the contribution of the Gallium d-bands to the GaN plasmon mode.

Understanding the properties and function of materials of interest for energy research requires modeling large, hierarchical systems in atomistic detail. Currently, we do not have computational methods that can handle both this scale and fidelity. Ab initio molecular dynamics (AIMD) can describe materials chemistry (including polarizability and the making/breaking of bonds) because the system is evolved in time according to forces solved quantum mechanically (plane-wave DFT). This is crucial for characterizing material properties such as catalysis and the complex chemistry at interfaces. However, AIMD simulations are costly and therefore severely limited by system size and timescale of the dynamics. Classical MD, on the other hand, can model much larger systems on longer timescales. Because MD evolves according to a parametrized force field, however, it cannot describe complex, changing chemical phenomena. A new method that combines the speed of classical MD with the fidelity of AIMD is necessary. We are developing chemically aware force fields for modeling materials dynamics using machine learning (ML) techniques. Our first goal is to predict atomic charges in hybrid perovskite solar cell (HPSC) materials from descriptors (bispectrum coefficients) which characterize the environment around each atom. The descriptors can be readily calculated for a given configuration, while the true atomic charges are obtained via an expensive AIMD calculation. We explore several supervised learning techniques to predict the atomic charges, including support vector regression and neural networks.

In 2018 the DOE launched the ATHENA project: Advancing Transportation Hubs’ Efficiency Using Novel Analytics. The goal of this project is to build a model of airports' traffic (both passenger and freight, and across both air and land) that will allow possible changes and optimizations to be simulated. This will allow long-term planning to take into account the impact of certain decisions, potentially reducing energy consumption and cost. When I joined the team in June 2019 modeling and simulations of the Dallas/Fort Worth International Airport was already underway. For my project, I built customized visualizations of model performance and well as airport simulation results. We were immediately able to draw insights from such visualizations and improve model and simulation accuracy.

With the rising penetration of distributed energy resources, distribution system ancillary services and enabling techniques such as state estimation have become essential to distribution system operation. However, traditional state estimation techniques have difficulty coping with (1) the low-observability conditions often present on the distribution system due to the paucity of sensor measurements, and (2) the noisy nature of the measurements that do exist. To address these limitations, we propose a state estimation algorithm that employs matrix completion (a tool for estimating missing values in low rank matrices) augmented with noise-resilient power flow constraints. We empirically evaluate our method on the 33 and 39 bus IEEE test systems. We find that our method provides acceptable state estimation performance (within 5%) in low-rank regimes where traditional state estimation algorithms may not be able to operate. Our method additionally outperforms the state-of-the-art algorithm of weighted least squares with pseudo-measurements over a wide range of data availability scenarios.

The discovery of new materials for solar panels is considered one of the biggest challenges in materials science and renewable energy. There have been several unique databases that have been compiled over the years that use Density Functional Theory (DFT) to calculate a wide variety of bulk properties of crystalline compounds to help with this lofty goal. One of these properties, the bandgap, is particularly important in the context of photovoltaics. However, DFT calculations are very computationally expensive. In this project, we utilized a new flavor of neural networks built to operate on graphs to predict the bandgap of crystals using just the structure of that crystal and the properties of the atoms in that crystal. We applied our model to the NREL Materials Database and obtained a promising root-mean-square error of 0.21 eV and calculation times orders of magnitudes lower than DFT calculated bandgaps. Furthermore, we explore the failure modes of our model and where improvements could be made. This work could be used in the future on experimental datasets to explore new crystal structures and help with material discovery for solar cells.

In the future, it is likely that our houses will adjust when our heaters, refridgerators, and electrical vehicles are active in reponse to real-time electrical prices. However, if a large group of houses make the same decisions, insufficient power may be available, and the power grid overloaded. The purpose of this project was to develop and simulate distributed algorithms which maximize individual benefits while minimizing such negatives outcomes, which may emerge from consumers' collective behavior. As discussed in various responses below, the foregoing project did not succeed as planned. As a fallback, I developed new modules for GridLAB-D, which is the PNNL software underlying NREL's grid simulation software. This entailed developing an understanding of cosimulation, discrete event modeling, and discrete-continuous time hybrid models.

As supercomputers reach the exascale, power consumption is quickly becoming a limiting factor. Requiring leadership-class HPC systems to have dedicated power plants is clearly not a sustainable path. Smaller supercomputing facilities and data centers are also affected by power constraints, namely through surcharges due to exceedingly high peak power draws. To better understand power use in HPC workloads, the National Renewable Energy Laboratory (NREL) supercomputing facility has tracked the power used by each node in 10 second intervals over the course of the past year as well as a record of all jobs submitted during this time period and their metadata. Using these data, we have performed cluster analysis and dimension reduction with the goal of understanding typical power characteristics and patterns across a wide variety of applications. This analysis informs our efforts to predict, a priori, the power that will be used by a given job. These power predictions are integral for designing new power-aware schedulers and workload managers. To evaluate the efficacy of these predictions in practice, we utilize simulations of the supercomputing job schedule to determine if peak power use could be reduced. We found that with a minimal delay in mean job wait times the overall variability in total system power use can be mitigated.

Concentrated solar power (CSP) is a form of renewable energy. Unlike photovoltaic cells that most people are familiar with, this form of solar energy is based on focusing sunlight using specialized mirrors called heliostats; the sunlight heats water that powers a steam turbine. At the National Renewable Energy Laboratory, the PREDICTS team is developing a way to simulated the chemistry and physics of heliostats to understand the degradation of these mirrors with time.

We used Car-Parrinello Molecular Dynamics (CPMD) to study the reaction pathways involved in creation of lignin, one of the primary components of plant biomass. CPMD allows investigation of time-dependent phenomena while still explicitly treating electrons. It can thus be used for applications in which electrons are important, such as bond breaking and formation, which cannot be done with classical molecular dynamics.

The overarching project is learning how to degrade cellulose into simpler sugars suitable for making biofuels. Specifically, we were looking at the impact of cellulose oxidation on the decrystallization and binding affinity of cellulose. Four different oxidized products were studied, two each from the reducing and non-reducing ends, using a native-contact reaction coordinate to describe the decrystallization progression and the resulting free-energy profile. The binding affinities for the oxidized species to two different cellulose-degrading enzymes was calculated via thermodynamic integration calculations.