Application resilience is a key challenge that has to be addressed to realize the exascale vision. Online recovery, even when it involves all processes, can dramatically reduce the overhead of failures as compared to the more traditional approach where the job is terminated and restarted from the last checkpoint. In this paper we explore how local recovery can be used for certain classes of applications to further reduce overheads due to resilience. Specifically we develop programming support and scalable runtime mechanisms to enable online and transparent local recovery for stencil-based parallel applications on current leadership class systems. We also show how multiple independent failures can be masked to effectively reduce the impact on the total time to solution. We integrate these mechanisms with the S3D combustion simulation, and experimentally demonstrate (using the Titan Cray-XK7 system at ORNL) the ability to tolerate high failure rates (i.e., node failures every 5 seconds) with low overhead while sustaining performance, at scales up to 262144 cores.
Computing distance fields is fundamental to many scientific and engineering applications. Distance fields can be used to direct analysis and reduce data. In this paper, we present a highly scalable method for computing 3D distance fields on massively parallel distributed-memory machines. A new distributed spatial data structure, named parallel distance tree, is introduced to manage the level sets of data and facilitate surface tracking over time, resulting in significantly reduced computation and communication costs for calculating the distance to the surface of interest from any spatial locations. Our method supports several data types and distance metrics from real-world applications. We demonstrate its efficiency and scalability on state-of-the-art supercomputers using both large-scale volume datasets and surface models. We also demonstrate in-situ distance field computation on dynamic turbulent flame surfaces for a petascale combustion simulation. Our work greatly extends the usability of distance fields for demanding applications.
This report provides in-depth information and analysis to help create a technical road map for developing next-generation programming models and runtime systems that support Advanced Simulation and Computing (ASC) work- load requirements. The focus herein is on asynchronous many-task (AMT) model and runtime systems, which are of great interest in the context of "Oriascale7 computing, as they hold the promise to address key issues associated with future extreme-scale computer architectures. This report includes a thorough qualitative and quantitative examination of three best-of-class AIM] runtime systems – Charm-++, Legion, and Uintah, all of which are in use as part of the Centers. The studies focus on each of the runtimes' programmability, performance, and mutability. Through the experiments and analysis presented, several overarching Predictive Science Academic Alliance Program II (PSAAP-II) Asc findings emerge. From a performance perspective, AIV runtimes show tremendous potential for addressing extreme- scale challenges. Empirical studies show an AM runtime can mitigate performance heterogeneity inherent to the machine itself and that Message Passing Interface (MP1) and AM11runtimes perform comparably under balanced conditions. From a programmability and mutability perspective however, none of the runtimes in this study are currently ready for use in developing production-ready Sandia ASC applications. The report concludes by recommending a co- design path forward, wherein application, programming model, and runtime system developers work together to define requirements and solutions. Such a requirements-driven co-design approach benefits the community as a whole, with widespread community engagement mitigating risk for both application developers developers. and high-performance computing runtime systein
The one-dimensional turbulence (ODT) model is applied to a reactant-to-product counterflow configuration and results are compared with DNS data. The model employed herein solves conservation equations for momentum, energy, and species on a one dimensional (1D) domain corresponding to the line spanning the domain between nozzle orifice centers. The effects of turbulent mixing are modeled via a stochastic process, while the Kolmogorov and reactive length and time scales are explicitly resolved and a detailed chemical kinetic mechanism is used. Comparisons between model and DNS results for spatial mean and root-mean-square (RMS) velocity, temperature, and major and minor species profiles are shown. The ODT approach shows qualitatively and quantitatively reasonable agreement with the DNS data. Scatter plots and statistics conditioned on temperature are also compared for heat release rate and all species. ODT is able to capture the range of results depicted by DNS. However, conditional statistics show signs of underignition.
Application resilience is a key challenge that must be ad-dressed in order to realize the exascale vision. Previous work has shown that online recovery, even when done in a global manner (i.e., involving all processes), can dramatically re-duce the overhead of failures when compared to the more traditional approach of terminating the job and restarting it from the last stored checkpoint. In this paper we suggest going one step further, and explore how local recovery can be used for certain classes of applications to reduce the over-heads due to failures. Specifically we study the feasibility of local recovery for stencil-based parallel applications and we show how multiple independent failures can be masked to effectively reduce the impact on the total time to solution.
This paper reports the results of a joint experimental and numerical study of the flow characteristics and flame structure of a hydrogen rich jet injected normal to a turbulent, vitiated crossflow of lean methane combustion products. Simultaneous high-speed stereoscopic PIV and OH PLIF measurements were obtained and analyzed alongside three-dimensional direct numerical simulations of inert and reacting JICF with detailed H2/CO chemistry. Both the experiment and the simulation reveal that, contrary to most previous studies of reacting JICF stabilized in low-to-moderate temperature air crossflow, the present conditions lead to a burner-attached flame that initiates uniformly around the burner edge. Significant asymmetry is observed, however, between the reaction zones located on the windward and leeward sides of the jet, due to the substantially different scalar dissipation rates. The windward reaction zone is much thinner in the near field, while also exhibiting significantly higher local and global heat release than the much broader reaction zone found on the leeward side of the jet. The unsteady dynamics of the windward shear layer, which largely control the important jet/crossflow mixing processes in that region, are explored in order to elucidate the important flow stability implications arising in the inert and reacting JICF. The paper concludes with an analysis of the ignition, flame characteristics, and global structure of the burner-attached flame. Chemical explosive mode analysis (CEMA) shows that the entire windward shear layer, and a large region on the leeward side of the jet, are highly explosive prior to ignition and are dominated by non-premixed flame structures after ignition. The predominantly mixing limited nature of the flow after ignition is examined by computing the Takeno flame index, which shows that ~70% of the heat release occurs in non-premixed regions.
Three-dimensional direct numerical simulation results of a transverse syngas fuel jet in turbulent cross-flow of air are analyzed to study the influence of varying volume fractions of CO relative to H2 in the fuel composition on the near field flame stabilization. The mean flame stabilizes at a similar location for CO-lean and CO-rich cases despite the trend suggested by their laminar flame speed, which is higher for the CO-lean condition. To identify local mixtures having favorable mixture conditions for flame stabilization, explosive zones are defined using a chemical explosive mode timescale. The explosive zones related to flame stabilization are located in relatively low velocity regions. The explosive zones are characterized by excess hydrogen transported solely by differential diffusion, in the absence of intense turbulent mixing or scalar dissipation rate. The conditional averages show that differential diffusion is negatively correlated with turbulent mixing. Moreover, the local turbulent Reynolds number is insufficient to estimate the magnitude of the differential diffusion effect. Alternatively, the Karlovitz number provides a better indicator of the importance of differential diffusion. A comparison of the variations of differential diffusion, turbulent mixing, heat release rate and probability of encountering explosive zones demonstrates that differential diffusion predominantly plays an important role for mixture preparation and initiation of chemical reactions, closely followed by intense chemical reactions sustained by sufficient downstream turbulent mixing. The mechanism by which differential diffusion contributes to mixture preparation is investigated using the Takeno Flame Index. The mean Flame Index, based on the combined fuel species, shows that the overall extent of premixing is not intense in the upstream regions. However, the Flame Index computed based on individual contribution of H2 or CO species reveals that hydrogen contributes significantly to premixing, particularly in explosive zones in the upstream leeward region, i.e. at the preferred flame stabilization location. Therefore, a small amount of H2 diffuses much faster than CO, creating relatively homogeneous mixture pockets depending on the competition with turbulent mixing. These pockets, together with high H2 reactivity, contribute to stabilizing the flame at a consistent location regardless of the CO concentration in the fuel for the present range of DNS conditions.
This paper presents the results of DNS of a partially premixed turbulent syngas/air flame at atmospheric pressure. The objective was to assess the importance and possible effects of molecular transport on flame behavior and structure. To this purpose DNS were performed at with two proprietary DNS codes and with three different molecular diffusion transport models: fully multi-component, mixture averaged, and imposing the Lewis number of all species to be unity. Results indicate that At the Reynolds numbers of the simulations (Returb = 600, Re = 8000) choice of molecular diffusion models affects significantly the temperature and concentration fields;Assuming Le = 1 for all species predicts temperatures up to 250 K higher than the physically realistic multi-component model;Faster molecular transport of lighter species changes the local concentration field and affects reaction pathways and chemical kinetics. A possible explanation for these observations is provided in terms of species diffusion velocity that is a strong function of gradients: thus, at sufficiently large Reynolds numbers, gradients and their effects tend to be large. The preliminary conclusion from these simulations seems to indicate molecular diffusion as the third important mechanism active in flames besides convective transport and kinetics. If confirmed by further DNS and measurements, molecular transport in high intensity turbulent flames will have to be realistically modeled to accurately predict emissions (gaseous and particulates) and other combustor performance metrics.
