4.5.9. User Utilities

Warning

The use of user utilities should be considered a last resort, especially for production simulations - most custom capabilities can be implemented using GPU-compatible, well-tested capabilities like string functions, user expressions, and multi-dimensional tables.

User utilities are another way that users can extend Aria’s capabilities using custom compiled code. Unlike a plugin expression, which is evaluated locally at different nodes or integration points in an expression graph controlled by Aria, a user utility is a standalone class with an execute function that can be executed at various points during problem execution. A plugin expression cannot generally access field data or perform MPI calls, but a user utility can do both of those things.

The tradeoff is that the API for a user utility is much more general and open, and as such cannot be completely documented here. This section will provide some guidelines on how to define them, and some example utilities. For more advanced capabilities, you will likely need to involve an Aria developer.

4.5.9.1. Writing a Utility

In this section we’ll write an example utility to calculate a custom field we will use for a boundary flux. The first part of our plugin C file will define the utility class. The API requirements for this class are:

  1. It must inherit from User_Equation_System_Utility

  2. Its constructor must use the following signature - MyClass(Equation_System & equation_system, const PrsrParamMap & params)

  3. It must define an execute function as void execute() override

  4. It may define an initialization function as void initialize() override, but this is not required

Our class definition looks like:

#include <stk_mesh/base/Selector.hpp>
#include <eq_utilities/Aria_EquationSystemUtilityFactory.h>
#include <Aria_Equation_System.h>
#include <Aria_Plugin_Expression.h>
#include <Aria_DiagWriter.h>
#include <Aria_Mesh.h>
#include <tftk_mesh/tftk_FieldFunctions.h>

using namespace sierra::Aria;

class MyCustomFluxComputer : public User_Equation_System_Utility
{
public:
  MyCustomFluxComputer(
    Equation_System & equation_system,
    const PrsrParamMap & params
  );

  ~MyCustomFluxComputer() override = default;

  void initialize() override;
  void execute() override;

private:
  double compute_flux(double tN, double tNP1, double x, double y, double z);

  Region & my_region;
  String my_flux_field_name;
  double my_update_interval;
  tftk::mesh::FieldRef my_coordinates;
  tftk::mesh::FieldRef my_flux;
};

The next part of the utility C file is the constructor definition. This is where you read in any user-supplied parameters needed for your utility and define where in the execution sequence it should be run. Common choices for where the utility should be run are:

  • RUN_INITIALLY - Run only once during initialization (before the first time step)

  • RUN_ONCE - Run only once at the first time step

  • BEGIN_TIMESTEP - Run at the start of each time step before any equation systems begin

  • PRE_NONLINEAR_SOLVE - Run before beginning nonlinear iteration at each time step

  • POST_NONLINEAR_SOLVE - Run after completing nonlinear iteration at each time step

  • PRE_ITERATE - Run before each nonlinear Newton solve iteration (before matrix assembly)

  • POST_ITERATE - Run at the end of each nonlinear Newton solve iteration

Additionally, when you define the utility in your aria input file you can pass any number of key-value pairs to it, which will be available in the PrsrParamMap object passed in here. For example, the case below is called in the Aria input file with

Add Utility MyFluxUtil dt=1 flux_field=my_flux

In the code, we can then get the values passed in for “dt” and “flux_field” and save them in our utility for later use. Additional methods are available for handling optional parameters as well.

MyCustomFluxComputer::MyCustomFluxComputer(
  Equation_System & equation_system,
  const PrsrParamMap & params
)
: User_Equation_System_Utility(
    equation_system,
    PRE_NONLINEAR_SOLVE,
    "MyCustomFluxComputer"
  )
, my_region(*(equation_system.get_region()))
{
  my_flux_field_name = get_required_parameter<String>(params, "flux_field", ParamCtx{"MyCustomFluxComputer"});
  my_update_interval = get_required_parameter<double>(params, "dt", ParamCtx{"MyCustomFluxComputer"});
}

The next components of the plugin C file are the definitions for the other functions in the utility

void MyCustomFluxComputer::initialize()
{
  arialog << "Initializing MyCustomFluxComputer..." << Diag::dendl;
  auto& mesh = my_region.mesh();
  my_coordinates = mesh.get_node_field("model_coordinates");
  my_flux = mesh.get_node_field(my_flux_field_name);
}

void MyCustomFluxComputer::execute()
{
  arialog << "MyCustomFluxComputer setting flux field" << Diag::dendl;

  const double tN = my_region.time(STATE_N);
  const double tNP1 = my_region.time(STATE_NP1);

  arialog << "  TN = " << tN << ", TNP1 = " << tNP1 << Diag::dendl;

  auto& mesh = my_region.mesh();
  auto selector = mesh.active_not_aura_selector() & stk::mesh::selectField(my_flux);

  using F = stk::mesh::NgpField<double>;
  tftk::mesh::for_each_entity_run(
    "MyFluxCalculator",
    mesh,
    selector,
    my_flux,
    {my_coordinates},
    KOKKOS_LAMBDA(stk::mesh::FastMeshIndex mi, const F& flux, const F& xyz) {
      flux(mi,0) = -compute_flux(tN, tNP1, xyz(mi,0), xyz(mi,1), xyz(mi,2));
    },
    tftk::HostSpace()
  );
}

double MyCustomFluxComputer::compute_flux(double tN, double tNP1, double x, double y, double z) {
  return tNP1;
}

Finally, we need to register the utility. This is done with the following command:

EquationSystemUtilityFactory<MyCustomFluxComputer> init_flux_computer("MyFluxUtil");

The name used here (“MyFluxUtil”) is the name you will use in the Aria input file to activate this utility. The name of the variable here (init_flux_computer) does not matter.

4.5.9.2. Compiling a Utility

User utilities are compiled to shared object (.so) files using the same procedure as User Subroutines and User Plugins. The following command will scan the input file for import .so files and compile them from source files of the same name.

sierra --make aria -i input.i

4.5.9.3. Using a Utility

In the Aria input file, the plugin is loaded using the command (assuming the plugin is defined in My_Utility_Plugin.C)

Begin Sierra My_Sierra_Job
   Load User Plugin file ./My_Utility_Plugin.so

Inside the Aria region, we activate it with the Add Utility command. The example below shows how we can define a nodal field, activate our utility to populate the field, and use the field in a flux boundary condition.

Begin Aria Region Region
  ...
  User field real node scalar my_flux on surface_1 value = 0
  Add Utility MyFluxUtil dt=1 flux_field=my_flux
  BC Flux for Energy on surface_1 = user_field name = my_flux
  ...
End

4.5.9.4. Debugging a Utility

Because of the broad, nearly unlimited access a utility provides to Aria internal code mechanics it can be difficult to write and debug a utility without access to and understanding of the broader Aria codebase, or at least other STK-based codes. If you encounter difficulties getting a utility working, we recommend contacting sierra-help@sandia.gov for assistance. Adding logging to your utility with the arialog command is recommended as a way of checking that it is functioning as expected as well.