4.14.1. User Subroutines

Warning

The use of user subroutines should be considered a last resort, especially for production simulations - most custom capabilities can be implemented using GPU-compatible, well-tested capabilities like string functions.

Fuego supports the inclusion of user subroutines for property evaluation, boundary conditions, and initial conditions. It is worth noting that simulations that use user subroutines will not run fully on the GPU on platforms where they are available and there may be a significant performance penalty relative to using string functions in those cases.

Fuego offers support of C and Fortran user subroutines through fixed-interface signatures. It is important to note that C subroutines provide some level of variable-type safety and bounds checking that is not supported with Fortran subroutines.

The C subroutine API is described in the following sections - Fortran is left as an exercise for the reader.

4.14.1.1. Building Usersubs

The process for building user subroutines is the same for C and Fortran subroutines. When you invoke the sierra --make command, the specified input file is scanned for imported shared object (.so) files and they are compiled from source files of the same name.

sierra --make fuego -i input.i

Note

The shared object generally does not have to be re-compiled between Fuego runs unless you change the contents of the C or F file, however it is a good practice to re-compile them when you change what version of Fuego you are running.

Subroutines can also be compiled using the buildsub command, which should produce an equivalent output

buildsub -e fuego -i input.i

4.14.1.2. Property Subroutines

A C user subroutine must be a text file of suffix .C that will be compiled to a shared object file of the same name with the .so extension. For example, a file named mysubs.C would be compiled to a file named mysubs.so and referenced from the Fuego input file using the line command

load user plugin file ./mysubs.so USING FUNCTION register_usersubs

Note that, since mysubs.so is a filename, its input file name is case-sensitive, and considerations must be made concerning its location path.

An example C file to provide a subroutine for enthalpy is shown below.

#include "Afgo_UserSubSig.h"

extern "C" void enthalpy_prop(
  const int * num_results,       // result size (usually 1 or num_species)
  double * result,               // result data
  const int * num_inputs,        // number of inputs (e.g. "2" for "temperature mass_fraction" inputs)
  const unsigned int * inp_size, // size of each input (e.g. [1, num_species] for example above)
  const int * num_input_data,    // total inputs size (e.g. 1+num_species for example above)
  const double * input_data,     // flattened input data (e.g. [T, Y_0, Y_1, Y_2] for example above)
  const int * num_user_params,   // size of user supplied data (1 here)
  const double * user_params     // user supplied data (data = [Cp])
)
{
  const double T = input_data[0];
  const double Cp = user_params[0];

  // h = Cp*T
  result[0] = Cp*T;
}

extern "C" void register_usersubs() {
  sierra::Afgo::AfgoPropSub::instance().
    registerFunction(std::string("enthalpy_prop"), enthalpy_prop);
}

In the code above, the actual function (named enthalpy_prop) is declared with the required signature for a property. The function is then registered for use in Fuego with the following command (which must be after the function declaration)

sierra::Afgo::AfgoPropSub::instance().registerFunction

The registration function takes two arguments:

"enthalpy_prop" - The name of the evaluator, to be referenced in the input file.

enthalpy_prop - The function being registered.

Once registered, the subroutine can be referred to in a Fuego material using the registered name:

Begin Property Specification for Fuego Material air
  # ...

  subroutine for enthalpy = enthalpy_prop
  input variables for enthalpy_prop = temperature
  real data for enthalpy_prop = 500
End

4.14.1.3. Source Subroutines

Subroutines for source terms have a different signature than for properties, and cannot take custom inputs.

An example C file to provide a subroutine for a source term for enthalpy is shown below.

#include "Afgo_UserSubSig.h"

extern "C" void enth_src(
  const int * num_results,       // result size (usually 1)
  const int * ndims,             // problem dimension (2 or 3)
  const int * nnodes,            // number of nodes per element
  const double* coords,          // 2D array of coordinates (ndims X nnodes)
  const double* vol_scv,         // 1D array of scv volumes (nnodes)
  const double* x_vel,           // 1D array of X velocity (nnodes)
  const double* y_vel,           // 1D array of Y velocity (nnodes)
  const double* z_vel,           // 1D array of Z velocity (nnodes)
  const double* density,         // 1D array of density (nnodes)
  const double* pressure,        // 1D array of pressure (nnodes)
  const double* time,            // Current time (scalar)
  double* lhs,                   // 2D LHS array (nnodes x nnodes)
  double* rhs                    // 1D RHS array (nnodes)
)
{
  for(int n = 0; n < *nnodes; ++n) {
    rhs[n] = vol_scv[n]*500;
  }
}

extern "C" void register_usersubs() {
  sierra::Afgo::FMMSFgoUserSub::instance().
    registerFunction(std::string("my_enth_src"), enth_src);
}

This is referred to in the input file in the Solution Options block with

Source term subroutine = my_enth_src for EQUATION ENTHALPY

4.14.1.4. IC/BC Subroutines

The IC and BC subroutine API is not currently documented. Use at your own peril, and contact sierra-help@sandia.gov if you need help.