3.12. Coupling

In complex simulation, features may be required that are outside the capabilities of Aria. In these cases, coupling to another application is required to combine functionality. Examples of this could include

  • A thermo-mechanical simulation, where displacement is solved by Adagio while temperature is solved by Aria.

  • A reentry vehicle in flight, where the fluid domain is solved by Sparc while the solid domain is solved by Aria.

The following sections discuss the concept and implications behind such coupling approaches

3.12.1. One Way Coupling

Assume some “leader” application has state vector \vector{\phi}_1, and its “follower” application has state vector \vector{\phi}_2. In a one way coupling, the solution of the leader app is not a function of the follower app (or its influence can be neglected or approximated offline). In this case, the system of equations is

\frac{\partial \vector{\phi}_1}{\partial t} & = F_1 \left( \vector{\phi}_1 , t \right) \\ \frac{\partial \vector{\phi}_2}{\partial t} & = F_2 \left( \vector{\phi}_1 , \vector{\phi}_2 , t \right)

Because the leader does not depend on the follower, one needs only to run the leader first and record output over the simulation time. By interpolating these recorded results over time in the follower simulation, both results can be obtained. With some interpolant \vector{\phi}'_1(t) acting on these recorded results, the system of the follower takes the form

\frac{\partial \vector{\phi}_2^{n+1}}{\partial t} = F_2 \left( \vector{\phi}_1'(t^{n+1}) , \vector{\phi}_2^{n+1} , t^{n+1} \right)

The benefit of this approach is that applications can run on their own timescales, and need not adhere to the same MPMD coupling pattern. See File Based Coupling for more information on how one can setup a case like this.

3.12.2. Two Way Coupling

Consider now a leader/follower system where their equations now implicitly depend on one another over some coupled subdomain \Omega_c. Unlike the One Way Coupling, the solutions to both problems must now be advanced together

\frac{\partial \vector{\phi}_1}{\partial t} & = F_1 \left( \vector{\phi}_1 , \vector{\phi}_2 |_{\Omega_c} , t \right) \\ \frac{\partial \vector{\phi}_2}{\partial t} & = F_2 \left( \vector{\phi}_1 |_{\Omega_c} , \vector{\phi}_2 , t \right)

In a two way coupled simulation, the leader first takes a step with the lagged values of the follower

\frac{\partial \vector{\phi}_1^{n+1}}{\partial t} = F_1 \left( \vector{\phi}_1^{n+1} , \vector{\phi}_2^{n} |_{\Omega_c} , t^{n+1} \right)

This updated \vector{\phi}_1 information is then communicated to the follower, which can then use the new solution to update its own state

\frac{\partial \vector{\phi}_2^{n+1}}{\partial t} = F_2 \left( \vector{\phi}_1^{n+1} |_{\Omega_c} , \vector{\phi}_2^{n+1} , t^{n+1} \right)

In practice, the communication phase is implemented in differing forms depending on the coupling type:

Alg. 3.4 Explicit strategy for two way coupling

\begin{algorithmic}[1] \STATE Leader \& Follower initialize $\vector{\phi}_1^0$/$\vector{\phi}_2^0$ \STATE Follower sends $\vector{\phi}_2^0 |_{\Omega_c}$ to Leader \WHILE{$t < t_{end}$} \STATE Negotiate $\Delta t^{n+1}$ \STATE Leader updates $\vector{\phi}_1^{n+1}$ given $F_1 \left( \vector{\phi}_1^{n+1} , \vector{\phi}_2^{n} , t^{n+1} \right)$ \STATE Leader sends $\vector{\phi}_1^{n+1} |_{\Omega_c}$ to Follower \STATE Follower updates $\vector{\phi}_2^{n+1}$ given $F_2\left( \vector{\phi}_1^{n+1} , \vector{\phi}_2^{n+1} , t^{n+1} \right)$ \STATE Follower sends $\vector{\phi}_2^{n+1} |_{\Omega_c}$ to Leader \STATE Print results \ENDWHILE \end{algorithmic}

See Solution Strategy for more details on how Aria updates its solutions.

3.12.2.1. Implicit

This explicit coupling strategy can lead to convergence issues, particularly if the coupling between leader and follower is strong or if the coupling quantities significantly influence the behavior of each system. Since the systems are solved sequentially, there is a lag in the response of each system to changes in the other, restricting the accuracy of the approach. It can also lead to numerical instability, as perturbation in the solution could cause further divergence from the true solution. To address potential convergence issues, an implicit coupling approach can be used instead. In this case, the update step is repeated until the solutions of both apps stop changing. While this increases the computation cost, it ensure convergence in strongly coupled systems and can often allow for larger step sizes.

Alg. 3.5 Implicit strategy for two way coupling

\begin{algorithmic}[1] \STATE Leader \& Follower initialize $\vector{\phi}_1^0$/$\vector{\phi}_2^0$ \STATE Follower sends $\vector{\phi}_2^0 |_{\Omega_c}$ to Leader \WHILE{$t < t_{end}$} \STATE Negotiate $\Delta t^{n+1}$ \STATE Initialize $\vector{\phi}^*_{i}$ \WHILE{Check Convergence with $F_1\left(\vector{\phi}_1,\vector{\phi}_1^*\right)$ and $F_2\left(\vector{\phi}_2,\vector{\phi}_2^*\right)$} \STATE Leader updates $\vector{\phi}_1^{n+1}$ given $F_1\left( \vector{\phi}_1^{n+1} , \vector{\phi}_2^* , t^{n+1} \right)$ \STATE Leader sends $\vector{\phi}_1^{n+1} |_{\Omega_c}$ to Follower \STATE Follower updates $\vector{\phi}_2^{n+1}$ given $F_2\left( \vector{\phi}_1^* , \vector{\phi}_2^{n+1} , t^{n+1} \right)$ \STATE Follower sends $\vector{\phi}_2^{n+1} |_{\Omega_c}$ to Leader \STATE Record $\vector{\phi}_{i} \to \vector{\phi}^*_{i}$ \ENDWHILE \STATE Print results \ENDWHILE \end{algorithmic}

Convergence is determined based on the change in the solution vector between successive implicit iterations. The simulation is considered to have converged when this difference falls below a given tolerance for both applications, indicating that subsequent iterations yield only negligible changes in the solution vector. This tolerance is chosen based on the desired accuracy of the simulation and the magnitude of the DOFs, and can be given either as an absolute or relative tolerance. Aria supports implicit tolerances \epsilon_N for a set of different order N norms, which return converged when

\frac{| \vector{\phi}_i - \vector{\phi}^*_i |_N}{S} \le \epsilon_N

where S is defined as

No Scaling

S=1

DOF_AMAX

S=| \vector{\phi}_i |_\infty

DOF_RMS

S=\sqrt{ \left( | \vector{\phi}_{i} |_2 \right)^2 / n_\text{nodes} }

3.12.2.2. Subcycling

Depending on the phenomena being modeled, there could be a difference in timescales between the two coupled apps. An explicit/implicit coupling would require that both apps step at the smaller of the two timescales, leading to unnecessary work for the app with the larger timescales. To overcome this, subcycling can be used. In this case, one app is permitted to take multiple smaller steps to reach the desired “sync step” when results are synced between the two apps. This can significantly alleviate computational load, at the expense of reducing the feedback between apps.

Alg. 3.6 Subcycling strategy for two way coupling

\begin{algorithmic}[1] \STATE Leader \& Follower initialize $\vector{\phi}_1^0$/$\vector{\phi}_2^0$ \STATE Follower sends $\vector{\phi}_2^0 |_{\Omega_c}$ to Leader \WHILE{$t < t_{end}$} \STATE Negotiate sync step $\Delta t^{n+1}$ \FOR{Each Leader substep (or single step if not subcycling)} \STATE Leader calcutes its own $\Delta t^*$ \STATE Leader updates $\vector{\phi}_1^*$ given $F_1 \left( \vector{\phi}_1^{*} , \vector{\phi}_2^{n} , t^{*}_1 \right)$ \ENDFOR \STATE Leader records $\vector{\phi}_1^* \to \vector{\phi}_1^{n+1}$ \STATE Leader sends $\vector{\phi}_1^{n+1} |_{\Omega_c}$ to Follower \FOR{Each Follower substep (or single step if not subcycling)} \STATE Follower updates $\vector{\phi}_2^*$ given $F_2\left( \vector{\phi}_1^{n+1} , \vector{\phi}_2^{*} , t^{*}_2 \right)$ \ENDFOR \STATE Follower records $\vector{\phi}_2^* \to \vector{\phi}_2^{n+1}$ \STATE Follower sends $\vector{\phi}_2^{n+1} |_{\Omega_c}$ to Leader \STATE Print results \ENDWHILE \end{algorithmic}

This is accomplished using either Solution Control for a Sierra Coupling, or setting the MPMD timestep mode for an MPMD Coupling.