COMSOL 6.3 - Surrogate Model Training

Use a study node (

) to add functionality for training a data-driven surrogate model. A surrogate model is a simpler and computationally cheaper model, which you can use to approximate the behavior of a more complex and computationally expensive model (the full finite element model). In COMSOL apps, for example, faster model evaluation using the surrogate model provides users of the apps with a more interactive user experience. You create surrogate models by training a deep neural network (DNN), for example. The surrogate model training is typically based on output data from a large parametric sweep of the model for which you want to create a surrogate model. The training of a surrogate model needs a number of inputs and outputs, where the outputs are considered as functions of the inputs, which replace the full finite element solution. A large number of data points are needed in order to fully describe how the inputs map to the outputs. The outputs can be captured using domain point probes, for example.

A straightforward parametric sweep could densely and uniformly distribute input points, but such an approach would be inefficient. Random sampling is another alternative but with inherent drawbacks such as nonuniform sampling and potential failure to cover the entire input space. Instead, a more strategic approach is to use a design of experiments (DOE) method, carefully sampling within the parameter space. The study uses Latin hypercube sampling (LHS), which is a DOE method that generates a dataset that uniformly covers the input space without requiring an excessive number of finite element computations. This property of LHS makes it an efficient method for data generation intended for the purpose of training a surrogate model.

The more data points that are available, the more accurately the surrogate model will be able to represent the actual solution. However, generating a large number of data points requires a large number of simulations to be run, so there is a tradeoff between the time it takes to generate all the data points and the desired accuracy the surrogate model.

	Tubular Reactor Surrogate Model Application: Application Library path COMSOL_Multiphysics/Applications/tubular_reactor_surrogate. If you have the Battery Design Module, see Surrogate Model Training of a Battery Rate Capability Model: Application Library path Battery_Design_Module/Applications/lib_rate_capability_surrogate.

For additional information about surrogate models in connection with uncertainty quantification, see the Uncertainty Quantification Module User’s Guide.

To add a study, right-click a node and choose . You can only have one node in each study.

Click the button (

) to initiate the surrogate model training and create a surrogate model.

If you choose from the node, the study generates and sets up a job and its subsequence node (see the Uncertainty Quantification Module User’s Guide). The job is controlled by and synchronized to the study. If you click from the node, the study identifies and runs its job if it exists; else it first creates a job.

The window for the study include the following sections:

From the list, choose one of the following options: (the default) or . will discard data from the previous run (if the same table is used) or start a new run using a new table. will add more data to an existing table, which can be useful if you realize after having examined the trained SM function that it does not seem accurate enough. The study outputs training data tables and trained surrogate models for each quantity of interest. If you choose for the surrogate model in the list below, a training data table will be the only result.

From the list, choose to create a new output table group or choose any existing table group. All tables generated by study will be grouped under this output table group.

From the list, choose (the default) to only generate training data, or choose , , , or to additionally set up surrogate models. and require a license for the Uncertainty Quantification Module.

•

For , also specify a . Choose (the default) or . If you chose , specify a value in the field (default value: 30), which terminates the increase of order for the PCE construction. You can also specify a value in the field (default: 0.5), which determines the truncation level of the polynomial basis. If you chose , the training will automatically determine the required polynomial degree needed to obtain suitable accuracy.

•

For , under , define the layers in the DNN. The first layer is always the input layer of the network and specifies the number of input features of the network, which will be automatically detected. The last layer is the output layer, and the layers in between are the hidden layers. In the column, choose for the input layer (the default), and for the hidden layers and output layer, choose (the default). In the column for the current layer, you can see its settings. You can edit the table using the buttons under the table: Use the (

), (), and (

) buttons and the fields under tables to edit the table contents. Or right-click a table cell and select , , or . The button (

) adds a new hidden layer to the list right before the output layer. Use the button (

) to clear all the hidden layers of the table and revert the input and output layers to their default. Underneath the table of layers, in the field for the hidden layers, specify the number of output features, which are the number of nodes (default: 1). Choosing the number of layers and nodes in a neural network is often an iterative process that involves a combination of knowledge about the specific problem and data, empirical testing, and a bit of trial and error.

You can also specify an activation function using the list. An activation function in a DNN defines how the weighted sum of the input is transformed into an output from a node or nodes in a layer of the network. The default activation is , for a hyperbolic tangent function, which is an S-shaped function. You can also choose for a linear (that is, no) activation function; for a rectified linear unit, an activation function defined as the positive part of its argument; for an exponential linear unit; or for a sigmoid function. The default activation, , is usually a good choice. and are less smooth than the other functions, so avoid them if the trained function later needs to be differentiated. Choosing is only useful for the last layer.

•

For , specify the definition of the function in the field. For example, a1*x1+a0 for a linear function with two parameters, a0 and a1, and one input parameter to the surrogate model, x1, defined under the section. In the table underneath, add the parameters that are used in the expression in the column. For the linear function example above, this would correspond to adding a0 and a1 in the column. In the column, choose the initial values (default: 0) for the parameters. The column can be used to rescale each parameter with its specified scale. By default there is no scaling. Use the and columns to set lower and upper bounds on the parameters. Use the (

), (

), and (

) buttons underneath the table to move and remove rows and to clear the table. Use the (

) and (

) buttons to load or save data to or from the table. You can also click the downward arrow beside the button and choose (

) to open the fullscreen window.

Select the checkbox if the surrogate models should also be trained after the training data has been generated and the surrogate models have been set up. By default, this checkbox is not selected.

In the table under , define the outputs from the surrogate models (quantities of interest). The quantities of interest can be either global or nonglobal. Global quantities of interest can only depend on global input parameters (for example, point probe expressions). Nonglobal quantities of interest can additionally depend on space, time, and other study-dependent inputs.

In the column, type the expression for the output (comp1.ppb1, for example, for a in ). In the column, select if the quantity of interest is global or if the quantity of interest is nonglobal. In the column you can see a small summary of the settings for each quantity of interest. If the setting is not , meaning it is either , , , or , there is an additional column. Here, you can give the name for the corresponding output in the surrogate model. To avoid name clashing, the resulting function name in the surrogate model will be the concatenation of the tag of the surrogate model and the name given in this column. For example, if the surrogate model has the tag dnn1, and the name provided in this table is func1, the resulting function name in the surrogate model corresponding to this quantity of interest will be dnn1_func1. If left empty, a valid default function name will be provided. Use the (

), (), and (

) buttons and the fields under tables to edit the table contents. Or right-click a table cell and select , , or . The button (

) adds a new output to the list. Use the button (

) to clear the entire table.

Underneath the table various settings are available — for each quantity of interest — based on whether the quantity of interest is global or nonglobal.

When there is a parametric sweep in the study, a selection can be made over the outer solutions (parametric solutions). From the list, choose , , or . For , the quantity of interest is defined as the summation of the expression over all the outer solutions. For (or ), the quantity of interest is defined as the maximum (or minimum) of the expression taken over all the outer solutions.

A selection can also be made over the inner solutions. First the selection is made over the outer solutions and then over the inner solutions. From the list, choose (the default), , , , , or . The solution to use will be the last solution for time-dependent and parametric solutions, while for eigenvalue and eigenfrequency solutions, it will be the first solution. You can override this mechanism by selecting any of the other methods. For , the quantity of interest is defined as the summation of the expression over all the solutions. For (or ), the quantity of interest is defined as the maximum (or minimum) of the expression taken over all the solutions. Also note that evaluation operators like at() and with() can be used in the expression, making it possible to evaluate even more general quantities from dynamic solutions.

Depending on the selection for , an additional setting will be available:

•

For , , , or , select from the , , , or list, respectively, which existing surrogate model function to use or select if a new surrogate model function should be created. If an existing surrogate model function is selected and a table is already attached to it, the data will be written to this table.

Only quantities of interest evaluated with the same sampling and selection methods can be written to the same table (or file). This means that global and nonglobal quantities of interest will be separated in separate tables (or files). In the same way, not all nonglobal quantities of interest will be written to the same file. Automatically all quantities of interest evaluated with the same sampling and selection methods will be written to the same file. If is not set to , a separate surrogate model will also be created for every group.

Specify which sampling method to use from the list to sample the quantity of interest on the geometry. Select if the quantity of interest does not have spatial dependence. To create a node, right-click the node, choose , and then the component to which you want to add it. Or create the node directly under the of the using > .

Specify with a selection for the innermost solutions. In a time-dependent parametric simulation with an auxiliary sweep, the innermost parameter will be time. The parameters in the auxiliary sweep are also inner parameters, but only one parameter can be the innermost parameter. For frequency-dependent simulations, the innermost parameter will be frequency, and for stationary simulations, the innermost parameter will be the continuation parameter. Choose from , , , , , and . For eigenvalue and eigenfrequency simulations the default is ; otherwise, the default is . For , the quantity of interest is defined as the summation of the expression over all the innermost solutions. For (or ), the quantity of interest is defined as the maximum (or minimum) of the expression over all the innermost solutions. For (or ), the quantity of interest is defined as the expression evaluated with the last (or first) innermost solution. Select all innermost solutions with . If the innermost parameter is time you can also choose . The quantity of interest is then defined as the expression interpolated at the times specified in . Ranges and vector-valued expressions can be used; see Entering Ranges and Vector-Valued Expressions for more on ranges and vector-valued expressions.

If there are more parameters in the study besides the innermost parameter, the can be used to make further selections in the other parameters. In the column all the parameters besides the innermost parameter are listed. Specify, in the column, which solutions to select for each parameter. Choose from and (the default). For , the quantity of interest is defined as the expression evaluated using the last solution for that parameter. Select all solutions with . can only be selected for inner parameters when the sweep type for the auxiliary sweep is . Select the checkbox in the column if the surrogate model should have an input for this parameter. By default, an input will be created for the surrogate model.

As the data generated for nonglobal quantity of interests is typically large, the data is stored on file. Specify with the list if the file should be saved embedded in the model or external. Choose if the file should be stored embedded in the model — the file will be attached to a table. Choose to save the file outside the model at the path specified with . It is not possible to directly specify the filename; instead specify the , which is shared among all quantities of interest. It can be either a folder or a filename, and both relative and absolute paths are allowed. The file where the quantity of interest will be saved is in a file at the base file path with the name as specified in the field. A small identifier has been added to the filename to mark which quantities of interest are in the file. Click the button (

) to browse the file system.

Depending on the selection for , an additional setting will be available:

•

For , , , or , select from the , , , or list, respectively, which existing surrogate model function to use or choose if a new surrogate model function should be created. If an existing surrogate model function is selected and a table is already attached to it, the data will be written to this table.

In this section you define the input parameters for the surrogate model training.

For the input parameters, in the column, choose the parameter from a list of all global parameters in the model. In column, choose (the default) or .

If you chose , you can define the following properties for the selected parameter underneath the table:

•

In the list, choose the distribution for the input parameter: (the default), μσ, μσ, θ, αβ, λ, or μβ. All distributions except the uniform distribution have two distribution parameters shown under the list, such as the and for a normal distribution and and for a gamma distribution. You specify the distribution parameters in the corresponding text field. All distributions except the uniform distribution and beta distribution have , shown under the list.

The chosen analytic distribution appears in the column.

If you chose in the column, you can instead specify the following properties:

If the source type for the parameter is , you can add correlations groups. Doing so allows sampling of parameters that are not statistically independent by specifying a correlation matrix. You could select a subset of all input parameters in the same correlation group and set the correlation matrix to specify the correlation between each pair of two parameters. Multiple correlation groups can be added in one study, and one parameter can only appear in one correlation group. The correlation matrix is a symmetric semidefinite matrix where all the diagonal elements are equal to 1 and all the off-diagonal elements are between [−1, 1].

Under , specify the following setting if the source type is :

•

If is set to , then in the field, if is set to , or , if is set to , specify the number of input points to use (default: 20 or 10). You may want to use a larger number to train the surrogate model to a sufficient degree of accuracy. The number of input points must be chosen empirically.

You can edit the table using the buttons under the table:

Select the checkbox to add accumulated probe tables for the result from the surrogate model training. From the list, choose to create a new table, or choose any existing table. The checkbox is selected by default to accumulate all the model probes in the table. If you clear this checkbox, the probes selected from the list are used.

From the list, choose (the default) or .

From the list, choose (the default) to only keep the last model evaluation in memory, or choose to keep all model evaluations.

From the list choose (the default) or . Use this setting to control whether a solver sequence should be generated using global parameters or each parameter tuple.

Select the checkbox if you want the surrogate model training algorithm to reuse the solution from the previous step. It is useful if you are using an iterative solver and the solutions for different parameter values are similar enough so that it is faster to start from the previous solution instead of starting the solver from scratch.