COMSOL 6.4 - The Uncertainty Quantification Study

The study (

) contains tools for setting up UQ studies of different types for evaluating the uncertainty and sensitivity in a simulation model with respect to some input parameters with variations described by some statistical distribution. To add an study, right-click a node and choose . You can only have one node in each study.

At the top of the window, you can use the action button to run the study. The effect of this action is determined by the settings as described below.

The window contains the following sections:

At the top of this section, use the list to select one of the following: ; ; ; or . For the UQ study type, when there is no previous data to use, the only option will be , but when the QoI table has been populated, can also be selected. For the UQ method, when there is no previous data to use, the only option will be , but when the QoI table has been populated, and can also be selected. For surrogate model based UQ methods, when the surrogate function is new, the only option will be , but when the surrogate function has been populated, and can also be selected. appends data (to the table) and analyzes the total amount of data. performs the UQ analysis based on the data and surrogate function as is, and does not perform any new COMSOL model evaluations. When verification has been performed, is also an option. This option considers the union of data from the trained surrogate function and the verification table.

From the list, choose one of the following main study types for the UQ:

From the list, choose (the default) or . For , you can choose any existing table group or from the list to use an existing table group or create a new table group for the table data output from the UQ study. automatically generates an output table group for each UQ study type. When you switch between different UQ study types, the result table group automatically points to the output table group corresponding to the study type. The checkbox is selected by default. Clear it to keep existing result tables.

For all UQ study types except , there are also settings under :

From the list, choose one of these surrogate models:

The following surrogate models are available for the and UQ study types.

•

. This method compares the leave-one-out cross-validation error for its polynomial to the . The method starts from the lowest possible polynomial order and then increases it, while still fulfilling the (default: 0.5), until the error is smaller than the tolerance and the surrogate model construction is done. If the maximum polynomial degree is reached without the tolerance criteria being reached, the surrogate model construction terminates, but a warning is printed to the window. The setting for the maximum polynomial degree can be set on the level; see Uncertainty Quantification Job Configurations. The q-norm is real valued in the interval [0, 1], which determines the mixed terms included in the expansion. Mixed terms means terms that involve more than one parameter. A q-norm of 0 means that no mixed terms are used for the polynomial, while 1 allows mixed terms of the same combined order as the nonmixed terms. Also, from the list, choose to create a new function or choose any existing function. See Surrogate Models — Polynomial Chaos Expansion for more information.

•

(the default for Sensitivity analysis). This method finds the q-norm automatically during the surrogate model construction. This mechanism favors a small q-norm. The leave-one-out cross-validation error cannot be used to find new sampling points for the adaptation process. Instead, new sampling points are added with the LHS method, and a new error estimate is computed. The adaptation process is terminated if the error is smaller than the . See the theory section Surrogate Models — Polynomial Chaos Expansion for more information.

•

. This surrogate model uses the standard deviation as its built-in error estimate. This error estimate can be computed for any GP evaluation point. An optimization approach is used to find the maximum error in the input parameter domain. Use the list: (the default) or . The method is normally preferred because it is faster and gives a better estimate. In each iteration of this method, the number of GP evaluations can change. Initially, only a few evaluations are done, but the number can quickly grow. Therefore, you can set a bound, both on the number of iterations and the total number of GP evaluations. Enter values in the (default: 10,000) and fields (default: 500). The method can be used for comparison with the method. If the error obtained from the optimization approach is not smaller than the , a warning is printed to the window. For this surrogate model, also specify a covariance and a mean. From the list, choose , (the default), , , or . From the list, choose (the default), , or . Also, from the list, choose to create a new function or choose any existing function. See Surrogate Models — Gaussian Process for more information.

•

. This is the only applicable surrogate model for . This method also uses the GP standard deviation for error estimation. Here, the same optimization methods as for the GP can be used to find the maximum. If the error obtained from the optimization approach is smaller than the (default: 0.001), the adaptation iterations are terminated. The error estimate is used to add new sampling points, one at a time. For , the error estimation can be seen as a weighted error where the region for the QoIs away from the threshold are down-weighted to be less important. This means that most new points will be added close to the threshold, to enhance the quality of the surrogate model for this analysis. See the Theory section Reliability Analysis — Efficient Global Reliability Analysis for more information on this weighting.

When a surrogate model has been computed, you will, at the bottom of this section, find a list: , , and . Next to this list, you find an action button that executes the action selected. The action independently adds new sampling points, considering the ones you have already used for the surrogate model buildup. For these, new COMSOL model evaluations are performed. This data is then used to compute error estimates for the surrogate model. These error estimates are done independently of any surrogate model built-in error estimates and should be seen as an independent quality test of the surrogate model. Notice that no new UQ analysis data is produced by this action. It will therefore not affect these results. The action can be used when an initial verification computation has already been done. It can be used to further add verification sampling points and COMSOL model evaluations to extend the verification. The action uses an existing verification table and only performs the verification error estimation, without adding any new sampling points or performing any new COMSOL model evaluations. This action can be useful, for example, if there is some COMSOL table with computational results for the same parameters and the same QoIs from a previous computation. See the section Surrogate Model Verification for more information.

From the list, choose to create a new table, or choose any of the available tables. This table will be used to store not only sampling points but also the model evaluation of the quantities of interest (QoIs) for these points. This data serve as a building block for the surrogate models. This table is used when the compute action is . It can also be used as a starting point for the compute action . This table is used for all UQ analyses, and is not put in the Output table group.

The list contains the solution in a COMSOL model to use as the QoI. Choose (the default), , , , , or . The solution to use will be the last solution for time-dependent and parametric solutions, while for eigenvalue and eigenfrequency solutions, it will be the first solution. You can override this mechanism by selecting any of the other methods. For , the QoI is defined as the summation of the over all the solutions. For (or ), the QoI is defined as the maximum (or minimum) of the expression taken over all the solutions. Also note that evaluation operators like at() and with() can be used in the expression, making it possible to evaluate even more general quantities from dynamic solutions. This setting only applies to global QoIs.

The list is available when the UQ study is using a parametric sweep, function sweep, or material sweep in the same study. The choice refers to the solutions in the parametric sweep type study to use as the QoI. Choose (the default), , or . For , the QoI is defined as the summation of the QoIs over all the solutions computed in the parametric-sweep-type studies. For (or ), the QoI is defined as the maximum (or minimum) of the QoIs taken over all the parametric sweep solutions. This setting only applies to global QoIs.

For with multiple QoIs, choose (the default) or from the list. With , the condition for each QoI must be fulfilled. With , the condition for at least one QoI must be fulfilled.

For all UQ studies the QoI table includes at least the following columns:

For , the and columns are also required. The value for defines the relationship between the expression and the threshold. means that the reliability probability is defined as the condition that the value of the expression is larger than the threshold. is the opposite condition. For study-dependent QoI the threshold can depend on study-dependent inputs such as time and frequency.

Underneath the table various settings are available - for each quantity of interest - based on whether the quantity of interest is global or study-dependent.

For global QoI the setting is available. From the list in this column, you can use a specific solution for that QoI. The default, , takes the solution from the list above the table.

For study-dependent QoI, from the , , , or list, specify a selection for the corresponding solver parameter. For a time-dependent simulation, a can be selected, for a frequency-dependent simulation, a can be selected, and so forth. Choose from , , , , , and . For eigenvalue and eigenfrequency simulations the default is ; otherwise, the default is . For , the quantity of interest is defined as the summation over all inner solutions. For (or ), the quantity of interest is defined as the maximum (or minimum) of the expression over all inner solutions. For (or ), the quantity of interest is defined as the expression evaluated with the last (or first) inner solution. Select all inner solutions with .

•

If the parameter is time or frequency you can also choose . The quantity of interest is then defined as the expression evaluated at the values specified in . Select with how the quantities of interest should be evaluated, evaluates with the closest computed solution. Interpolation will interpolate between the closest computed solutions. Extrapolation is not supported with this setting, instead NaN (Not-a-Number) will be written to the QoI table. It is recommended to solve for exactly the times and frequencies you want to evaluate the QoI for and only use this setting to select a specific subset. Ranges and vector-valued expressions can be used; see Entering Ranges and Vector-Valued Expressions in the COMSOL Multiphysics Reference Manual for more on ranges and vector-valued expressions.

•

If the parameter is eigenfrequency or eigenvalue you can instead choose . In the list you can specify which eigenfrequencies or eigenvalues to select. Here, 1 represents the first mode as ordered by the eigenfrequency/eigenvalue solver, 2 represents the second mode and so forth. Ranges and vector-valued expressions can be used.

In this section, you define the input parameters for the UQ study and their related settings.

For each input parameter, from the list, choose (the default) or (for the , , , and study methods only). See the Sampling Settings section below for information about the data generation settings.

Under , specify the input parameters for the UQ study in the table below, which contains the following columns:

When is chosen for , the analytic distribution settings are shown under the table:

•

In the list, choose the distribution for the input parameter: (the default), μσ, μσ, θ, αβ, λ, or μβ. All distributions except the uniform distribution have two distribution parameters shown under the list, such as the and for a normal distribution and and for a gamma distribution. You specify the distribution parameters in the corresponding text field. All distributions except the uniform distribution and beta distribution have , shown under the list.

You can edit the table using the buttons under the table:

For all GP-type surrogate function based UQ methods expect the method, under , specify the input parameters correlation settings for the UQ study in the table below, which contains the following columns:

•

In the column, enter the upper-triangle and diagonal elements of the symmetric positive semidefinite correlation matrix. Each off-diagonal element of the correlation matrix should be between −1 to 1, and the diagonal element of the correlation matrix should be 1. For a correlation group with three input parameters p1, p2 and p3 in successive order, the matrix element (1, 2) represents the correlation between p1 an p2. Likewise, the matrix element (1, 3) represents the correlation between p1 and p3, and the matrix element (2, 3) represents the correlation between p2 and p3. Due to the symmetric nature of the correlation between parameter pairs, the correlation matrix is symmetric.

You can edit the table using the buttons under the table:

Depending on how you choose other settings, you will have different options for sampling settings for data generation. This section is available unless you chose for the .

When you have chosen for all input parameters, the following additional settings are available (except for the study):

From the list, choose (the default) or . The type gives you the possibility to specify the , which is the number of points generated and simulated (with the COMSOL model) before the adaptation starts. The is the total number of input points used by the adaptation. Both of these numbers are important for the adaptation process. The initial number needs to be large enough to cover the sampling space and the complexity in the QoIs. Otherwise, there is a risk that the adaptation algorithm is not adding new input points correctly. An appropriate number depends not only on the dimension of this space, which is the same as the number of input parameters, but also on how large this space is in relation to the variation of the QoIs. The maximum number of input points puts a limit to the whole process. If the adaptation process has not terminated fulfilling the tolerance criteria when this limit is reached, a warning is printed in the window. When adaptation is not used, there is only one setting, the , which dictates the exact size of the training data. Choosing this number will naturally be a tradeoff between surrogate model quality and training cost. Notice that the number of COMSOL model evaluations will be the same as the number of input points. The method chooses these numbers based on the number of input parameters and the compute action used. The number used for these properties can be inspected in the user interface, but they cannot be edited. Table 3-1 lists the default and automatic values for the different compute actions.

Table 3-1: Default and automatic values for the input points properties.
Compute action	Number of input points type	Maximum number of input points (adaptation)	initial number of input points (adaptation)	number of input points (nonadaptation)
Compute and analyze	Automatic	20m	10m	20m
Compute and analyze	Manual	20	10	20
Improve and analyze	Automatic	10m	5m	10m
Improve and analyze	Manual	10	5	10

For the study, the following additional settings are available:

For all UQ study types where at least one input parameter has chosen from the , specify the following additional settings:

From the list, choose (the default) to generate a random seed automatically (which is then displayed below), to enter a seed in the field, or to use that time as the random seed. A random seed is used not only the first time sampling data is generated but also for subsequent generation, for example, for the compute action. To avoid the same sampling data being generated over and over again, another random seed is used for subsequent runs. The method adds 1 to the seed each time you run the study, so that subsequent runs never use the same seed.

When you have chosen , you can choose the data source.

From the list, choose (the default) or .

When you have chosen , you can choose the input parameter data source table and select the data from the table where the list is automatically populated with the table’s column headers.

When you have chosen for all the input parameters, you can choose the sweep type.

From the list, choose (the default) or . These sweep types work exactly as if a parametric sweep study type was used for the input parameters.

The is computed based on the number of data points from or number of data points in the selected table , where the number of and number of data points in table should be the same. When is chosen from the list, the are the multiplication of the number of data points of each .

The section is available for surrogate models used in , , and . For , it is available for the and surrogate models. , , and are all sample-based analyses, which means that a large number of samples are often required to achieve high accuracy of the analysis results. Given the computational cost, it is always forbidden to run a large number of COMSOL model evaluations to a large-scale simulation problem. Here, Monte Carlo analysis performs repeated evaluations using the surrogate model with random input parameter values, according to their distribution settings. A COMSOL model evaluation is only needed to build the surrogate model and is not needed in this analysis.

By default, the is set to . Change it to enable a separate Monte Carlo parameters table below, where you can specify parameters for the Monte Carlo analysis.

When is chosen for the or for all parameters are when is chosen for the , you can enter a suitable number of samples for the Monte Carlo analysis (default: 10,000 samples) in the field. Otherwise, the field is automatically populated based on the number of samples from or data selected from the surrogate-based Monte Carlo analysis data source table.

When you have selected a study type and is chosen for the or for all parameters are when is chosen for the , you can choose (the default) or . With the latter, the same field appears. With selected, you can enter an (default: 1000), a (default: 10,000), and a (default: 0.05).

If desired, select the checkbox to enter a random seed in the field. Another seed can be used to generate another set of random samples, to check how sensitive the UQ analysis result is to the particular set used. Ideally, the results should not change much. If they do, you should consider increasing the number of samples.

When is chosen for the , the Monte Carlo parameters setting table is shown under list.

From the list, choose (the default) or . All settings related to the settings for the are the same as those in the section with one difference, where the difference is that can be selected from the list. When is selected, you can specify a nominal value of the corresponding parameter to be used for the Monte Carlo analysis.

Using settings for the Monte Carlo analysis other than what has been specified under input parameters is an advanced feature, and needs to be used with some care. The specification given here will be used for the UQ analysis. It is therefore possible to modify the specification for the input parameters and get new UQ analysis results through the compute action , without making any new COMSOL model evaluation. Notice, however, that this will only be accurate if the surrogate model is also an accurate model for the new input parameter specification. One simple case where this approach can fail is if the surrogate model is constructed or trained for a smaller domain with stricter bounds, and here is used for a significantly larger domain that involves extrapolation.

The error estimation for the surrogate models are estimations based on the trained surrogate function. The verification error here defined the error, which directly measures how well the surrogate model predicts the QoIs unknown to the model. The section is only available for surrogate models used in , , and . It is not available for because the surrogate model for reliability analysis is not optimized for the global accuracy.

From the list, choose for a new table, or choose an existing table. This table will contain the verification sample points, as well as COMSOL model evaluation of the QoIs, followed by the surrogate model predictions. The verification analysis will also produce the root mean square (RMS) of the difference between the model evaluations and the predictions as well as the relative RMS difference. These numbers can be found in the table under the UQ analysis table group, one row per QoI. The RMS difference can be found in the column and the relative RMS difference in the column.

By default, the is set to . Change it to enable a separate verification parameters table below, where you can specify parameters for the surrogate model verification. This can be useful, for example, if you want to verify the surrogate model in a certain part of the input parameter domain by using other bounds. Notice, however, that the surrogate model is trained under certain conditions and assumptions regarding the input parameters, so it is natural that the predictability (and error) for other input parameter settings is not as good.

When is chosen for the or for all parameters are when is chosen for the , you can enter the desired number of verification points (default: 10) in the field. This is the number of independent COMSOL model evaluations performed for the verification of a surrogate model. The sampling of points will be different from the one used to build the model. For example, another random seed will be used. Otherwise, the field is automatically populated based on the number of points from or data selected from the surrogate model verification data source table.

From the list, choose (the default) to generate a random seed automatically (that initial random seed is then display below), choose to enter a seed in the field, or choose to use that time as the random seed.

In this section, you can generate response surface data for your surrogate function, which can be used from results for visualization. In this section, you can use the button and generate a filled COMSOL table. The filled format of the data makes it possible to use a plot and select two parameters at a time and one QoI. When the table and plot are generated, it is possible to change the two parameters to use and the QoI to visualize. All this is set up for you by the action.

The filled-data structure can be controlled in the four-column table found at the top of the section. The first column, , contains the input parameters. This column is dictated by the surrogate function and cannot be changed. If the surrogate function has been trained, then this column will be populated automatically. If the surrogate function has not been trained, it will be empty, and no response surface data can be generated. The second column is the . For each parameter in this column, you can select , , or . For , the column can be used to prescribe how many points should be used in this parameter dimension. The actual distribution function will here be used to add points close to the mean value. For the method, the column can be used to specify freely which points to use. The method combines the methods and adds points based on the last two columns. Notice that filled data means that all combinations of values for the parameters will be generated or predicted. So, for example, if 5 points are generated for m parameters, 5m rows are generated in the .

The target table for the data generation can be set by . Since the table data can be large, there are also settings for how to store the table. Select one of the methods in the list: , , or . See Table 3-1 for more information. When a file is used, you also have to give a filename in the field. There is also a setting where you can limit how much data is stored from the data generation.

The Table Surface plot can be selected from the .

In this section, you can specify the experimental data source for the inverse uncertainty quantification study.

From the list, choose an existing table that contains the experimental data. This table should contain data columns of measured experimental parameters and the corresponding QoIs.

Under , specify the data for each data column of the table:

When is chosen from the list, the list and are shown under the table. The list is automatically populated with the in the quantities of interests settings, and the defaults to . If you change the type to , an input field appears where you can specify a positive measurement uncertainty; when is chosen from the list, the list and are shown under the table, where the list is automatically populated with the in the input parameters settings; when is chosen from the list, the corresponding data column is ignored in the uncertainty quantification study.

No repeated QoI expressions or input parameters should be used as the and respectively. Once the experimental data settings are specified, the column from the input parameters settings table will be populated with label of calibration parameter or experimental parameter. The input parameters that are not selected as experimental parameters are labeled as calibration parameters. Since the goal of the inverse uncertainty quantification is to compute the posterior distribution of the calibration parameters, at least one input parameters should be considered as calibration parameter.

In this section, you control what to output when solving the UQ study.

Select the checkbox to choose a plot group for plotting from the list. This plot group will be updated while solving the underlying COMSOL model for the model evaluations. No plots will be updated during Monte Carlo analysis. When using the choice, the first automatically added plot group will be used.

Select the checkbox to display numerical results from the UQ study in a table graph group that you choose from the list. Choose to create a new table graph group, or choose any existing table graph group.

From the list, choose which probes to output values from: (the default), , or . If you choose , add any available probes to the table below.

To output to an accumulated probe table, select the checkbox and then choose a probe table for the output from the list. Choose to create a new output table, or choose any existing output table.

From the list, choose (the default) to stop the solution process directly if an error occurs, or choose to skip problematic parameters where the COMSOL model evaluation fails, and continue the solution process for the UQ study. When is chosen, the possible errors are collected and retained in the Uncertainty Quantification Job Configuration node. This node has a warning overlay for its icon when there are collected errors.

From the list, choose (the default) to only keep the last model evaluation, or choose to keep all model evaluations.

From the list, choose (the default) to use the values of the global parameters or choose for a parametric run with parameter tuples. See Parametric Sweep in the COMSOL Multiphysics Reference Manual for more information.

Select the checkbox if desired. This option can be useful for cases where the solutions for the input parameters are close to each other and it benefits the nonlinear solution process to start from the previously computed solution. Notice that because the UQ analysis is global, there can be large differences between these solutions so this option is not always beneficial.

Select the checkbox if you run COMSOL on a distributed system and want to make use of a distributed evaluation. The benefit of distributed evaluation is that during the sampling with the COMSOL model evaluations, these can be done in parallel. However, for the AGP, this is only done for the initial sample points, since only one model evaluation at a time will be performed for subsequent adaptation steps. Without adaptation, or when using the SPCE surrogate model, there is no such limitation.

From the list, choose (the default) to only keep the log from the uncertainty quantification study, choose to only keep minimal log from the uncertainty quantification study, or choose to keep the log from the uncertainty quantification study and the model evaluations from the uncertainty quantification study.