Cluster Computing

To enable this option in the context menu, click the button (

) and select .

) study when you want to submit COMSOL Multiphysics batch jobs to a job scheduler that in turn runs the batch job on a second computer or cluster. For more information, see Cluster Computing (Job Configurations) and its related functionality. Also see Figure 20-5.

Use Cluster Computing to utilize a cluster or cloud to solve a single large model using distributed memory. For maximum performance, the COMSOL cluster implementation can utilize shared-memory multicore processing on each node in combination with the Message Passing Interface (MPI) based distributed memory model. This brings a major performance boost by making the most out of the computational power available.

Cluster Computing is also intended as an interface for setting up distributed COMSOL Multiphysics batch jobs on the computer where the COMSOL Desktop is running. If you are running in distributed mode by starting the COMSOL Desktop with a distributed computing command (for example, comsol -nn 4 -f hostfile), the node is not needed.

Once you have specified the settings, click

to start a COMSOL process that solves the current study.

When the cluster computing process starts, the COMSOL Desktop follows the progress in The External Process Window. When the process finishes (or you click the button to no longer follow the process), an External Process node is selected that represents the current running process. If you are running COMSOL Multiphysics in distributed mode, the model runs in the current process. In this case it is recommended that you do not create a study step.

Figure 20-6: An example of the two kinds of Cluster Computing nodes available under Study and Job Configurations.


	Micromixer — Cluster Version: Application Library path: COMSOL_Multiphysics/Tutorials/micromixer_cluster


	You cannot create another Batch, Batch Sweep, Cluster Sweep, or Cluster Computing in the same study at the same time.

Batch Settings


	After making these settings, click the Save as Default () button on the Settings window toolbar to save the current directory settings as the default preference.

Specify the file in which to store the results of the cluster job in the field. The directory in which to store the model is the one in the field on the page in the dialog box.

Select the check box to delay the generation of the solver sequence (for example, to avoid expensive mesh generation on the desktop before submitting the batch job).

Enter the (physical nodes) to use for this job (the default is 1 node).

Select the check box to run using batch licenses. Batch licenses can be used to run multiple batch jobs for different models that only depend on a parameter. Usually you should use the node. This option makes it possible submit a single batch job from the COMSOL Desktop and continue working there while the cluster job is computing in the background — this approach works with a license for one concurrent user.

From the list, choose (the default) to use settings from the part of the page in the dialog box. Choose to enter the following cluster computing settings:

Choose the : (the default), , , , , or .

General

Select (the default) to configure to run on many types of clusters and schedulers, including Linux clusters.

•

The entry in the field specifies the host file used for the job. If left empty, MPD looks for a file mpd.hosts in the Linux home directory.

•

Select which bootstrap server MPI should use in the setting.

•

Enter details for the Directory Settings for all Cluster Types.

HPCS 2008/2012/2016

Select to use the Windows HPC Server 2008, Windows Server 2012 with HPC Pack 2012, and Windows Server 2016 with HPC Pack 2016 job scheduler to submit the batch job.

When is selected:

•

The entry in the field is the IP address of the enterprise adapter of the head node or the DNS name of the head node. The default is localhost, and it should only be changed if you submit your job from a different computer than the head node.

•

The entry in the field is the user account that COMSOL Multiphysics uses for submitting the job. You provide the password in a separate command window that opens at execution time with the possibility to save the credentials.

•

Enter details for the Directory Settings for all Cluster Types.

OGS/GE

Select to use the open grid scheduler/grid engine job scheduler to submit the batch job. When is selected:

•

Select which bootstrap server should be used by MPI using the setting.

•

Specify the name of the scheduler queue in .

•

Enter the for the cluster queue.

•

Enter details for the Directory Settings for all Cluster Types.

SLURM

Select to use the SLURM job scheduler to submit the batch job. When is selected:

•

The entry in the field is the user account that the COMSOL Multiphysics software uses for submitting the job.

•

Enter the for the cluster queue.

•

Enter details for the Directory Settings for all Cluster Types.

PBS

Select to use a PBS-based (Portable Batch System) job scheduler to submit the batch job. When is selected:

•

The entry in the field is the user account that the COMSOL Multiphysics software uses for submitting the job.

•

Enter the for the cluster queue.

•

Enter details for the Directory Settings for all Cluster Types.

Not Distributed

Select when you want to run a batch job rather than using distributed computing. Enter details for the Directory Settings for all Cluster Types.

Directory Settings for all Cluster Types

•

If you are connected to a COMSOL Multiphysics server on another computer, you can control the working directory used by the COMSOL Multiphysics server if you select the check box and enter the path to the server or for the path to the mounted file system directory where the cluster job result file will be saved. This should be a fully qualified UNC path — for example, \\head1\shared. Otherwise, a temporary directory on the COMSOL Multiphysics server is used to save files. The server directory path refers to the location where the COMSOL Multiphysics server reads and writes the corresponding data (assuming a client-server configuration).

•

If the batch job has another path to the directory, select the check box and enter the path to the external process (batch) directory in the field or click . The cluster job uses this path from the compute node to access the input file and write back the result. On Windows this must be a fully qualified UNC path, for example, \\head1\shared\clusterprojects. The external COMSOL batch directory path refers to the location where the batch process reads and writes the data.

•

If the COMSOL software is installed in a directory other than where the COMSOL Desktop runs, select the check box and then specify the installation directory (click or enter the path to the ). This can occur if you are submitting jobs to a job scheduler. Typically on Windows this is the UNC path to the COMSOL installation root directory for the compute nodes to access the required COMSOL binaries — for example, \\head1\shared\COMSOL54.

See How to Specify Directory Paths for Batch Jobs and Cluster Jobs for some examples of which paths to specify for some common cluster or batch job configurations.

Additional Batch Settings

Enter the . The default is 0. This is the maximum number of times the job can be restarted if it fails to complete.

Enter a value for the . The default is 300 seconds. This is the longest time the process is allowed to run before it must inform that it is still running. Failure to do so means that the process is considered dead, and a new process is started unless the maximum number of job restarts is reached.

Cluster Settings

If you are running a parametric sweep and want to distribute it by sending one parameter value to each compute node, select the check box. This requires that your study includes a parametric sweep. In the node’s window, you must also select the check box in the section.

Remote and Cloud Access

This settings can be specified here or in the dialog box, on the page. From the list under , choose to specify the remote computing settings in this section., or choose to use the remote computing settings in the dialog box. The remote computing settings are the same in both places.

Enable the check box if you want to run COMSOL Multiphysics on a remote machine using a remote start command such as SSH and using a file transfer program such as SCP to transfer the files to and from the remote computer. This allows you to run on a machine installed on your network without a client-server connection or on a machine installed on a remote cloud. Then specify the settings described below.


	COMSOL Multiphysics must be installed on the remote machine and all settings must be specified correctly in the Specify external COMSOL batch directory path and Specify external COMSOL installation directory path settings according to how COMSOL Multiphysics is installed on the remote machine and the remote machines working directories. You must also be able to access the remote machine without a password using the access method selected. This can be achieved by using something that does not require a password, for example, SSH or similar.

Remote Invoke Command

Choose the method for starting COMSOL Multiphysics remotely from the list. You can choose , , , , or defined.

For the method you can choose between using , , or a defined SSH command. You can select the SSH commands installation directory in the SSH directory if the SSH command is not available on the PATH. The SSH key file directory is set in the setting. You can set ports you want to forward in and the host you want to forward the ports to in . This is useful when you have the license manager installed locally, but the machine where COMSOL Multiphysics is running cannot access the license manager; for instance, if the machine is in the cloud. Specify the user name to use on the remote machine with SSH user.

The method writes all batch command to a file that you specify in the field. This file can then be used to run the command on another computer.

The method uses MPI to launch COMSOL batch jobs on the hosts listed in the table or file. On Windows, you want to launch smpd on the hosts (see Using Microsoft MPI and SMPD on Standalone Computers).

For the -defined , you can enter a command in the setting. In the setting, any use of the keyword {remotehost} is replaced by the name of the remote host when COMSOL Multiphysics starts.

File Transfer Command

Choose the method for transferring files to the remote computer in . You can choose , , or defined.

For the method, you can choose between using , , or a defined SCP command. You can select the SCP commands installation directory in if the SCP command is not available on the PATH. The SCP key file directory is set in the setting. Specify the user name to use on the remote machine with SCP user.

The method stores a list of all files that should be transferred to another computer where the batch job runs. In , it stores a list of all files to transfer to the other computer before running the batch job. In , it stores a list of all files to transfer from the other computer after the batch jobs have finished. Note that unless you detach, COMSOL Multiphysics will be waiting for the data to be transferred to that remote computer and back and then update its solutions.

For the -defined , you can enter a command in the for the transfer of files from the local computer to the remote machine and for the transfer of files from the remote machine to the local computer. In the settings, any use of the keyword {remotehost} is replaced by the name of the remote host. Any use of the keyword {localfile} is replaced by the name of the local file, and any use of the keyword {remotefile} is replaced by the name of the remote file.

Remote Hosts

From the , select if the Remote host list should be read from the table or if it should be read from a file.

In the table, list the hosts you want to run on. If several hosts are listed, COMSOL Multiphysics allocates a job on the first host that is free. The points to a file on the server side (in a client-server configuration; otherwise, on the current computer) that contains the list of hosts to run on remotely.

Use the to specify if the remote computer is running the same OS () or is running or .

Study Extensions

Select the check box when the batch process should run results nodes that create graphical contents such as exporting to file.

Enter the . The default is 1. This is the maximum number of batch processes that are allowed to run simultaneously.


	You can also make changes to how some of these settings are displayed throughout COMSOL Multiphysics in The Preferences Dialog Box in the Remote Computing and Multicore and Cluster Computing sections.


	Running COMSOL Multiphysics in Client-Server Mode