Cluster Computing (Job Configurations)
The Cluster Computing () job configuration is useful when you want to submit a batch job to a job scheduler or you want to run the model in distributed mode as a batch job. This node is created the first time you submit a job. If you want to change its settings before the first job submission, right-click the Study node and select Show Default Solver.
When you have specified the cluster computing settings, click the Save as Default button () in the Settings window’s toolbar to save the current setting as default.
These settings are saved to The Preferences Dialog Box in the Multicore and Cluster Computing section.
General
Use the Defined by study step list to specify if the settings are synchronized with the corresponding Cluster Computing study step (see Cluster Computing), or select User defined to define cluster settings in the Cluster Settings section below.
From the Batch job list, select which batch job to submit. Click the Go to Source button ) to move to the Settings window for the selected Batch node.
Cluster Settings
After making these settings, click the Save as Default () button in the Settings window toolbar to save the current directory settings as the default preference.
Enter the Number of nodes (compute nodes; see Overview of Simulations on Shared- and Distributed-Memory Computers) to use (the default is 1 node). Available with Preferences controlled settings and for the General, HPCS 2008/2012/2016, and SLURM scheduler types.
Enter the Number of processes on host. The default is 1. Available with Preferences controlled settings and for the General and SLURM scheduler types.
Cluster Computing Settings
From the Settings list, choose Preference controlled (the default) to use settings from the Cluster computing part of the Multicore and Cluster Computing page in the Preferences dialog box. Choose User controlled to enter the following cluster computing settings:
If you want to use a job scheduler or do some other operations before or after the job, you set the command line in the Prepend command and Postpend command fields. You can define a default command line with the system properties cs.precmd and cs.postcmd. If the command line contains {nn} or {perhost}, they are replaced by the values in the Number of nodes field and Number of processes on host field, respectively.
Choose the Scheduler type: General (the default), Microsoft® HPC Pack, OGS/GE, SLURM, PBS, IBM LSF®, or Not distributed:
General
Select General (the default) to configure to run on many types of clusters and schedulers, including Linux clusters.
•
The entry in the Host file field specifies the host file used for the job. If left empty, MPD looks for a file mpd.hosts in the Linux home directory.
•
Select which bootstrap server should be used by MPI using the Bootstrap server setting.
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
Microsoft® HPC Pack
Select Microsoft® HPC Pack to use the Microsoft HPC Pack job scheduler to submit the batch job.
•
Select a Node granularity: Node (the default), Socket, or Core. Node allocates one process on each host, Socket allocates one process on each socket, and Core allocates one process on each core.
•
The Exclusive nodes check box is selected by default. Keep it selected if a compute node is used exclusively for a job — that is, no other jobs can interfere. Click to clear it if you want to run on nodes shared by other users.
•
Set the names of Requested nodes to use specified nodes only. The job scheduler only allocates jobs on the nodes listed by you.
•
Enter the Node group. As above, but can use node group names if defined by the system administrator. The job scheduler only allocates jobs on the nodes belonging to the group.
•
Enter the minimum required Cores per node — the number of concurrent threads per physical node. The default is 0, which means all cores. The job scheduler only allocates jobs to nodes with at least as many cores as set.
•
Enter the minimum required Memory per node (MB). The default is 0, which means no limit. The job scheduler only allocates jobs to nodes with at least as much memory as set.
•
Enter the Runtime (minutes) before the job is canceled. The default is Infinite; that is, the job is never canceled.
•
Select a Priority — Highest, Above normal, Normal (the default), Below normal, or Lowest — for the scheduled job based on the standard scheduler.
Under Cluster computing settings:
•
If you want to include scheduler arguments, add them to the Additional scheduler arguments field (for example, for mpiexec).
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
Under Advanced:
•
The entry in the Scheduler field is the IP address of the enterprise adapter of the head node or the DNS name of the head node. The default is localhost, and it should only be changed if you submit your job from a different computer than the head node.
•
The entry in the User field is the user account that COMSOL Multiphysics uses for submitting the job. You provide the password in a separate command window that opens at execution time with the possibility to save the credentials.
OGS/GE
Select OGS/GE to use the Open Grid Scheduler/Grid Engine job scheduler to submit the batch job.
When OGS/GE is selected:
•
Select a Slot granularity — Host, Slot, or Manual — to specify if COMSOL Multiphysics should parallelize on the physical Host level or on the OGS/GE-allocated Slot level. For Host and Slot, specify the Number of slots to allocate. The Manual setting can be used to control the granularity more. In this case set the number of computational nodes to use in the Number of nodes. For Slot and Manual, the number of processes on each node is set in the Number of processes on host field; usually this is 1.
•
Enter the Number of processes on host to set the number of processes on each host.
•
The Sun Grid Engine priority is set in the Priority value field. The default is 0.
Under Cluster computing settings:
•
If you want to include scheduler arguments, add them to the Additional scheduler arguments field (for example, for mpiexec).
•
Select the Bootstrap server that should be used by MPI.
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
•
Enter the Queue name to set the name of the Sun Grid Engine.
SLURM
Select SLURM to use the SLURM job scheduler to submit the batch job.
When SLURM is selected:
•
The Exclusive nodes check box is selected by default. Keep it selected if a compute node is used exclusively for a job — that is, no other jobs can interfere. Click to clear it if you want to run on nodes shared by other users.
•
Set the names of Requested nodes to use specified nodes only. The job scheduler only allocates jobs on the nodes listed by you.
•
Enter the minimum required Memory per node (MB). The default is 0. The job scheduler only allocates jobs to nodes with at least as much memory as set.
•
Enter the Runtime (minutes) before the job is canceled. The default is Infinite; that is, the job is never canceled.
•
The SLURM priority value is set in the Priority value field. The default is 0.
Under Cluster computer settings:
•
If you want to include scheduler arguments, add them to the Additional scheduler arguments field (for example, for mpiexec).
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
•
Enter the Queue name to set the name of the SLURM job scheduler.
Under Advanced:
•
The entry in the Scheduler field is the IP address of the enterprise adapter of the head node or the DNS name of the head node. The default is localhost.
•
The entry in the User field is the user account that the COMSOL software uses for submitting the job. You provide the password in a separate command window that opens at execution time with the possibility to save the credentials.
PBS
Select PBS to use a PBS-based (Portable Batch System) job scheduler to submit the batch job. When PBS is selected:
•
Enter the minimum required Memory per node (MB). The default is 0. The job scheduler only allocates jobs to nodes with at least as much memory as set.
•
Enter the Runtime (minutes) before the job is canceled. The default is Infinite; that is, the job is never canceled.
•
The PBS priority value is set in the Priority value field. The default is 0.
Under Cluster computer settings:
•
If you want to include scheduler arguments, add them to the Additional scheduler arguments field (for example, for mpiexec). To designate nodes as exclusive, use the Additional scheduler arguments -n. This means that a compute node is used exclusively for a job — that is, no other jobs can interfere.
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
Under Advanced:
•
The entry in the Scheduler field is the IP address of the enterprise adapter of the head node or the DNS name of the head node. The default is localhost.
•
The entry in the User field is the user account that the COMSOL software uses for submitting the job. You provide the password in a separate command window that opens at execution time with the possibility to save the credentials.
IBM LSF®
Select IBM LSF® to use an LSF-based (IBM® Platform Load Sharing Facility) job scheduler to submit the batch job. When IBM LSF® is selected:
Under Cluster computer settings:
•
If you want to include scheduler arguments, add them to the Additional scheduler arguments field (for example, for mpiexec). To designate nodes as exclusive, use the Additional scheduler arguments -n. This means that a compute node is used exclusively for a job — that is, no other jobs can interfere.
•
Specify the installation directory for the MPI in the MPI directory field. Click Browse to browse for a directory on the file system.
•
If you must provide extra arguments to MPI, use the Additional MPI arguments field.
Under Advanced:
•
The entry in the Scheduler field is the IP address of the enterprise adapter of the head node or the DNS name of the head node. The default is localhost.
•
The entry in the User field is the user account that the COMSOL software uses for submitting the job. You provide the password in a separate command window that opens at execution time with the possibility to save the credentials.
Not Distributed
Select Not distributed when you want to submit a batch job to a job scheduler without running a distributed job.
Remote and Cloud Access
See Remote and Cloud Access described for Cluster Computing.
Micromixer — Cluster Version: Application Library path COMSOL_Multiphysics/Cluster_and_Batch_Tutorials/micromixer_cluster.