NEMO-3.6 Run Description File

Example Run Description YAML Files for NEMO-3.6

Basic Example

# nemo command processor run description example for NEMO-3.6

# Name of the NEMO-3.6 configuration to use for the run;
# i.e. your NEMO-3.6-code/NEMOGCM/CONFIG/ configuration directory
config name: mackenzie03

# How is the domain to be distributed over the processors in the
# i and j grid directions?
# These values are used to set the nammpp namelist jpni & jpnj values
# and to set the number of processors and nodes in the PBS script
MPI decomposition: 6x6

# For runs on systems that use qsub/PBS/TORQUE
#
# The run_id value appears in the list of jobs display by the showq command
run_id: example
# Maximum run time requested/allowed for job
# Too low and you job will be terminated before it finishes
# Too high and you will have to wait longer on the queue for your job to start
# You have to experiment to find the "just right" value
walltime: 10:00:00
# Email address to send job begin, end, and abort notifications to
email: you@example.com

paths:
  # Absolute path to CONFIG/ directory in NEMO-3.6 code tree.
  # You can use ~ or $HOME or $USER if you wish.
  NEMO code config: $HOME/CANYONS/NEMO-3.6/CONFIG/
  # If relative, paths below are taken from current directory
  # You can use ~, $USER, $HOME if you wish.
  XIOS: $HOME/CANYONS/XIOS/
  # Optional directory to use as base for relative paths in grid:
  # and forcing: sections below
  forcing: ../../../NEMO-forcing/
  # Directory in which to create temporary run directories
  # Typically NOT inside a version control repository
  runs directory: $HOME/CANYONS/runs/

grid:
  # If relative, paths are taken from the grid/ directory in the forcing
  # path above
  coordinates: coords_02.nc
  bathymetry: ideal_bathy_05.nc

forcing:
  # The keys below are the names of symlinks that will be created.
  # The targets of those symlinks will be the paths named by the associated
  # "link to:" keys;
  # e.g. a symlink named NEMO_files will be created to
  # $HOME/CANYONS/mackenzie_canyon/conditions/NEMO_files/
  #
  # The keys are directory names that you use as "cn_dir" values in your
  # namelists.
  #
  # You only need to include keys that are used in the namelist(s) for
  # your run.
  #
  # If relative, paths are taken from forcing path above
  NEMO_files:
      link to: $HOME/CANYONS/mackenzie_canyon/conditions/NEMO_files/

restart:
  # The keys below are the names of symlinks that will be created.
  # The targets of those symlinks will be the paths/filenames associated
  # with the keys;
  # e.g. a symlink named restart.nc will be created to
  # $HOME/CANYONS/results_mackenzie/idealized/sbcana_forcings/forcing01/GYRE_00000030_restart.nc
  #
  # You only need to include keys for the types of restart files
  # that are used in your run.
  #
  # If relative, paths are taken from current directory
  restart.nc: $HOME/CANYONS/results_mackenzie/idealized/sbcana_forcings/forcing01/GYRE_00000030_restart.nc

namelists:
  # The namelist section files in the lists below will be concatenated
  # to create a namelist file whose name is the key under which the files
  # are listed. The keys are the names of the namelist files that NEMO-3.6
  # expects.
  #
  # The only required key is namelist_cfg.
  #
  # If relative, paths are taken from current directory
  namelist_cfg:
    - namelist.time
    - namelist.domain
    - namelist.surface
    - namelist.lateral
    - namelist.bottom
    - namelist.tracer
    - namelist.dynamics
    - namelist.vertical
    - namelist.compute
  namelist_top_cfg:
    - namelist_top_cfg
  namelist_pisces_cfg:
    - namelist_pisces_cfg

output:
  # Whether or not to run the XIOS server(s) in separate process(es)
  separate XIOS server: True
  # How many XIOS servers to use
  XIOS servers: 1
  # If relative, paths are taken from current directory
  #
  # The path and name of the output definitions file to use for the run
  iodefs: iodef.xml
  # The path and name of the output domains definitions file to use for the run
  domaindefs: ../domain_def.xml
  # The path and name of the output field definitions file to use for the run
  fielddefs: $HOME/CANYONS/mackenzie_canyon/output/field_def.xml

vcs revisions:
  git:
    # Absolute paths to Mercurial repos that you want revision records of
    # in your run results
    # You can use ~ or $HOME or $USER if you wish.
    - $HOME/CANYONS/NEMO-3.6-code/
    - $HOME/CANYONS/XIOS/
    - $HOME/CANYONS/mackenzie_canyon/

Please see the explanations in the sections below, starting with Basic Run Configuration, for details about each section of the file, and about optional sections that you may need.

Example for Runs on `computecanada.ca` Systems

This is an example run description file for use on cedar and graham.

# nemo command processor run description example for NEMO-3.6
# running on a computecanada.ca system like cedar or graham

# Name of the NEMO-3.6 configuration to use for the run;
# i.e. your NEMO-3.6-code/NEMOGCM/CONFIG/ configuration directory
config name: GYRE_OFF

# How is the domain to be distributed over the processors in the
# i and j grid directions?
# These values are used to set the nammpp namelist jpni & jpnj values
# and to set the number of processors and nodes in the PBS script
MPI decomposition: 2x3

# The run_id value appears in the list of jobs display by the squeue command
run_id: gyre_TRC

# Maximum run time requested/allowed for job
# Too low and you job will be terminated before it finishes
# Too high and you will have to wait longer on the queue for your job to start
# You have to experiment to find the "just right" value
walltime: 0:10:00

# Email address to send job begin, end, and abort notifications to
email: you@example.com

# Account name to charge resources used by the job to
account: def-allen


paths:
  # Absolute path to CONFIG/ directory in NEMO-3.6 code tree.
  # You can use ~, $HOME, $USER, or other system-defined environment variables
  # if you wish.
  NEMO code config: $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/
  # If relative, paths below are taken from current directory
  # You can use ~, $USER, $HOME if you wish.
  #
  # Directory tree in which to find XIOS
  XIOS: $PROJECT/$USER/GEOTRACES/XIOS-2/
  # Directory in which to create temporary run directories
  # Typically NOT inside a version control repository
  runs directory: $SCRATCH/


grid:
  # Path to the coordinates file
  coordinates: $PROJECT/$USER/GEOTRACES/runs/GYRE_OFF/dummy-coord.nc
  # Path to the bathymetry file
  bathymetry: $PROJECT/$USER/GEOTRACES/runs/GYRE_OFF/dummy-bathy.nc
  # Optional path/filename for land processor elimination MPI-LPE mapping
  # file that matches bathymetry;
  # If "land processor elimination:" key is absent or has the value "False",
  # land processor elimination is disabled
  land processor elimination: False

forcing:
  # The keys below are the names of symlinks that will be created.
  # The targets of those symlinks will be the paths named by the associated
  # "link to:" keys;
  # e.g. a symlink named dynamics_fields will be created to
  # $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/data/
  #
  # The keys are directory names that you use as "cn_dir" values in your
  # namelists.
  #
  # You only need to include keys that are used in the namelist(s) for
  # your run.
  dynamics_fields:
      link to: $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/data/
  mesh_mask.nc:
      link to: $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/mesh_mask.nc


namelists:
  # The namelist section files in the lists below will be concatenated
  # to create a namelist file whose name is the key under which the files
  # are listed. The keys are the names of the namelist files that NEMO-3.6
  # expects.
  #
  # If relative, paths are taken from current directory
  namelist_cfg:
    - $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/namelist_cfg
  namelist_top_cfg:
    - $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/namelist_top_cfg
  namelist_my_trc:
    - $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/namelist_my_trc
  namelist_ref:
    - $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/NEMOGCM/CONFIG/GYRE_OFF/EXP00/namelist_ref


output:
  # Whether or not to run the XIOS server(s) in separate process(es)
  separate XIOS server: True
  # How many XIOS servers to use
  XIOS servers: 1

  # If relative, paths are taken from current directory
  #
  # The path and name of the output definitions file to use for the run
  iodefs: $PROJECT/$USER/GEOTRACES/runs/GYRE_OFF/iodef.xml
  # The path and name of the output domains definitions file to use for the run
  domaindefs: $PROJECT/$USER/GEOTRACES/runs/GYRE_OFF/domain_def.xml
  # The path and name of the output field definitions file to use for the run
  fielddefs: $PROJECT/$USER/GEOTRACES/runs/GYRE_OFF/field_def.xml


vcs revisions:
  git:
    # Absolute paths to Mercurial repos that you want revision records of
    # in your run results
    # You can use ~, $HOME, $USER, or other system-defined environment variables
    # if you wish.
    - $PROJECT/$USER/GEOTRACES/NEMO-3.6-code/
    - $PROJECT/$USER/GEOTRACES/XIOS-2/
    - $PROJECT/$USER/GEOTRACES/XIOS-ARCH/
    - $PROJECT/$USER/GEOTRACES/NEMO-Cmd/


modules to load:
  # Modules to load for the run
  - StdEnv/2020
  - netcdf-fortran-mpi/4.5.2
  - python/3.9.6

Please see the explanations in the sections below, starting with Basic Run Configuration, for details about each section of the file, and about optional sections that you may need.

Basic Run Configuration

The following key-value pairs provide the basic configuration for the run:

config name

The name of the NEMO configuration to use for runs. It is the name of a directory in NEMOGCM/CONFIG/ in the NEMO-3.6-code code directory given by the NEMO key in the paths Section.

This key may also be spelled config_name for backward compatibility.

MPI decomposition

Specify how the domain is to be distributed over the processors in the i (longitude) and j (latitude) directions; e.g. 8x18. Those values are used to set the nammpp namelist jpni and jpnj values.

run_id

The job identifier that appears in the qstat listing.

walltime

The wall-clock time requested for the run. It limits the time that the job will run for on all machines, and it also affects queue priority for runs on shared HPC cluster machines such as Westgrid. It is important to allow some buffer time when calculating your walltime limits to allow for indeterminacy of the NEMO run, as well as the time required to post-process results files at the end of the run.

email

The email address at which you want to receive notification of the beginning and end of execution of the run, as well as notification of abnormal abort messages.

account

Optional. The account name to include in the #SBATCH directives section of the SalishSeaNEMO.sh job script. This key-value pair is required on systems like cedar.computecanada.ca and graham.computecanada.ca that use the Slurm Workload Manager and that you submit runs to with the sbatch command.

`paths` Section

The paths section of the run description file is a collection of directory paths that the nemo command processor uses to find files in other repos that it needs.

NEMO code config

The path to the CONFIG/ directory in the NEMO-3.6 code tree (typically a version control system clone/checkout) where the NEMO configuration directories are to be found; e.g. $HOME/MEOPAR/NEMO-3.6/CONFIG/.

An absolute path is required because the path is used in both the current directory and the temporary run directory created in the runs directory. You can use ~ or $HOME in the path, if you wish.

This key may also be spelled NEMO-code-config for backward compatibility.

XIOS

The path to the XIOS code tree (typically a version control system clone/checkout) where the XIOS executable for the run is to be found.

This path may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

forcing

Optional. The path to the directory tree where the netCDF files for the grid coordinates, bathymetry, initial conditions, open boundary conditions, etc. are to be found.

If this path is provided it is used as the base for relative paths in the grid Section and the forcing Section.

This path may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

runs directory

The path to the directory where run directories will be created by the nemo prepare) sub-command.

This path may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

`grid` Section

The grid section of the run description file contains 3 key-value pairs that provide the paths/filenames of the grid files to use for the run:

coordinates
bathymetry

An example grid section:

grid:
  coordinates: coords_02.nc
  bathymetry: ideal_bathy_05.nc
  land processor elimination: False

If simple file names are provided, those files are presumed to be in the grid/ sub-directory of the directory tree pointed to by the forcing key in the paths Section.

If relative paths are given, they are appended to the grid/ directory of the forcing path.

Relative or absolute paths may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

coordinates

The name of, or path to, the coordinates file to use for the run. It is symlinked in the run directory as coordinates.nc (the file name required by NEMO).

bathymetry

The name of, or path to, the bathymetry file to use for the run. It is symlinked in the run directory as bathy_meter.nc (the file name required by NEMO).

land processor elimination

The name of, or path to, the MPI-LPE mapping CSV file that corresponds to the bathymetry to use for the run. That file contains the information necessary to configure NEMO-3.6 to perform calculations only on the MPI subdomains that contain water. Please see Land Processor Elimination for a detailed explanation. The NEMO Command Processor takes care of setting the correct number of processors based on your chosen MPI decomposition.

A value of False disables Land Processor Elimination

If the land processor elimination key is missing, a warning message is printed to let you know that the run is proceeding on the assumption that you want to run without land processor elimination.

`forcing` Section

The forcing section of the run description file contains sub-sections that provide the names of directories and files that are to be symlinked in the run directory for NEMO to use to read initial conditions and forcing values from.

An example forcing section:

forcing:
  NEMO-atmos:
    link to: /results/forcing/atmospheric/GEM2.5/operational/
  open_boundaries:
    link to: open_boundaries/
  rivers:
    link to: rivers/

The sub-section keys (NEMO-atmos, open_boundaries, and rivers above) are the names of the symlinks that will be created in the run directory. Those names are expected to appear in the appropriate places in the namelists. The values associated with the link to keys are the targets of the symlinks that will be created in the run directory.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

If relative paths are given, they are appended to the forcing path given in the paths Section.

To provide links to directories containing initial fields for a run to use, include a sub-section like:

forcing:
  ...
  initial_strat:
    link to: initial_strat/
  initial_green:
    link to: initial_green/

You are free to use any keys that you wish with the understanding that the key will be the name of the symlink that will be created in the run directory, and that name will also need to appear as a directory name in the appropriate namelist.

The nemo prepare sub-command and the nemo_cmd.api.prepare() API function confirm that the targets of the symlinks exist, and exit with an error message if not.

Atmospheric Forcing File Checks

Additional checking can be performed on the files in the atmospheric forcing directory. That checking confirms the existence of all of the atmospheric forcing files for the date range of the run. Doing so ensures that a run won’t fail part way through due to a missing atmospheric forcing file. To enable the additional checking add a check link section at the same level as the link to key:

forcing:
  NEMO-atmos:
    link to: /results/forcing/atmospheric/GEM2.5/operational/
    check link:
      type: atmospheric
      namelist filename: namelist_cfg

The type key provides the type of checking to perform on the link. The value associated with the namelist filename key is the name of the namelist file in which the atmospheric forcing link is used.

Link checking can be disabled by excluding the check link section, or by setting the value associated with the type key to None.

`restart` Section

The optional restart section of the run description file contains key-value pairs that provide paths and file names of restart files to be used to initialize the run.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

If relative paths are given, they are appended to the directory containing the run description file.

Here is an example restart section:

restart:
  restart.nc: $HOME/CANYONS/results_mackenzie/idealized/sbcana_forcings/forcing01/GYRE_00000030_restart.nc

NEMO requires that the name of the model restart file be restart.nc, so that is the key that you must use. For an (optional) tracers restart file the required file name (key) is restart_trc.nc.

The restart section is optional because it is not required for runs that are initialized with fields provided in a directory linked in the forcing Section.

The nemo run and nemo prepare commands and the nemo_cmd.api.prepare() API function confirm that the targets of the symlinks exist, and exit with an error message if not.

`namelists` Section

The namelists section of the run description file contains one or more dicts of lists of NEMO namelist section files that will be concatenated to construct namelist*_cfg files (the file names required by NEMO) file for the run.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

If relative paths are given, they are appended to the directory containing the run description file.

Here is an example namelist section:

namelists:
  namelist_cfg:
    - namelist.time
    - namelist.domain
    - namelist.surface
    - namelist.lateral
    - namelist.bottom
    - namelist.tracer
    - namelist.dynamics
    - namelist.vertical
    - namelist.compute
  namelist_top_cfg:
    - namelist_top_cfg
  namelist_pisces_cfg:
    - namelist_pisces_cfg

A namelist_cfg key must be present, other namelist*_cfg keys are optional. Each namelist*_cfg section must be a list containing at least 1 namelist section file.

Namelist sections that are specific to the run such as namelist.time where the starting and ending timesteps and the restart configuration are defined are typically stored in the same directory as the run description file. That means that they are simply listed by name in the appropriate namelist*_cfg section:

namelists:
  namelist_cfg:
    - namelist.time

On the other hand, when you want to use a namelist section that contains your research group’s current consensus best values, list it as a relative or absolute path from the location of your run description file to the “standard” nameslist section files in the directory tree in which you also store your model configuration elements:

namelists:
  namelist_cfg:
    - ../../nemo3.6/namelist.bottom

For each namelist*_cfg key a NEMOGCM/CONFIG/config_name/EXP00/namelist*_ref file is symlinked into the run directory to provide default values that will be used for any namelist variables not included in the namelist section files listed in the namelists section. config_name is the value of the config name key in the run description file.

So, NEMOGCM/CONFIG/config_name/EXP00/namelist_ref will always be symlinked and, if the namelist_top_cfg key is present, the NEMOGCM/CONFIG/config_name/EXP00/namelist_top_ref file will also be symlinked into the run directory.

You can override the use of *_ref namelists from CONFIG/config_name/EXP00/ by including a *_ref namelist key. For example:

config name: SMELT

...

namelists:
  namelist_ref:
    - $HOME/MEOPAR/test-sponge/namelist_ref

will cause the namelist_ref file in the $HOME/MEOPAR/test-sponge/namelist_ref directory to be symlinked into the temporary run directory instead of CONFIG/SMELT/EXP00/namelist_ref.

`output` Section

The output section of the run description file contains key-value pairs that provide the names of the files that define the output files, domains, and fields to be used by the XIOS server for the run.

The paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

If relative paths are given, they are appended to the directory containing the run description file.

iodefs

The path and name of the iodef.xml output files definitions file to use for the run. It is copied into the run directory as iodef.xml (the file name required by XIOS). The value is typically either:

a relative or absolute run-specific output files definitions file
a relative or absolute path to an iodef.xml file in the directory tree in which you also store your model configuration elements; e.g. $HOME/CANYONS/mackenzie_canyon/output/iodef.xml

This key may also be spelled files for backward compatibility.

domainefs

The path and name of the domain_def.xml output domains definitions file to use for the run. It is copied into the run directory as domain_def.xml (the file name required by XIOS). The value is typically either:

a relative or absolute path to a domain_def.xml file in the directory tree in which you also store your model configuration elements e.g. $HOME/CANYONS/mackenzie_canyon/output/domain_def.xml
a relative or absolute run-specific output domains definitions file

This key may also be spelled domain for backward compatibility.

fielddefs

The path and name of the field_def.xml output fields definitions file to use for the run. It is copied into the run directory as field_def.xml (the file name required by XIOS). The value is typically a relative or absolute path to CONFIG/SHARED/field_def.xml.

This key may also be spelled fields for backward compatibility.

filedefs (optional)

The path and name of the file_def.xml output domains definitions file to use for the run. This item is optional because it is only used by XIOS-2 (but it is required by XIOS-2). It is copied into the run directory as file_def.xml (the file name required by XIOS-2). The value is typically either:

a relative or absolute run-specific output domains definitions file
a relative or absolute path to a file_def.xml file in the directory tree in which you also store your model configuration elements e.g. $HOME/CANYONS/mackenzie_canyon/output/file_def.xml

The output section also contains key-value pairs that control how the XIOS server is run and, in the case where it is run as a separate server, the number of XIOS servers to run.

separate XIOS server: Boolean flag indicating whether the XIOS server should be run on separate processors from NEMO (True), or in attached mode on every NEMO processor (False). The nemo prepare sub-command sets the value of the using_server variable in the xios context in the copy of the iodef.xml file in the temporary run directory to reflect the separate XIOS server value.
XIOS servers: The number of XIOS servers to run when the value of separate XIOS server it True. The number of XIOS servers is added to the number of NEMO processors calculated from the MPI decomposition value to specify the total number of processors requested in the #PBS directives section of the NEMO.sh script generated by the nemo run sub-command.

`vcs revisions` Section

The optional vcs revisions section of the run description file contains lists of version control system repositories of which the revision and status will be recorded in the temporary run and run results directories.

Here is a (slightly non-sensical) example vcs revisions section:

vcs revisions:
  git:
    - $HOME/CANYONS/NEMO-3.6-code/
    - $HOME/CANYONS/XIOS/
    - $HOME/CANYONS/XIOS-ARCH/
    - $HOME/CANYONS/mackenzie_canyon/
    - $HOME/CANYONS/NEMO-Cmd/
  hg:
    - $HOME/MEOPAR/FVCOM-VHFR-config

The sub-section keys (hg and git above) are the names of the version control tools to use for the repositories listed below them. At present only Mercurial (hg) and Git (git) are supported.

The paths listed under the version control tool keys are the repositories of which the revision and status will be recorded.

The repository paths may be relative or absolute, and may contain ~ or $HOME as alternative spellings of the user’s home directory, and $USER as an alternative spelling of the user’s userid.

For each repository, a file will be created in the temporary run directory. The file names are the repository directory names with _rev.txt appended. So, from the example above, the files created will be:

NEMO-3.6-code_rev.txt
XIOS_rev.txt
mackenzie_canyon_rev.txt
NEMO-Cmd_rev.txt
FVCOM-VHFR-config_rev.txt

For Git repositories, each _rev.txt file will contain the output of the commands:

git branch --show-current
git log -1
git show --pretty="" --name-only

for the repository. That is a record of the last committed revision of the repository that will be in effect for the run. For example, NEMO-3.6-code_rev.txt might contain:

branch: master
commit: 4126141d62f22065eca9e95758c9b9fb850f4f6d
author: Michael Dunphy
date:   Mon Nov 14 09:49:40 2016 -0800
files:  NEMOGCM/CONFIG/SalishSea/MY_SRC/bdyini.F90 NEMOGCM/CONFIG/SalishSea/MY_SRC/tideini.F90
message:
Fix uninitialized variables in bdyini.F90 and tideini.F90

If any of the listed repositories contain uncommitted changes, the nemo prepare command will generate a warning message like:

nemo_cmd.prepare WARNING: There are uncommitted changes in $HOME/MEOPAR/NEMO-3.6-code/

and the list of uncommitted changes and their status codes, the output of the git diff --name-status command, will be appended to the _rev.txt file.

For Mercurial repositories, each _rev.txt file will contain the output of the hg parents -v command for the repository. That is a record of the last committed revision of the repository that will be in effect for the run. For example, FVCOM-VHFR-config_rev.txt might contain:

changset:   950:35fc362f3d77866df8c0a8b743aca81359295d59
tag:        tip
user:       Doug Latornell <djl@douglatornell.ca>
date:       Fri Mar 08 15:08:15 2019 -08:00
files:      namelists/namelist.numerics.template
description:
Change namelist.numerics to a template file.

If any of the listed repositories contain uncommitted changes, the nemo prepare command will generate a warning message like:

nemo_cmd.prepare WARNING: There are uncommitted changes in $HOME/MEOPAR/FVCOM-VHFR-config/

and the list of uncommitted changes and their status codes, the output of the hg status -mardC command, will be appended to the _rev.txt file.

`PBS resources` Section

The optional PBS resources section of the run description file contains a list of HPC resource key-value pairs for which #PBS -l directives will be added to the NEMO.sh script in the temporary run directory. You will need to use this section if you are running on an HPC system and need to request special resources; otherwise, you can omit the PBS resources section.

An example of a PBS resources section is:

PBS resources:
  - partition=QDR

That will result in a line like:

#PBS -l partition=QDR

being included in the NEMO.sh script in the temporary run directory.

Note

The nemo run Sub-command uses information in the run description files to provide the values for the following resources for you:

procs (number of processors)
pmem (memory per processor)
walltime (maximum run time for job)

so you should not include them in your PBS resources section.

Some HPC systems may require or suggest that you use a nodes=n:ppn=p resource request, where n is the number of nodes, and p is the number of processors per node. nemo run will calculate n for you based on the values you give for p, MPI decomposition, and XIOS servers. So, the value you use for n is unimportant because it will be replaced in the NEMO.sh script in the temporary run directory with the value that is appropriate for the run. For example, the following run description file entries:

MPI decomposition: 3x4
...
output:
  ...
  XIOS servers: 1
...
PBS resources:
  - nodes=n:ppn=12

will result in a #PBS -l nodes=2:ppn=12 directive being included in the NEMO.sh script.

`modules to load` Section

The optional modules to load section of the run description file contains a list of HPC environment modules for which module load commands will be added to the NEMO.sh script in the temporary run directory. You will need to use this section if you are running on an HPC system that uses environment modules; otherwise, you can omit the modules to load section.

An example of a modules to load section is:

modules to load:
  - StdEnv/2020
  - netcdf-fortran-mpi/4.5.2
  - python/3.9.6
  - nco/4.9.5

You will need to determine the specific list of modules to load and how to spell them for the HPC system that you are running on. In general, you will probably need to load modules for:

the compiler you used to build NEMO and XIOS
the netcdf, netcdf-fortran, and hdf5 libraries that you used to build NEMO and XIOS
the nco library to make the ncks available for the nemo deflate Sub-command (if you are not using XIOS-2 on-the-fly deflation)
the Python language

NEMO-3.6 Run Description File

Example Run Description YAML Files for NEMO-3.6

Basic Example

Example for Runs on computecanada.ca Systems

Basic Run Configuration

paths Section

grid Section

forcing Section

Atmospheric Forcing File Checks

restart Section

namelists Section

output Section

vcs revisions Section

PBS resources Section

modules to load Section

Example for Runs on `computecanada.ca` Systems

`paths` Section

`grid` Section

`forcing` Section

`restart` Section

`namelists` Section

`output` Section

`vcs revisions` Section

`PBS resources` Section

`modules to load` Section