Skip to content

Getting Started

If you are new to SURF-NEMO, this page will walk you through the basic concepts, show you how to run, monitor and customize your first experiment.

The SURF Command-Line Interface

The SURF Command-Line Interface (CLI) is the primary tool for running and monitoring your SURF experiments.

To use the CLI, simply open a Terminal and type the surf_nemo command, followed by a list of options:

1
surf_nemo [OPTIONS]

For example, display the help message by typing:

1
surf_nemo --help

The most important options that you need to know are:

Argument Short Form Description Required Examples
--output-dir -o Absolute path to the output directory where the Experiment Directory will be created. Yes -o $SURF_NEMO_DIR
--config-file -c Absolute path to the input SURF Configuration File. Yes -c $SURF_NEMO_DIR/configurations/config_base.json
--help -h Display the help message and exit. No -h
--version -v Display the program's version and exit. No -v

Most of the time, you will run SURF as follows:

1
surf_nemo --output-dir [/ABSOLUTE/PATH/TO/OUTPUT/DIRECTORY] --config [/ABSOLUTE/PATH/TO/CONFIGURATION/FILE]

More advanced options (less frequently used) are:

Argument Short Form Description Required Example(s)
--interactive -i Enable interactive mode (e.g. for debugging purposes, or for advanced use-cases such as Recompiling NEMO). No -i
--gui -g Display the SURF GUI to configure the experiment (This option is not supported yet! Stay tuned). No -g
--image -I Docker image tag. It can be useful if you have a custom SURF-NEMO version (see Modify the source code and create a custom image). No -I surf_nemo:X64-2.0.0-recompiled

Running your First Experiment

Once SURF-NEMO is installed, the best way to get started is to run a pre-configured case study.

In this example, you will implement a high-resolution model in the Gulf of Taranto, located in the northern Ionian Sea. The simulation is a hindcast starting on 5 October 2014 at 00:00 and running until 7 October 2014 at 24:00.

Gulf of Taranto. Gulf of Taranto.

To run this case study, you have two options:

  1. Use automated data download: It is the most common option in real-world applications, ideal for beginner users. It requires a reliable internet connection as well as personal credentials for online data providers. It may take a few minutes to download the necessary data.
  2. Use local data: It ensures full reproducibility by avoiding the data download from third-party sources. Requires to download the necessary data manually from the SURF website. It is recommended only for advanced users.
  1. In order to organize your work, we recommend to create the following directory structure: 
    1
    mkdir -p $SURF_NEMO_DIR/configurations
  2. Download the configuration file for the Gulf of Taranto case study: 
    1
    wget -P $SURF_NEMO_DIR/configurations https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/case_studies/gulf_taranto/config_base_gulf_taranto.json

  3. Create a personal profile on the following data providers:

    Then, upload your ECMWF and CMEMS credentials into the configuration file that you downloaded in the previous step ($SURF_NEMO_DIR/configurations/config_base_gulf_taranto.json). Please refer to the Select and download input data products section for more details.

  4. You’re all set to run the experiment:

    1
    surf_nemo --output-dir $SURF_NEMO_DIR --config $SURF_NEMO_DIR/configurations/config_base_gulf_taranto.json
  1. In order to organize your work, we recommend to create the following directory structure: 
    1
2
    mkdir -p $SURF_NEMO_DIR/configurations
mkdir -p $SURF_NEMO_DIR/datasets
  2. Download the configuration file for the Gulf of Taranto case study: 
    1
    wget -P $SURF_NEMO_DIR/configurations https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/case_studies/gulf_taranto/config_expert_gulf_taranto.json

  3. Download and extract the input datasets for this case study: 

    1
2
    wget -P $SURF_NEMO_DIR/datasets https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/case_studies/gulf_taranto/gulf_taranto_input.tar.gz
tar -xzf $SURF_NEMO_DIR/datasets/gulf_taranto_input.tar.gz -C $SURF_NEMO_DIR/datasets

  4. You’re all set to run the experiment:

    1
    surf_nemo --output-dir $SURF_NEMO_DIR --config $SURF_NEMO_DIR/configurations/config_expert_gulf_taranto.json

Monitoring your Experiment

To monitor the progress and diagnose the status of your experiment, you have several options:

  1. On-screen status
    Check the status provided on-screen during execution. This will directly display the progress of the experiment and let you know what part of the SURF workflow is currently running.

Experiment status on-screen monitor Experiment status on-screen monitor.

  1. SURF Log files
    Check the SURF log files, which are automatically generated and stored in the experiment directory under the logfiles directory. The log files are designed for maximum verbosity, and in case of any error, they will be truncated and ending with the error traceback.

  2. NEMO simulation logs
    If you are running the simulation step, it can be useful to monitor directly the NEMO log files, available in the simulation directory.

Viewing the Results

Once the experiment is complete, you can find all the automatically generated visualizations in the postprocessing directory.

To know more about how to customize the results, please refer to the How To section.

Example of high-resulution temperature forecast in the Gulf of Taranto. Example of high-resolution temperature forecast in the Gulf of Taranto.

Run a Custom Experiment

To run a custom experiment, you must create a new SURF Configuration File and pass it to the surf_nemo command. You have three options:

  1. Modify an Existing File
    Use the configuration file from the Gulf of Taranto case study, and adapt it to your use case.

  2. Use a Base Template
    If you want to build a configuration from scratch, download the empty Base Configuration File template: 

    1
    wget -P $SURF_NEMO_DIR/configurations https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/config_base.json

  3. Use the Expert Configuration (Advanced Users)
    If you need more advanced customization options for your simulation, download the empty Expert Configuration File template: 

    1
    wget -P $SURF_NEMO_DIR/configurations https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/config_expert.json
    Alternatively, you can get the expert configuration file pre-configured for the Gulf of Taranto case study: 
    1
    wget -P $SURF_NEMO_DIR/configurations https://www.surf-platform.org/repository/surf_nemo/surf_nemo_2.0.0/case_studies/gulf_taranto/config_expert_gulf_taranto.json