Differences

This shows you the differences between two versions of the page.

--- scenario_file_format [2017/09/01 09:07] – created mimbert
+++ scenario_file_format [2023/08/29 15:29] (current) – Add example of passive keyword in new syntax cmorin
@@ Line 1: / Line 1: @@
 The scenario of a minus task is a file named ''scenario.yaml'', located at the toplevel of the task directory.
-The syntax of the file is [YAML](http://www.yaml.org/start.html), a popular file format. The full YAML specification is quite large but only a small subset of YAML is needed for scenario files, and this subset is very intuitive, and easily understandable just by looking at a ''scenario.yaml'' example.
+====== Syntax ======
+The syntax of the file is [YAML](http://www.yaml.org/start.html), a popular file format. The full YAML specification is quite large but only a small subset of YAML is needed for scenario files, and this subset is very intuitive, and easily understandable just by looking at a ''scenario.yaml'' example. See for example [[gnu_radio_benchmark_example#inspecting_the_example_task|the GNURadio benchmark example scenario]]. Otherwise, you can read the [[#Grammar]] section below to get a more formal description of the scenario grammar.
 Please bear in mind that YAML is indentation sensitive, just like python. YAML will follow whatever indentation system the scenario file uses, provided that it is consistent within the file. Note that in particular you should not mix spaces and tabs (beware, some text editors do that silently and need to be properly configured to avoid that). The suggested best practice is to use two spaces for indentation.
+Comments in YAML start with a ''#''
+The syntax of a yaml file may be checked with http://www.yamllint.com/
+====== Grammar ======
+We're progressively adding new capabilities to the CorteXlab platform. There is now two different syntaxes for the scenario file, the legacy syntax, where commands are run on the nodes, and the new syntax, where [[docker|docker]] containers are run on the nodes. Both syntaxes are allowed, even in the same scenario, but only one syntax is allowed per-node.
+The scenario.yaml file is structured as follows:
+  * It //MUST// contain a ''description'' key, value pair whose value is a string.
+  * It //MUST// contain a ''duration'' key, value pair, whose value is an integer, the max experiment duration expressed in seconds.
+  * It //MUST// contain a ''nodes'' key, whose value is a mapping (aka hash, or dictionnary). This mapping //MUST// contain:
+    * Nodes key, value pairs, whose keys are the node names (in the set ''node1'' to ''node40''), and whose values are mappings:
+      * which //MUST// contain **//ONLY ONE//** of the following:
+        * **(new syntax)** a ''container'' key, value pair, whose value is a mapping with the following key, value pairs:
+          * ''image'' //MANDATORY// a string, the name of the docker image, which must be accessible on a public docker repository (such as the docker hub). See [[docker images]]
+          * ''command'' //OPTIONAL// a string, the command to run in the container, which overrides the image default command (see https://docs.docker.com/engine/reference/run/#cmd-default-command-or-options https://docs.docker.com/engine/reference/builder/#cmd)
+          * ''exec'' //OPTIONAL// a string or array of strings: additionnal commands which are executed in the container (see https://docs.docker.com/engine/reference/commandline/exec/)
+          * ''name'' //OPTIONAL// a string for specifying the container's name. If not given, a default name will be automatically generated, with the minus task number and the container rank on the node.
+        * **(old syntax)** a ''command'' key, value pair, whose value is the command line string which will be run on the corresponding node. Note that any filesystem path may be absolute or relative. If relative, it will be relative to the task top directory on the node. Note also that minus does NO magic with filesystem permissions, thus if a script or binary is to be executed, it needs to have executable permissions set before creating the task. The ''command'' may also, if needed, contain a list of commands to be executed, instead of a single command. All the commands will be executed in parallel on the node. In this case, the syntax is multiline: <code>
+nodes:
+  node1:
+    command:
+    - sleep 10000
+    - ./my_script
+</code> In the usual case where only one command is given, its stdout/stderr go to files ''stdout.txt'' and ''stderr.txt''. In the case where more than one commands are given, their stdout/stderr go to files ''stdout<X>.txt'' and ''stderr<X>.txt'', where X is the command index (starting from zero).
+      * which //MAY// contain a ''passive'' key, value pair, whose value is a boolean (default is ''false''). A minus task will terminate when the first of the following conditions occur: its //duration// is exceeded or all its //active// nodes terminate.
+An example scenario (new syntax):<code>
+description: Example of new syntax with docker containers
+duration: 120
+nodes:
+  node3:
+    container:
+    - image: m1mbert/cxlb-gnuradio-3.8:1.0
+    - image: m1mbert/cxlb-gnuradio-3.8:1.0
+      command: /usr/sbin/sshd -p 2223 -D
+  node4:
+    container:
+      image: m1mbert/openbts-xenial:1.1
+    passive: true
+</code>Note that in this example, on node3, two containers are instanciated, based on the same image. The first container does not specify a ''command'', so the default command of the image is used, which is an ssh daemon listening on port 2222. The second container overrides the default command with an ssh daemon listening on port 2223 (if we try to instanciate two identical containers from this image, both ssh daemons will try to bind to port 2222 and the second one will fail)
+An example scenario (old syntax):<code>
+description: Dummy scenario example
+duration: 60
+nodes:
+  node1:
+    command: sleep 1000
+    # a bash command
+    passive: true
+  node2:
+    command: ./myscript
+    # relative path to a script embedded in the task
+  node3:
+    command: gnuradio-config-info -v
+    # calling a gnuradio executable which is in the $PATH
+  node4:
+    command: /cortexlab/toolchains/current/share/gnuradio/examples/uhd/usrp_wfm_rcv.py
+    # absolute path to a script or binary
+</code>