User Tools

Site Tools


experimental_workflow

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
Next revisionBoth sides next revision
experimental_workflow [2015/02/26 14:37] – created mimbertexperimental_workflow [2015/02/26 15:11] lcardoso
Line 5: Line 5:
   - **submit experimental tasks**, with //Minus//   - **submit experimental tasks**, with //Minus//
  
-===== Book nodes with //OAR// =====+===== Book nodes with OAR =====
  
-Only one person can use the whole CorteXlab testbed at the same time. The //OAR// scheduler is used to book nodes on the platform. As soon as you book one or more nodes, the CorteXlab room is reserved for your usage.+To avoid cross interference between multiple experiments, only one person can use the whole CorteXlab testbed at the same time. The //OAR// scheduler is used to book nodes on the platform. As soon as you book one or more nodes, the CorteXlab room is reserved for your usage during the requested time.
  
 The role of //OAR// is to schedule node reservations. It manages **jobs** associated with users, which has a start time, a duration (walltime), and uses some resources (CorteXlab nodes). The role of //OAR// is to schedule node reservations. It manages **jobs** associated with users, which has a start time, a duration (walltime), and uses some resources (CorteXlab nodes).
Line 13: Line 13:
 The principle of operation of CorteXlab is that users submit jobs to OAR. When the job starts, the user gets exclusive access to the platform, and inside an OAR job, the user can perform (interactively, or in batch) one or several experiments. The principle of operation of CorteXlab is that users submit jobs to OAR. When the job starts, the user gets exclusive access to the platform, and inside an OAR job, the user can perform (interactively, or in batch) one or several experiments.
  
-A basic example to submit an OAR interactive job requesting all available nodes:+A basic example to submit an OAR interactive job requesting all available nodes, for the default duration (which is 2 hours):
  
 <code>$ oarsub -I -l nodes=BEST</code> <code>$ oarsub -I -l nodes=BEST</code>
 +
 +This command will wait for the resources to be available, and as soon as they are, a job is allocated, is started, and a subshell is instanciated where you can work on experiments. As soon as the subshell is closed, the job ends. (It can be usefull to work in a [screen](https://www.gnu.org/software/screen/) session to avoid loosing jobs in case of network disconnection).
 +
 +By default OAR submissions are scheduled as soon as possible. It is also possible to ask for an OAR //reservation// where you choose the date at which the job will be scheduled.
  
 ===== Submit experimental tasks ===== ===== Submit experimental tasks =====
  
-===== Interactive or batch experiments =====+Once inside a running OAR job, you can submit one or several **Experimental tasks** by using minus:
  
 +<code>$ minus task submit <task_file_name></code>
 +
 +All submitted minus tasks are enqueued in a simple FIFO and will execute sequentially.
 +
 +Of course, to be able to schedule experimental tasks, you need to have a running OAR job.
 +
 +When your OAR job ends, all running and remaining tasks are aborted and removed for the FIFO.
 +
 +===== Interactive or batch experiments =====
  
 +When submitting or reserving an OAR job, you have the choice between interactive or batch:
 +  - In //interactive// mode, an interactive subshell is instanciated where you work interactively. The job ends as soon as the interactive shell is closed
 +  - In //batch// mode, you pass to command ''oarsub'' the name of an executable file which will be run as soon as the job starts. It is thus possible to write scripts which submit several experimental tasks (for example, to explore the variation of experimental parameters), and to submit OAR jobs which will be run automatically and asynchrously (even during the night).
experimental_workflow.txt · Last modified: 2015/09/16 10:39 by trisset

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki