CorteXlab's Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
# FIT/CorteXlab 103

In this tutorial we go one step deeper into the process of running an experiment on FIT/CorteXlab, this time starting from an empty GNU Radio project on your computer.

For practical purposes, we will use a readily available GNU Radio example instead of starting from a clean sheet, but this will introduce the same procedure you'll use when you want to run your own project on FIT/CorteXlab. We will also briefly introduce how to add custom C++ blocks to your project.

This time, the division of resources is slightly different and will be according to the table below:

^ Group      ^ SSH login       ^ Nodes to use ^ TCP Port ^
| 1          | tuto1           | node 3       | 6663     |
| 2          | tuto2           | node 4       | 6664     |
| 3          | tuto3           | node 7       | 6665     |
| 4          | tuto4           | node 8       | 6666     |
| 5          | tuto5           | node 6       | 6667     |
| 6          | tuto6           | node 23      | 6668     |
| 7          | tuto7           | node 27      | 6669     |
| 8          | tuto8           | node 24      | 6670     |
| 9          | tuto9           | node 28      | 6671     |

__Henceforth you'll replace <number of group> by the number of your group read in the table above.__

##Setup

The purpose of this tutorial is to receive an ongoing OFDM transmission on a given USRP node. One transmitter node is set up for the whole group and each group will control a receiver node. Instead of starting from an empty project, we are going to use the GNU Radio examples for an OFDM transmitter and receiver. We are also going to change the ''channel estimation'' block such that it produces frequency channel estimations that can be seen in the fft-web interface.

### Start you virtual FIT/CorteXlab work environment

Let us start by firing up the virtual FIT/CorteXlab environment that you should've installed before, in the [[g5k-fit-school#prerequisites|Prerequisites]] section.

Assuming you have correctly followed the steps described in that section, you can now fire up VirtualBox

Log into the virtual machine with ''username:cxlbusr'' and with an empty password

Once the LXDE interface is fired up, open an ''LXTerminal'' from the ''Start Menu > System tools''

__For now we'll be working only on the virtual FIT/CorteXlab environment.__

### Making a copy of the OFDM receiver

Copy the file ''rx\_ofdm.grc'' located in ''/home/cxlbusr/cortexlab/build/gnuradio.git/gr-digital/examples/ofdm/'' to a new folder called ''tuto_ofdm'' in a place of your choice, for example ''˜/tasks/tuto\_ofdm/''. This folder will be your local task folder.

As you can guess ''rx\_ofdm.grc'' is an OFDM receiver, but a standalone one capable to run in simulation mode under GNU Radio.

For the moment, the file tree should look like that:
<code>
...
├── tuto_ofdm
│   └── rx_ofdm.grc
...
</code>

## Preparing the RX python script

The .grc file we copied from the examples folder were made to work in simulation mode, i.e. without actual transmissions and through a simulated channel. You can see this by examining the reception chain with GNU Radio Companion (GRC).

Lets start by opening ''rx\_ofdm.grc'' with GRC.

<code>
cxlbusr@debian-jessie-cortexlab:~/tasks/tuto_ofdm$ gnuradio-companion rx_ofdm.grc
</code>

The flowgraph starts with blocks like ''Random Source'', ''OFDM Transmitter'' and ''Channel Model'', as seen in the figure below. These blocks implement the whole transmitter chain as well as the simulated channel.

{{:virtualbox_cortexlab_v4_26_03_2018_12_21_27.png?700|}}

We will need to modify the flowgraph by replacing the transmitter and the simulated blocks by a UHD source block (''UHD: USRP Source'') to stream the waveforms from the USRP through the receiver chain.

Using GNU Radio Companion  we are going to open and edit the .grc file in order to use it with USRP nodes.

### rx_ofdm.grc

We are now going to proceed to change the RX OFDM graph into something compatible with FIT/CorteXlab.

Initially we need to prepare the whole graph to make it run under FIT/CorteXlab. The main requirement is that the graph runs without graphical user interface (GUI) and that it stops automatically in the end without requiring user intervention. Bear in mind that the code will run on a remote host and that the user has no direct access to it. Do not forget to remove all GUI interface elements from the graph if any.

Open the ''Options block'' (in the top left corner of the graph) and:
  - Switch the ''Generate Options'' from ''QT GUI'' to ''No GUI''
  - Change ''Run Options'' to ''Run to Completion''


The next thing to do is to replace encoding and transmission parts and to put a ''USRP Source'' block instead:
  - Remove the ''Random Source'', ''Stream to tagged stream'', ''OFDM Transmitter'' ''Channel Model'' and ''throttle'' blocks. The ''Throttle'' is not needed anymore since the ''USRP Source'' block will cadence our graph. 
  - Place a ''UHD: USRP Source'' block and connect it to the ''Schimdl and Cox OFDM synch.'' and ''Delay'' blocks.
  - Place a ''fft web'' block and connect it to the ''UHD: USRP Source'' block. This will be used to debug the signal into the RX chain.
  - Open the ''tag debug'' block and add the key filter "packet_num". It is highly recommended to add a filter in the ''Tag Debug'' block in order to avoid printing a huge amount of useless data. 

Next, we need to configure the USRP parameters. Open the ''UHD: USRP Source'' block. 
  - In the ''General'' tab, verify that the ''Samp Rate'' field contains ''samp\_rate'', the variable that controls the sample rate throughout the graph (we will set it later),
  - In the ''RF Options tab'', set the ''Ch0: Center Freq (Hz)'' to ''2.49e9'' (2.49 GHz),
  - Set the ''Ch0: Gain Type'' to ''Absolute (dB)'',
  - Set the ''Ch0: Gain Value'' to 20,
  - Set the antenna port ''Ch0: Antenna'' to ''TX/RX'' (''RX2'' can't be used in FIT/CorteXlab at the moment) and 
  - The ''Ch0: Bandwidth (Hz)'' parameter should remain at 0.

Then, we need to configure the fft web parameters. Open the ''fft web'' block and change the following parameters: 
  - Set the ''port'' to the port value for your group, available in the above table,
  - Set the ''sample rate'' to the variable ''samp\_rate'',

Finally, in the ''Variable'' block located in top of the flow graph, whose ID is ''samp\_rate'', set the ''Value'' field to ''2000000'' (2 MHz).

In the end your graph should look like this (The areas marked with a red square indicate the changed areas):

{{ :ofdm_rx_final.png?700 |}}

### Generate the python files

The last thing we need to do within GRC is to generate the python file that will be used as the experiment executable. Press the ''generate flow graph'' button and that's it! You now have the python file''rx\_ofdm.py'' in the same folder as the .grc file.

__Hint: the button is located in the toolbar, near the play button and looks like this ->__ {{screen_shot_2018-03-23_at_14.57.24.png?nolink&40}}

## Create the scenario

As we've seen before, the experiment description file is called `scenario.yaml` and will be loaded by the experiment scheduler to figure out which nodes and what executable will be used during the experiment. It also gives the necessary startup scripts and parameters that the user provides for his experiment.

### Example scenario

Here is a simple example of what our `scenario.yaml` file should look like:
<code>
# Example scenario description file
#
#   All lines starting with "#" and empty lines are ignored


# Scenario textual description
#   simple string (a one liner)
description: CorteXlab tutorial number 3

# Experiment maximum duration
#   Time after which the experiment is forced to stop
#   integer (seconds)
duration: 120

# Node list
#
#   format:
#
#   nodes:
#     (machine):
#       command: (entry point script relative to the task root)

nodes:
  node4:
    command: ./rx_ofdm.py
</code>

This file uses the yaml syntax and is self-documented.

### Adapting the example scenario

We'll need to change the scenario file in order to reflect the nodes that your group is going to use. Please refer to the table at the top for the information you're going to use. 

Use your preferred unix editor to edit the file and change ''node4'' to the node that __has been assigned to your group__. 

## Launch the experiment in FIT/CorteXlab

Up to now we've been working on the virtual FIT/CorteXlab environment. If everything went well, we should have on your virtual machine a file tree that looks like this:
<code>
...
├── tuto_ofdm
│   ├── rx_ofdm.py
│   ├── scenario.yaml
│   └── rx_ofdm.grc
...
</code>

The .grc files will not be used by CorteXlab but you can leave them in the same directory.

### Upload the files on airlock

Upload the tuto\_ofdm directory on Airlock. For example, on Linux, it will look like this: 
<code>
cxlbusr@debian-jessie-cortexlab:~/tasks/tuto_ofdm$ cd ..
cxlbusr@debian-jessie-cortexlab:~/tasks$ scp -r tuto_ofdm/ tuto<number of group>@airlock:~
</code>

Dont forget to replace <number of group> below by the number of your group!

### Create the task

Connect to Airlock through ssh as described [[access|here]]. For example : 
<code>
cxlbusr@debian-jessie-cortexlab:~/tasks$ ssh tuto<number of group>@airlock
</code>

You will find in your home directory the previously uploaded ''tuto\_ofdm'' directory.

Now, use the Minus CLI to create the task file:
<code>
tuto#@srvairlock:~$ minus task create tuto_ofdm
</code>

The success (or failure) of the creation will be printed on screen. The task file will be created at the same level and with the same name as the targeted experiment folder but with a `.task` suffix, in our case ''tuto\_ofdm.task''.

**Note:** You can get help on the Minus CLI at anytime with `minus -h`

### Book the testbed with OAR

As explained [[reserve|here]], we need to book the testbed with OAR in order to run our experiment. Once you have booked some nodes in the FIT/CorteXlab, you will be the exclusive user of those nodes; this will prevent any experimentation problems.

For the purposes of this tutorial, a shared reservation has been done in advance. This parent reservation's 'id' will be given to you by the tutorial instructors. We will need to create a child reservation inside of the parent one.

To reserve your nodes in the FIT/CorteXlab room, use this following command. Please replace ''mnode4'' with the node number assigned to your group. Bear in mind that the preceding letter 'm' should be kept! Also, replace the ''<id of container>'' by the number of the parent reservation.

<code>
tuto#@srvairlock:~/examples$ oarsub -t inner=<id of container> -l {"network_address in ('mnode4.cortexlab.fr')"}/nodes=1,walltime=1:00:00 -I
</code>

More documentation on oar can be found [[reserve|here]]. Among the reservation messages OAR outputs, the system will give you a reservation 'id'. __Be sure to write your's down__. It can be used for removing the job if necessary.

You can check if the job was properly created as well as monitor the current jobs in the [gantt web interface](http://xp.cortexlab.fr/drawgantt/).

### Submit the task 

Now, we have booked the testbed and we have a `.task` file containing our experiment. In order to run it, we need to submit it to the testbed scheduler.

To submit a task to the scheduler, use the Minus CLI:
<code>
tuto#@srvairlock:~$ minus task submit tuto_ofdm.task
</code>

On screen will be prompted the id of your task in the scheduler. Note it down so that you can easily retrieve your results or monitor the progress of the experiment.

### Observe the result 


You can check the status of your experiment through the testbed scheduler. To do so, use the Minus CLI:
<code>
tuto#@srvairlock:~$ minus testbed status
num total tasks:   2540
num tasks waiting: 0
num tasks running: 0
tasks currently running:
  (none)
</code>

The information given by this command enables you to deduce your experiment status. It's quite self-explanatory.

__Are you having problems with your task? ''minus log'' might help you debug your scenario and GNU Radio problems.__

## Checking the results

Once it's finished (we have set the duration to 2 minutes), Minus will take care of copying the results and output messages back to your home folder in srvairlock, so that you can analyze it. 

__For readability, all examples below consider task 15 with nodes node4 and node6. Please consider the values assigned to your group in the following.__

All results are stored by task number in the results folder, inside your home folder.

Go in this folder :
<code>
tuto#@srvairlock:~/tuto_ofdm$ cd ~/results
tuto#@srvairlock:~/results$ ls
task_15
tuto#@srvairlock:~/results$ cd task_15
tuto#@srvairlock:~/results/task_15$ ls
node4.tgz  node6.tgz
</code>

So we see that we have a folder for each task and inside each folder one compressed file per participant node. Let's extract one of those files and see what's inside:

<code>
tuto#@srvairlock:~/results/task_15$ tar -zxf node4.tgz
tuto#@srvairlock:~/results/task_15$ ls 
node4  node4.tgz  node6.tgz
tuto#@srvairlock:~/results/task_15$ cd node4
tuto#@srvairlock:~/results/task_15/node4$ ls
rx_ofdm.grc  scenario.yaml  stdout.txt   tx_ofdm.py
rx_ofdm.py   stderr.txt     tx_ofdm.grc
</code>

We see that all of the files we used to create the task are inside. The other two are:

  * ''stdout.txt'': all output messages from your GNU Radio python script are written here. These include GNU Radio messages as well as all "print"s you include in your code. Seeing the contents of this file is useful to assert a correct operation.
  * ''stderr.txt'': all error messages are printed here. If you see strange things on the ''stdout.txt'' or nothing at all, it might be interesting to take a look at the ''stderr.txt'' to debug your code.

Let's take a look inside ''stdout.txt'':

<code>
tuto#@srvairlock:~/results/task_15/node4$ less stdout.txt
</code>

You should get a long file that looks more or less like this:

<code>
linux; GNU C++ version 4.9.2; Boost_105500; UHD_003.010.git-33-g9401bdbd

-- Opening a USRP2/N-Series device...
-- Current recv frame size: 1472 bytes
-- Current send frame size: 1472 bytes
-- Detecting internal GPSDO.... Found an internal GPSDO
-- Setting references to the internal GPSDO
Using Volk machine: avx_64_mmx_orc
Press Enter to quit: 
----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 0  Source: n/a     Key: packet_num   Value: 114
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 96  Source: n/a     Key: packet_num   Value: 115
  Offset: 192  Source: n/a     Key: packet_num   Value: 116
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 288  Source: n/a     Key: packet_num   Value: 117
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 384  Source: n/a     Key: packet_num   Value: 118
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 480  Source: n/a     Key: packet_num   Value: 119
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 576  Source: n/a     Key: packet_num   Value: 120
----------------------------------------------------------------------

----------------------------------------------------------------------
Tag Debug: Rx Bytes
Input Stream: 00
  Offset: 672  Source: n/a     Key: packet_num   Value: 121
----------------------------------------------------------------------
</code>

Eventually, the output file can start with some decoding error looking like this :

<code>
INFO: Detected an invalid packet at item 432
INFO: Parser returned #f
</code>