Differences

This shows you the differences between two versions of the page.

--- gnu_radio_lora_dynamic_phy_layer [2021/06/15 17:09] – First draft, TODO : Run experiment section and explanation of the project structure and files pesteve
+++ gnu_radio_lora_dynamic_phy_layer [2021/06/23 15:58] (current) – [GNU Radio LoRa dynamic PHY layer] pesteve
@@ Line 1: / Line 1: @@
 # GNU Radio LoRa dynamic PHY layer
 This tutorial executes a LoRa transmission between a node and a gateway using docker in CorteXlab, transmitting dummy packets between them.
@@ Line 5: / Line 7: @@
 This implementation stands on the original LoRa version made by [[https://www.epfl.ch/labs/tcl/resources-and-sw/lora-phy/|EPFL]]
-## Pull the docker image
+In this tutorial, we will first deploy on CorteXlab a basic upper layer connected to the LoRa physical layer. After showing the functioning of this simple scenario, we will then pull the docker image in order to modify it, add a basic functionnality on the upper layer and redeploy it on CorteXlab.
-In order to follow this tutorial, we will use the docker image found on [Dockerhub](https://hub.docker.com/repository/docker/estevep/lora_dyn_phy-cxlb), which includes the CorteXlab toolchain, alongside a [GitHub repository](https://github.com/AmauryPARIS/LoRa_PHY_Cxlb/) properly configured.
+## Basic upper layer/original scenario
-Thanks to docker, it is possible to deploy a suitable environment on CorteXlab, with gnuradio and needed software.
+### Docker
-The necessary commands have already been executed on the given image, so it is not necessary to modify it or to commit and push a new image. However, if you want to create new scenarios or applications, follow the guidelines in the section below. If not, go directly to the [[gnu_radio_lora_dynamic_phy_layer#create_sc|create scenario]] section.
+In order to follow this tutorial, we will use the docker image found on [Dockerhub](https://hub.docker.com/repository/docker/estevep/lora_dyn_phy-cxlb), which includes the CorteXlab toolchain, alongside a [GitHub repository](https://github.com/AmauryPARIS/LoRa_PHY_Cxlb/) providing the LoRa physical layer properly configured and a basic upper layer.
-## Create your docker image
+Thanks to docker, it is possible to deploy a suitable environment on CorteXlab, with gnuradio and needed software.
-Using docker, we will enable on your laptop an environment suitable for Cortexlab with gnuradio and needed software.
+### Create the scenario
-Your goal is to develop this image (with new software and files) so you can run your experiment. It will only modify the environment of the image and not your laptop. Then you will be able to save and deploy the new image on cortexlab, without having to re-install everything and dealing with software compatibility and versioning.
-Therefore, you can retrieve and run the following image.
+In order to deploy this suitable environment, you will have to create the following files and folders.
-<code>
-you@yourpc:~$ docker pull estevep/lora_dyn_phy-cxlb
-you@yourpc:~$ docker run -dti --net=host --expose 2222 --privileged [--name CONTAINER_NAME] lora_dyn_phy-cxlb
-</code>
-Access the docker container using ssh :
-<code>
-you@yourpc:~$ ssh -Xp 2222 root@localhost
-</code>
-Add this option to the run command if you want to use a USRP plugged in with a USB port on you local machine : '' -v /dev/bus/usb:/dev/bus/usb ''.
-Also, a new host key is created when running the container. The following command could be necessary to add the new key to the known hosts list before using ssh.
-<code>
-you@yourpc:~$ ssh-keygen -f "/home/[YOUR NAME]/.ssh/known_hosts" -R "[localhost]:2222"
-</code>
-When in the container, you can modify any of the given files and python scripts. To understand more depply the structure of the project and the role of all the python scripts, see **TODO : create the section explaining each file and add a reference here**.
-We can test it on our local machine before sending it to cortexlab (it won't work if you don't have an USRP available).
-Once you are satisfied with your modifications and the scripts are working properly, you can exit the container with Ctrl + d. Now, let's save our updated container as a new docker image. We can then push it on dockerhub (dockerhub is a repository of docker images, you need to create an account on it).
-<code>
-you@yourpc:~$ docker ps -a
-you@yourpc:~$ docker commit [CONTAINER ID] [NEW IMAGE NAME]
-you@yourpc:~$ docker tag [NEW IMAGE NAME] [DOCKER USERNAME]/[NEW IMAGE NAME]
-you@yourpc:~$ docker login
-you@yourpc:~$ docker push [DOCKER USERNAME]/[NEW IMAGE NAME]
-</code>
-## Create the scenario
 For the following, the ''task'' will refer to the instructions given by Airlock, the scenario file and minus. The ''experiment'' will refer to the LoRa transmissions.
@@ Line 105: / Line 72: @@
 </code>
-## Access Airlock
+### Access Airlock
 You can now access the Airlock SSh server that will allow you to manage your task.
@@ Line 113: / Line 80: @@
 </code>
-## Creating the task file
+### Create the task file
-On airlock, before submitting the task to the nodes, we need first to put the task into a format that can be readily understood by Minus. //See [here|gnu_radio_docker_benchmark_example#creating_the_task_file] for more information on minus.//
+On airlock, before submitting the task to the nodes, we need first to put the task into a format that can be readily understood by Minus. __See [[gnu_radio_docker_benchmark_example#creating_the_task_file|here]]__ for more information on minus.
+<!-- Let's prepare the task, but first we need to go back to the folder containing the task: -->
+<!-- <code>
+you@srvairlock:~/ cd /cortexlab/homes/[YOUR CORTEXLAB USERNAME]/workspace/
+</code> -->
+<!-- And now, instruct Minus to create a task file: -->
 <code>
@@ Line 124: / Line 99: @@
 </code>
-## Submitting the task
+<!-- And now we have a new file called ''task\_lora.task'' which is ready to be submitted. Warning, do not leave a slash after the directory task name (i.e. ''task_lora'' ) or the command will fail. -->
+### Submit the task
 Now we need to give the task to Minus, so that it can operate its magic.
@@ Line 173: / Line 150: @@
   * ''tasks currently running'': This is a detailed list of tasks currently running.
-## Run the experiment
-//Work in progress//
+### Run the experiment
+Now the node we want to use are up and running with the docker image we set up earlier. We can access them with ssh to run our experiment. To do so, open a new terminal per node you want to use, here two, access Airlock and run the following command, here for node 14:
+<code>
+you@srvairlock:~/workspace$ ssh -p 2222 root@mnode14
+</code>
+You may need to wait a few seconds so that the task is running before being able to connect to the nodes.
+You are now connected to the node with the docker environment you set up earlier.
+In order to run this example you should launch 4 terminals: 2 for each USRP. Therefore, use the same `ssh` command to connect to the node 14 or 16 on each of the 3 remaining terminals.
+The part we are intersted in can be found in `/root/LoRa_PHY_Cxlb.git`, and more specifically in the `gr-lora_sdr/apps` subfolder.
+<code>
+root@mnode14:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# ls -1
+CMakeLists.txt
+'Hier blocks'
+LoRa_node.grc
+README.txt
+lora_dyn_node.py
+setpaths.sh
+udp_BS.py
+udp_Node.py
+</code>
+Let's go over each file in this folder:
+* `CMakeLists.txt`: build file
+* `Hier blocks`: folder containing the .grc files for the hierarchical blocks of the LoRa receiver and transmitter
+* `setpaths.sh`: script that add the library and python paths for the current shell process. Already executed during the creation of the docker container.
+`LoRa_node.grc` and the corresponding python file `lora_dyn_node.py` contains a LoRa physical layer connected to a dynamic udp JSON interface.
+In order to show its utility, two python scripts are provided:
+* `udp_Node.py`: basic upper layer for a node, connected to the `lora_dyn_node` script through its udp JSON interface. Sends the message given by the user
+* `udp_BS.py`: basic upper layer for a base station, connected to the `lora_dyn_node` script through its udp JSON interface. Sends an acknoledgment.
+:warning: Both `udp_BS.py` and `udp_Node.py` have to be run with python3, whereas `lora_dyn_node` has to be run with python2.
+Enter the following commands on the specified terminal to begin the experiment:
+<code>
+# From node 14 - on the 1st terminal, refered to as 14-a
+root@mnode14:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# python2 lora_dyn_node.py
+</code>
+<code>
+# From node 14 - on the 2nd terminal, refered to as 14-b
+root@mnode14:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# python3 udp_Node.py
+</code>
+<code>
+# From node 16 - on the 1st terminal, refered to as 16-a
+root@mnode16:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# python2 lora_dyn_node.py
+</code>
+<code>
+# From node 16 - on the 2nd terminal, refered to as 16-b
+root@mnode16:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# python3 udp_BS.py
+</code>
+On the 14-b terminal and the 16-b one, first choose your UDP TX and RX port. On CorteXlab using the default ports (6788 and 6790) for both the base station and the node will work since they are connected to different computers.
+Then enter one of the following keywords to change the corresponding parameter if needed on the 14-b terminal.
+```
+CR - Coding Rate
+SF - Spreading Factor
+GTX - Gain for TX chain
+GRX - Gain for RX chain
+FTX - USRP frequency for TX chain
+FRX - USRP frequency for RX chain
+MSG - Data to transmit
+```
+Enter the new value, then type `send` to send this new parameter to the physical layer. If a new `MSG` value is sent to the physical layer it is transmitted to the base station, which should respond with an acknowledgment.
+For example, try running the following commands on terminal 14-b in order to modify the RX gain of the node:
+<code>
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: print
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: GRX
+    Enter the new value of the Gain for RX chain :15
+    Command added to list : {'GRX': '15'}
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: send
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: print
+</code>
+Check out the results on terminal 14-a:
+<code>
+    root@mnode14:~/LoRa_PHY_Cxlb.git/gr-lora_sdr/apps# python2 lora_dyn_node.py
+    [INFO] [UHD] linux; GNU C++ version 8.3.0; Boost_106700; UHD_3.15.0.0-124-geb448043
+    [INFO] [USRP2] Opening a USRP2/N-Series device...
+    [INFO] [USRP2] Current recv frame size: 1472 bytes
+    [INFO] [USRP2] Current send frame size: 1472 bytes
+    [WARNING] [MULTI_USRP] Setting IQ imbalance compensation is not possible on this device.
+    set_min_output_buffer on block 8 to 10000000
+    /root/.gnuradio/prefs/vmcircbuf_default_factory: No such file or directory
+    vmcircbuf_createfilemapping: createfilemapping is not available
+    Press Enter to quit: TX UDP : {"FTX": "910e6", "FRX": "900e6"}
+    TX UDP : {"print": ""}
+    FTX = 910000000.0
+    FRX = 900000000.0
+    GTX = 30.0
+    GRX = 20.0
+    TX UDP : {"GRX": "15"}
+    TX UDP : {"print": ""}
+    FTX = 910000000.0
+    FRX = 900000000.0
+    GTX = 30.0
+    GRX = 15.0
+</code>
+The receiving gain should indeed be set to 15.
+Now enter the following commands in order to send a message `test` from the node to the base station
+<code>
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: MSG
+    Enter the new value of the Data to transmit :test
+    Command added to list : {'MSG': 'test'}
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: send
+</code>
+Check out the results on terminal 14-a.
+<code>
+    TX UDP : {"MSG": "test"}
+    U--------Header--------
+    Payload length: 8
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: ACK-test
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-2.77802', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-49.6229', 'pay_len': '8', 'SNR': '46.8449', 'msg': 'ACK-test', 'crc_valid': 'True', 'SF': '7'}
+</code>
+Let's try to understand it:
+* The first line confirms that the message "test" has indeed been sent.
+* The node then waits for an acknoledgment from the base station and enters receiving mode.
+* The next ones concern the received acknoldgement. Before understanding them, check out the other terminals, starting with terminal 16-a.
+<code>
+    --------Header--------
+    Payload length: 4
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: test
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-3.02165', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-94.9842', 'pay_len': '4', 'SNR': '91.9625', 'msg': 'test', 'crc_valid': 'True', 'SF': '7'}
+    TX UDP : {"MSG": "ACK-test"}
+</code>
+This confirms that the physical layer received a message "test", and the RX UDP line shows the information sent via UDP protocol to the upper layer on terminal 16-b.
+<code>
+    New message :
+        msg_avg : 0
+        snr_avg : 0
+        CR : 4
+        MSG_Energy : -3.02165
+        err_corrected : 0
+        err_detected : 0
+        Noise_Energy : -94.9842
+        pay_len : 4
+        SNR : 91.9625
+        msg : test
+        crc_valid : True
+        SF : 7
+    Sending Acknowledgement
+</code>
+On this last terminal, you should first see these information, and then the reaction of the base station upper layer: sending an acknoledgment.
+This acknoledgment is then sent by the physical layer as indicated by the TX UDP line on terminal 16-a, and finally received by the physical layer of the node:
+<code>
+    U--------Header--------
+    Payload length: 8
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: ACK-test
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-2.77802', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-49.6229', 'pay_len': '8', 'SNR': '46.8449', 'msg': 'ACK-test', 'crc_valid': 'True', 'SF': '7'}
+</code>
+## Customized experiment
+This experiment is quite basic, even though it demonstrates how to connect an upper layer to the dynamic LoRa physical layer and to change the physical layer parameter. The node only sends a message and waits for eternity for an acknoledgment of the base station. The purpose of the physical layer is to be used by more complex upper layers.
+In this part, we will see how to modify the upper layer and redeploy it on CorteXlab. If you want to develop your own upper layer, you can refer to this tutorial to deploy it.
+### Create your docker image
+Using docker, we will enable on your laptop an environment suitable for Cortexlab with gnuradio and needed software.
+Your goal is to modify the upper layer, and in this tutorial more specifically the base station upper layer. It will only modify the environment of the image and not your laptop. Then you will be able to save and deploy the new image on cortexlab, without having to re-install everything and dealing with software compatibility and versioning.
+Therefore, you can retrieve and run the image deployed on CorteXlab in the first part of the tutorial.
+<code>
+you@yourpc:~$ docker pull estevep/lora_dyn_phy-cxlb
+you@yourpc:~$ docker run -dti --net=host --expose 2222 --privileged [--name CONTAINER_NAME] lora_dyn_phy-cxlb
+</code>
+Access the docker container using ssh:
+<code>
+you@yourpc:~$ ssh -Xp 2222 root@localhost
+</code>
+Add this option to the run command if you want to use a USRP plugged in with a USB port on you local machine: '' -v /dev/bus/usb:/dev/bus/usb ''.
+Also, a new host key is created when running the container. The following command could be necessary to add the new key to the known hosts list before using ssh.
+<code>
+you@yourpc:~$ ssh-keygen -f "/home/[YOUR NAME]/.ssh/known_hosts" -R "[localhost]:2222"
+</code>
+#### Customize the UDP Base Station
+Here, we will just add a simple functionnality to the base station. If the SNR is higher than a threshold, then the GTX of the BS will be reduced, and if it is lower than a second threshold, then the GTx will be augmented.
+This simulates a simple logic behavior: if the receiving SNR is low, then the acknoledgment should be emitted with a high gain in order to be correctly received by the node. On the contrary, if the receiving SNR is very high, then there is no need to emit with a high gain.
+In order to do so, we will modify the `udp_BS.py` file: replace lines 48 to 53 included with the following code:
+```python
+    # Tx gain settings
+    snr = received_msg["SNR"]
+    # print("SNR : " + snr)
+    if float(snr) > 80:
+        print("High SNR - Decrease Tx Gain to 10")
+        cmd_dict = {
+                        "GTX": "10",
+                    }
+    elif float(snr) < 60:
+        print("Low SNR - increase Tx Gain to 30")
+        cmd_dict = {
+                        "GTX": "30",
+                    }
+    # Send ack
+    print("Sending Acknowledgement")
+    socket_tx = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+    socket_tx.connect((IP_ADDRESS, PORT_NO_TX))
+    cmd_dict.update({ "MSG": str("ACK-" + str(received_msg["msg"])) })
+```
+In order to edit `udp_BS.py` you can install vim in your docker container:
+<code>
+  root@yourpcwdocker:~# apt install vim
+</code>
+We can test it on our local machine before sending it to cortexlab (it won't work if you don't have an USRP available).
+Once you are satisfied with your modifications and the scripts are working properly, you can exit the container with Ctrl + d. Now, let's save our updated container as a new docker image. We can then push it on dockerhub (dockerhub is a repository of docker images, you need to create an account on it).
+<code>
+you@yourpc:~$ docker ps -a
+you@yourpc:~$ docker commit [CONTAINER ID] [NEW IMAGE NAME]
+you@yourpc:~$ docker tag [NEW IMAGE NAME] [DOCKER USERNAME]/[NEW IMAGE NAME]
+you@yourpc:~$ docker login
+you@yourpc:~$ docker push [DOCKER USERNAME]/[NEW IMAGE NAME]
+</code>
+### Deploy the experiment on CorteXlab
+In order to send this environment on CorteXlab, follow the same procedure as in subsections [[gnu_radio_lora_dynamic_phy_layer#create_the_scenario|Create the scenario]] to [[gnu_radio_lora_dynamic_phy_layer#run_the_experiment|Run the experiment]]. Just **change the docker image name in `scenario.yaml`** from `estevep/lora_dyn_phy-cxlb:latest` to `[DOCKER USERNAME]/[IMAGE NAME]`.
+### Results
+You should now follow the same steps as in the first part to connect to the nodes 14 and 16 with four terminals, refered to as 14-a to 16-b, and execute `lora_dyn_node.py`, `udp_Node.py` and `udp_BS.py`.
+#### Default GTX
+First send a message with the default GTX on terminal 14-b.
+<code>
+Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: MSG
+Enter the new value of the Data to transmit :test
+Command added to list : {'MSG': 'test'}
+Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: send
+</code>
+You should now see the following answer on terminal 14-a.
+<code>
+    TX UDP : {"MSG": "test"}
+    U--------Header--------
+    Payload length: 8
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: ACK-test
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-14.0058', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-75.117', 'pay_len': '8', 'SNR': '61.1112', 'msg': 'ACK-test', 'crc_valid': 'True', 'SF': '7'}
+    TX UDP : {"MSG": "test"}
+</code>
+And the following ones on terminal 16-a and 16-b
+- a
+<code>
+    --------Header--------
+    Payload length: 4
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: test
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-4.2716', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-95.7583', 'pay_len': '4', 'SNR': '91.4867', 'msg': 'test', 'crc_valid': 'True', 'SF': '7'}
+    TX UDP : {"GTX": "10", "MSG": "ACK-test"}
+</code>
+<code>
+    New message :
+        msg_avg : 0
+        snr_avg : 0
+        CR : 4
+        MSG_Energy : -4.2716
+        err_corrected : 0
+        err_detected : 0
+        Noise_Energy : -95.7583
+        pay_len : 4
+        SNR : 91.4867
+        msg : test
+        crc_valid : True
+        SF : 7
+    High SNR - Decrease Tx Gain to 10
+    Sending Acknowledgement
+</code>
+In this case, the SNR is higher than 90 so the base station GTX is set to 10 instead of 30.
+The acknoledgment is still received correctly by the node as shown in terminal 14-a.
+#### Simulate distance
+In order to simulate a less ideal situation and reduce the SNR, you can set the node GTX to 5 in terminal 14-b and send a new message.
+<code>
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: GTX
+    Enter the new value of the Gain for TX chain :5
+    Command added to list : {'GTX': '5'}
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: MSG
+    Enter the new value of the Data to transmit :test5
+    Command added to list : {'GTX': '5', 'MSG': 'test5'}
+    Enter the parameter OR "send" to send all stored commands OR "print" to display the current parameters: send
+</code>
+Look at the answers in the other terminals:
+* 14-a
+<code>
+    TX UDP : {"GTX": "5", "MSG": "test5"}
+    U--------Header--------
+    Payload length: 9
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: ACK-test5
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-3.34723', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-85.2143', 'pay_len': '9', 'SNR': '81.8671', 'msg': 'ACK-test5', 'crc_valid': 'True', 'SF': '7'}
+</code>
+* 16-a
+<code>
+    --------Header--------
+    Payload length: 5
+    CRC presence: 1
+    Coding rate: 4
+    Header checksum valid!
+    msg: test5
+    CRC valid!
+    RX UDP :{'msg_avg': '0', 'snr_avg': '0', 'CR': '4', 'MSG_Energy': '-25.0943', 'err_corrected': '0', 'err_detected': '0', 'Noise_Energy': '-95.796', 'pay_len': '5', 'SNR': '70.7018', 'msg': 'test5', 'crc_valid': 'True', 'SF': '7'}
+    TX UDP : {"GTX": "30", "MSG": "ACK-test5"}
+</code>
+* 16-b
+<code>
+    New message :
+        msg_avg : 0
+        snr_avg : 0
+        CR : 4
+        MSG_Energy : -25.0943
+        err_corrected : 0
+        err_detected : 0
+        Noise_Energy : -95.796
+        pay_len : 5
+        SNR : 70.7018
+        msg : test5
+        crc_valid : True
+        SF : 7
+Low SNR - increase Tx Gain to 30
+Sending Acknowledgement
+</code>
+The message is still received by the base station, but since the SNR is lower, the base station rises its GTX to 30, so that the acknoledgment is received by the node.
+## What next?
+Congratulations! You have finished your first tutorial on CorteXlab. Please, feel free to change the example task and try out different configurations, carrier frequencies, bands, and so on. You can always resubmit this task to test out different kinds of configurations.
+One good example of what to do is to nudge the carrier frequency of approximately +/- 20 KHz and see if the decoding process works better.
+To learn mode advanced concepts around creating and managing tasks on CorteXlab, please continue the tutorials.