Don't forget to read the FAQ.
Deep Learning datasets
Don't forget to read the FAQ.
Deep Learning datasets
Cortexlab users do not have direct access to the cortexlab radio nodes. They have to package their experiment in a minus scenario and launch it from the frontend (airlock).
This is simple as long as the scenario is made, for example, from simple python scripts or static binaries, using the right version of libraries and/or gnuradio than installed on the nodes. But things get more complex as soon as users want to make use of:
There are a few ways to tackle these problems.
There is basically no way to avoid this issue: users need to target the same gnuradio version than the one on the cortexlab nodes.
Actually this is not only about gnuradio, but about the whole package of software used to run the radio on the nodes. We refer to this package as the cortexlab toolchain. See Building a toolchain
There is currently three possibilities for working with the same version of toolchain than the one on the cortexlab nodes:
cxlb-build-toolchainscript. In this case, users are on their own. In the majority of situation this should cause no issue but there may be some slight incompatibilities with the toolchain on the nodes.
Ususally, one compiles an executable binary or library on the same system it will run on, and installs it in its final location. The difficulty when packaging such binaries in a minus task scenario is that it's not longer the case:
The last issue is that we need to configure the compilation of the binaries such that the install location of the binaries is the final location where they will be executed, on the radio nodes, but we will need to install them first into a staging area where we will assemble our task file. This is the concept of a staged install which tools such as automake or cmake support. Unfortunately, some other build systems (especially adhoc makefiles) will perhaps not directly support that functionnality and will need a bit of hacking to be able to do that.
The location on the cortexlab nodes where the task will be unpacked and executed is
Thus, the compilation / stage installation of a custom binary in a minus task scenario has the following scheme, assuming that the task is assembled in directory
/cortexlab/homes/<userlogin>/tasks/mytask and that the temporary staging directory for compiled binaries is
(in the following, we assume that we compile directly on
$ ./configure --prefix=/home/cxlbuser/tasks/task $ make $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install
$ cmake -DCMAKE_INSTALL_PREFIX=/home/cxlbuser/tasks/task $ make $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install
Then, in both cases, we will find in
/cortexlab/homes/<userlogin>/tasks/mytask/tmp the complete directory hierarchy
/cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task and directories such as
include, etc. below. Now we simply have to move this up to the toplevel of the task:
$ mv /cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task/* /cortexlab/homes/<userlogin>/tasks/mytask/ $ rm -rf /cortexlab/homes/<userlogin>/tasks/mytask/tmp
Then we can assemble our task:
$ minus task create /cortexlab/homes/<userlogin>/tasks/mytask
If we compile on a workstation instead of
airlock, the steps are identical except that the task directory has be be transfered to
airlock before running
minus task create on
A gnuradio OOT module is just a special case of custom binary, as described in the previous section.
The gnuradio build system is designed to allow building OOT gnuradio modules easily. For this, users need to use
gr_modtool to create their module.
gr_modtool will generate a cmake project which will automatically detect gnuradio, if running cmake from airlock (using the cortexlab toolchain instance on airlock), or from a user's workstation, with a gnuradio built with cxlb-build-toolchain and a properly configured environment.
Then, as described in the previous section, run cmake and install this way:
$ cmake -DCMAKE_INSTALL_PREFIX=/home/cxlbuser/tasks/task $ make $ make DESTDIR=/cortexlab/homes/<userlogin>/tasks/mytask/tmp install $ mv /cortexlab/homes/<userlogin>/tasks/mytask/tmp/home/cxlbuser/tasks/task/* /cortexlab/homes/<userlogin>/tasks/mytask/ $ rm -rf /cortexlab/homes/<userlogin>/tasks/mytask/tmp
The most complex situation is when you need to embed into a minus task a binary (be it a gnuradio OOT module, an executable or a library) which depends itself from other binaries (libraries, most of the time) which need to be also embedded in the minus task. These dependency chains may be longer (eg. OOT module A depends on library B, which depends on library C and D, and library C depends on D also, as well as on library E).
In this situation, you need to compile everything in reverse order starting from the end of the chain (the libraries which have no custom dependencies. In the previous example, the order would be E, D, C, B, A). The challenge is to be able to tell following libraries (in the order of compilation) where to find previous ones includes (for compilation) and libs (for linking)
explain how to, when compiling D, telling it where to find E