This webcast was originally published on September 3rd, 2019.
In this video, Paul Clark discusses the intricacies of Python code development for radio applications using the GNU Radio Companion GUI. They explain how to create modular, object-oriented code and the advantages of directly coding radio blocks for better customization and flexibility. Additionally, the video covers practical tips for real-time system management and dynamic flow graph modifications to enhance radio system performance.
- The importance of modular design in Python coding is emphasized to simplify the development process.
- GNU Radio Companion is highlighted as a tool that can convert graphical blocks into Python code, simplifying radio application development.
- The webinar explains the process of building configurable radio stacks that can adapt during runtime, enhancing flexibility in radio communication systems.
Highlights
Full Video
Transcript
Paul Clark
So the kind of the real key on building the python code, just like almost any code, is to make sure the design is as modular as possible.
And if you haven’t done object oriented programming in Python, it might look a little bit odd at first. Or if you haven’t done object oriented programming at all, I would say that it’s a lot easier to figure out object oriented code in Python than it is in c.
And so hopefully it’s not too much of a hurdle. I think the main thing to keep in mind is that classes are just a bunch of code you write and then objects are variables that are take on the type of that class.
And that class then has a bunch of data. It’s just a kind of a wrapper that has a whole bunch of data and functions in it that allow you to kind of easily make function calls from the very top level, that reduce hundreds of lines of code down to a pretty small amount.
as we’ll see later, I’m not going to walk through the whole code, but I will walk through a few key sections and hopefully give you some confidence that it can actually be done pretty simply once you get some of the grunt work out of the way.
But in terms of the Python code, one of the things that I also did that was kind of key is I coded the radios themselves in, or the radio applications themselves in native, Python using the GNU radio companion GUi.
A lot of times when you think about, let me see if I can, let me just get out of the presentation for a minute. A lot of times when you think about GNU radio, you think of this, right?
you think of this set of blocks that are dropped down into a graphical interface. And you have this nice kind of menu to the right.
You click and drag and then you connect them all up. If you’ve played around with GNU Radio 90 plus percent of the time it’s probably been something like this. And people don’t always know, but there’s a generate and execute button.
All that generate button does is take this code here and turn it into python code. And it essentially looks something like this.
And so you’ll have a block, it’ll have a bunch of boilerplate, it’ll basically be a class. And then you will, and this happens automatically.
You don’t even look at it, you don’t even see it. But every time you run a GNU radio companion flow graph, it’ll take all that graphical stuff and just create Python representations of it.
And each block will have an instance of itself with the variable, with the parameters that you’ve set for it, and then they’ll all be connected together with this connect function, and then it’ll simply just run that Python script.
That’s really all that’s happening when you are doing a Gunaradio companion design. What I did, and there are a couple advantages to doing this, is I kind of dispensed with GNU radio companion and coded the blocks directly.
And that allowed me to kind of build hierarchical blocks that would sort of swap in different types of radio elements as I needed them.
So I could use different types of tuners, different types of modulators, different types of bit synchronizers. And they were all configurable. So I could just kind of swap those in and out during runtime based on the way I built this thing.
Jason Blanchard
Hey Paul.
Paul Clark
Yes?
Jason Blanchard
Someone just asked, can you share the link to the python code generator?
Paul Clark
so the Python code generator is GNU radio companion itself. If you do an install of GNU radio on your machine, on your Linux box, or I guess you can get it installed on some other stuff, I’d advise against it, but GNU radio companion, this interface here, the Python code generator is inside here.
If I were to say, plot down all these blocks and build this flow graph graphically that you see here, just for example. then I were to press the generate button here.
So it’s right there, it’s right there. You will get a file in whatever your working directory is called, top block pyro.
That python file is going to have all sorts of boilerplate in it. And so some of it’s going to be a little bit hard to read because most flow graphs have some graphical stuff going on.
And so they’ll call the QT libraries. And so there’ll be 50, 60, 70 lines of code involving the Qt stuff. But after you get through that, you’re going to see variables like this, and then you’re going to see blocks.
And everything’s got a self in front of it, which means it’s part of the object, that you’re defining or part of the class you’re defining. I won’t dive too much into that, but essentially you’ve got a block definition based on each block that you’ll see in here.
And then you’ll have a connect function call that’s going to give, like for this one here, for example, it’ll say connect this particular block’s output to this particular block’s input.
And that’s just, there’s going to be one of those for each wire you see here.
Jason Blanchard
Like the arrows, right? Yeah, the connectors.
Paul Clark
Exactly.
Jason Blanchard
Thanks.
Paul Clark
Yeah, no, it’s a good question, because this is, I think, kind of confusing. So kind of the key point though is that I tried to design the radio stack in a way that was, well, kind of like a stack.
I mean, the data flows up and down through various, kind of levels of abstraction, and also that each level of the stack was configurable insofar as it could be.
And this is another thing, this kind of, this line here about the flow graphs being created and destroyed dynamically.
you can read documentation about how to start and stop flow graphs, you can read the documentation about how to destroy them, but that’s not typically something people talk about much. What I found is if you are working through trying to keep a system running in real time, I’ve had troubles with hackrf drivers where just some strange error gets spun up and the whole system kind of goes down.
I found that if you actually take the time to destroy a flow graph and then restart a new one completely, that you can get around that. So that was kind of one useful thing I found.
I’ll get into more of the useful tips later, but the key, and this may sound fantastical, but once the radio stack gets built and refined a bit, you can really get to a point where you can create fundamentally new types of radios with just three or four lines of code.
And that may be hard to believe, and you may wonder what kind of caveats I have around that, but I will show you that in a minute. So just kind of from a high level, the python design or the Python code, on the transmit side, it was a matter of kind of coming up with some form of processing the baseband data, which is essentially saying I’ve got some input data I want to send.
Do I need to stretch that out in time? Do I need to manipulate it in some way, add some error checking, what have you? So that’s kind of one level of this. Next, that data would get sent to RF modulator.
So it would take the information and it would create a, untuned RF signal with a on off keying modulation scheme or an FSK frequency shift keying modulation scheme.
And it would then take that untuned signal, shift it around so that it goes out at the frequency I want it to go out at, and then send it to an RF sync, which if you’re unfamiliar with GNU radio, that’s just the block that connects up to the SDR hardware and then causes the transmission to happen.
So that’s just something to keep in mind. On the receive side, it’s a little bit more complicated. Receive side tends to be more complicated in general.
so the transmitter basically is just talking. The receiver has to figure out what’s going on. So in general, the receiver is always going to be a little bit harder. and so at the top level it’s got an RF data source that’s again, this is the GNU radio’s way of interfacing with the SDR hardware.
So that means SDR is receiving data and then coming through this portion of the code and then goes into a tuner which figures out where the signal is that we want to get.
And then it goes into in some cases, an automatic game control. because I supported on off keying, I don’t know how much if anyone has questions about this, I can do a deeper dive.
But when you’re dealing with frequency, based systems, you have to worry about your frequency tuning, but you don’t have to worry about your amplitude as much. In general, on off keying systems, you always have to pay some attention to what the amplitude is and make a judgment call about when it’s high enough that you’re going to interpret it as a one and when it’s low enough that you’re going to interpret it as zero.
And unfortunately an ok system, if you just move it further apart, the signal gets weaker. And so the discrimination between the ones and the zeros starts to get more complicated or more arbitrary.
And so the automatic gain control that I implemented here was just a way to kind of resolve that. And then it just kind of automatically adjusts, the magnitude of the signal to kind of get around that issue.
And then demodulation. That’s just taking the tuned signal that we’ve got and saying, okay, are these squiggles here a one or a zero?
And that after that distinction is made, we have to do probably the most complicated part of all this, which is clock synchronization. And that’s something that, yeah, that’s something that is probably one of the most mysterious parts of GNU radio.
And there are definitely attempts to explain it, online, but it’s tricky. this is one of the things that I focus on a lot in the second class, the intermediate SDR class I teach.
The second one you’ll see on the black hat schedule, the clock synchronization it uses kind of, it uses a lot of strange software incantations in blackmagic that I can partially demystify.
But it’s essentially the difficulty is figuring out when one bit starts and the next bit ends, or rather when one bit ends and the next bit starts.
Figuring out that is really the heart of the clock synchronization. And there are blocks that can help with that and implement that. That’s another layer of the software stack on the receive side.
And then once you’ve got that figured out, there’s a little bit of baseband data processing, like figuring out what kind of error checking, whether or not you’ve got a decent payload, or not, based on the error checking result.
So this is kind of the 1000 foot view then of the received software. So one of the real helpful things that I use to build this system is the zero MQ framework.
If you’ve played around with GNU radio, you might have seen the ZMQ or the zero MQ blocks in the catalog. They are a really underappreciated tool, inside of GNU radio.
And they’re not just a GNU radio thing. They are something that GNU Radio adopted that you can use completely outside of GNU radio applications if you like. And they’re really just a pipe for getting data from one application to another is how I think of it.
If you for example have data running in one, if you have a flow graph that’s running inside of one of these GNU radio windows, that I showed you earlier, and you just want to get the data out and use it somewhere, somewhere else, maybe a python script, maybe another flow graph, who knows?
You can actually set up a zero mq socket on one side and set up a socket on the other side. And you can basically just funnel data from one application to the other.
This can go from one flow graph to another flow graph. It could go from a Python program to another Python script. It could go from a flow graph to a python script. That’s the beautiful thing is there’s really no significant constraints on that.
It’s also you can go from one machine to another because you’re using a TCP IP layer to our TCP layer to go for data transport.
This was useful for me in testing. It was useful for debugging, it was useful for getting payloads out of transmitter flow graphs or end of transmitter flow graphs and out of receiver flow graphs, which honestly is probably the least understood and least well demonstrated part of learning, how to use GNU radio.
So kind of got ahead of myself, talking to this slide, because I already mentioned that. Yeah, you can use any kind of combination, flow graph to flow graph or flow graph to code and the messaging.
that’s another kind of aspect that’s interesting. If you played around with GNU radio at all, you might have known, you might have noticed that there are message, type connections where you’re not just sending a stream of constant data, but you’re actually sending an asynchronous message.
Every so often you just say, hey, I’ve got some information, do this thing. And you can send that in the form of a message. And that can happen at any time. Based on external control code, you can generate this message.
And this zero MQ framework allows you to deal with both constant streams of data as well as the asynchronous messages, which is what makes it so incredibly useful.
I showed a lot of this stuff already, the stuff I learned, just kind of the big takeaway though. and I promised I wouldn’t step through code. I’m only going to do this one thing here.
Here for example, is given the stack that I built, I have a 65 line radio file, most of which is a bunch of imports and argument parsing and command line argument junk.
And then when you look at the actual radio stuff, I just have some classes, some some objects that take some default parameters. I modifies the parameter based on what kind of modulation scheme I want to use.
I create an object for the type of radio, I want to build just using those parameters, I switch it to receive mode and then just debug statement there.
I just have a while loop and it just receives data. That’s the entirety of the radio. And then I was shut down. So I mean, the goal really, when you’re building these radio stacks in my view, is to try to come up with something that’s abstracted enough that at the top level you can really focus on the control aspect of it.
What do you really want the radio to do at any given time and not have to get into all of the minutiae about the frequencies and how you’re modulating and whatnot. so that’s kind of the key and that’s hopefully what I want to impress people with, is that this stuff can be distilled into a very clean and usable type of solution.
Mhm.