[Ground-station] Call for Comment and Critique - KA9Q-SDR readme

Sat Apr 21 12:01:12 PDT 2018

Call for Comment and Critique. Contents are below. File is also attached!
See most recent weekly report for an introduction to this package.

-mdt

ka9q-radio

Overview

This is a collection of software defined radio (SDR) modules connected

by the Internet Real Time Protocol (RTP) and IP multicasting. It is

intended to facilitate experimentation and education, and to be easy

to interface to data decoders and digital communication programs. It

can also smoothly track satellite Doppler shift in an open-loop mode

when fed velocity and acceleration information from an external

program.

The ka9q-radio modules currently execute from the command line on

either Linux or OSX.  As yet there is no fancy graphical user

interface. (This may come later, especially if someone volunteers to

create one).

Multicasting and the Real Time Protocol

The ka9q-radio modules are designed for maximum versatility.  Unlike

shell pipelines or TCP/IP connections, ka9q-radio modules can be

individually started and stopped at any time, and one can feed any

number of others without prearrangement. They can run on the same

computer or on different computers on a multicast-capable network

(e.g., an Ethernet LAN).

RTP is widely used for point-to-point voice over IP (VoIP), though it

was designed for multicasting with any codec or data

format. ka9q-radio uses four RTP payload types: raw I/Q data with a

custom receiver status header; mono or stereo 16-bit uncompressed PCM

audio sampled at 48 kHz; and the new Opus codec.

It should be easy to add PCM/RTP input to any ham SDR program that

uses a computer sound card to aquire receiver audio (e.g. WSJT-X,

fldigi and many others). This would bypass the computer's sound system

and providing a much cleaner and more reliable interface. No analog

audio cables, "rigblaster" adapter boxes, audio ground loops,

equalization problems, or tricky level adjustments! With multicasting,

any number of programs can process the same receiver output.

Parts of the ka9q-radio package should also be well suited to

"turnkey" networked receiver applications such as receive-only

APRS-to-Internet gateways and Broadcastify feeds.

Opus Compression and Audio Monitoring

Although PCM is best for digital decoders, it requires 800 kb/s for

mono and 1.6 Mb/s for stereo.  Much lower data rates suffice for human

listening over WiFi or a remote Internet connection. I use the new

Opus codec standardized by the Internet Engineering Task Force

(IETF). A high quality, open-source Opus implementation is available,

and it can be freely used without patent restrictions (unlike, e.g.,

ABME).  Opus automatically switches algorithms as needed to do the

best it can with whatever data rate is available, from very high

fidelity stereo audio at up to (an unecessarily high) 510 kb/s down to

monaural voice at only 6 kb/s. (VK5DGR's CODEC2 can handle even lower

data rates, which is on my to-do list.)  I typically use 32 kb/s,

which is actually much higher than needed for communications-quality

audio.

Popular media players like VLC support PCM and Opus so you can listen

to ka9q-radio audio on many smartphones and tablets. Curently the

player must receive multicasts, which usually means they must be on

the same LAN as the computer(s) generating the audio streams. Remote

unicast Internet connectivity is on the to-do list.

The ka9q-radio package also includes 'monitor', a RTP audio player

that can receive and mix several multicast audio streams to the local

sound output. It has a novel 'pan' feature that uses gain and delay

adjustments (<1 ms) to place each source at a user-chosen point in a

stereo image. This makes it easier to distinguish multiple sources,

e.g., in a round table. The 'monitor' program currently supports only

Opus and PCM, though CODEC2 is on the list.

The KA9Q 'radio' program

The heart of the ka9q-radio package is the program 'radio', an

interactive general coverage receiver. It reads raw I/Q data multicast

by a SDR front end, sending tuning and gain adjustments as

needed. radio's output is a mono or stereo PCM audio multicast

stream.

If audio compression is needed, a separate 'opus' module transcodes

PCM to Opus. I.e., it receives PCM, compresses it with Opus and

multicasts it to a different IP address.  This lets listeners and

players select compressed or uncompressed audio by merely joining the

corresponding multicast group.

A simple but entirely usable textual interactive user interface is

provided. If desired, it can be disabled entirely with all parameters

given instead on the command line (e.g., in shell scripts).

Architecture

The 'radio' program accepts a generic I/Q (complex) sample stream

multicast by a computer with a direct conversion ("zero IF") SDR front

end. At present, only the AMSAT UK Funcube Pro+ dongle is supported

with the 'funcube' module. The (mostly higher) sample rates produced

by other receivers such as the SDRPlay and RTL-SDR are supported but

only the Funcube's 192 kHz sampling rate has actually been tested.

An I/Q stream at 192 kHz with 16 bit samples is a constant 6.5 Mb/s

network load; this is no problem for modern Ethernet but it should be

kept away from inexpensive WiFi base stations and public Internets.

(More on problems with multicast and WiFi in the end notes).

Hardware Artifact Removal

Although a SDR front end is typically direct conversion, the complete

ka9q receiver is actually a dual-conversion superheterodyne: the first

LO and mixer are in analog hardware and the second LO and mixer are in

software. The first mixer is the quadrature (I/Q) type that produces a

complex first IF centered on zero Hz and extending from -Fs/2 to

+Fs/2, where Fs is the A/D sample rate.  For the Funcube dongle, Fs =

192 kHz so the IF extends 96 kHz above and below the first LO

frequency. (For a complex signal, the Nyquist rate is equal to the

bandwidth; for a more familiar real signal, the Nyquist rate is twice

the signal bandwidth.)

This is often called a "zero IF" architecture but this is a slight

misnomer. The actual first IF can be anywhere between -Fs/2 and +Fs/2

provided it contains the entire signal bandwidth, though the 10 kHz

near each edge from a Funcube dongle is best avoided because of

imperfect anti-alias filtering before the A/D converters.

There is also a "DC spike" at 0 Hz from crosstalk from the first LO to

the receiver input. The first processing step in 'radio' completely

removes this spike, but reciprocal mixing produces some low frequency

phase noise that cannot be removed. With the Funcube dongle this noise

is only noticeable in without antenna noise covering it up, but it is

still best avoided. E.g., the range from +48 kHz to +51 kHz (or -51

kHz to -48 kHz) might be selected for SSB with a 3 kHz bandwidth.

This is still an extremely low IF by usual standards as adequate image

rejection cannot be provided by ordinary filtering. The use of complex

I/Q signals allows the frequency image to be cancelled with digital

signal processing. This requires the 'radio' program to compensate for

any slight gain and phase imbalances between the I and Q channels, and

this is its second processing step. Typical values for the Funcube

dongle might be 0.07 dB of gain imabalance and 0.4 degrees of phase

error.

Frequency conversion

The third step is to convert the first IF signal to baseband, i.e., 0

Hz. This is performed by the second (software) LO and mixer in complex

arithmetic, which suppresses the unwanted image in the other half of

the IF. The LO and mixer each require only one complex multiply per

sample. Sine and cosine calls are needed ony when the LO is retuned.

Another LO and mixer are optionally used for open-loop correction of

satellite Doppler shift. The 'radio' program reads velocity and

acceleration from an external program, calculates and applies the

appropriate frequency offset and rate.

Filtering and Demodulation

The baseband signal is filtered with fast correlation.  With complex

signals, the passband can be asymmetric, e.g., +100 to +3000 Hz

selects upper sideband (USB) and -100 to -3000 Hz selects LSB. A

post-detection frequency shift is provided for CW; this is equivalent

to retuning the radio while simultaneously sliding the filter to keep

the signal at the same point in the passband. Modes other than SSB

typically use filters centered on 0 Hz but this is not required; the

edges can be individually varied, e.g. to suppress adjacent channel

interference.

The filtered baseband signal is then fed to one of three demodulators:

'AM', 'FM' and 'linear.'

The AM detector implements ordinary noncoherent envelope detection of

AM signals.

The FM demodulator handles narrowband frequency or phase modulation.

Two submodes are provided: 'FM', with a high pass cut on the received

audio below 300 Hz to remove PL/CTCSS tones and a standard -6

dB/octave audio de-emphasis above 300 Hz. The 'FMF' mode is straight

FM without post-detection filtering. Both modes use an FFT to measure

PL/CTCSS tone frequencies to 0.1 Hz accuracy. SNR, frequency offset

and peak deviation are also measured and displayed.

The 'linear' detector is used for all other modes, including SSB/CW,

ISB (independent sideband) and raw I/Q. In linear mode the filter

directly feeds the output: both the I and Q channels for stereo modes

and just the I channel for mono. Stereo is implied by I/Q and ISB and

is optional for CW/SSB, where it provides an interesting effect

because of the 90 degree phase shift (Hilbert transform) between the

channels.  This may be useful to minimize fatigue during long

contests.

The I/Q and ISB modes are the same except that in ISB the filter

processes the positive and negative frequency components to force the

lower sideband onto the I (left) channel and the upper sideband onto Q

(right).

A PLL and mixer can track a carrier for coherent (synchronous) AM

detection, which is usually less noisy than regular AM envelope

detection. In "calibrate" mode the PLL can adjust a TCXO offset

estimate to bring the carrier to exactly 0 Hz.  This is useful for

automatic calibration of the SDR TCXO against WWV/H, CHU, an ATSC

(North American digital TV) pilot carrier, or an external frequency

reference.

A squarer can be inserted in the PLL to track the suppressed carrier

in DSB-AM and BPSK.

User Interface

The 'radio' program has a textual user interface that uses the

"ncurses" library; it may look primitive by 2018 standards, but it is

lightweight, simple and very efficient over slow Internet paths. The

'h' key toggles a list of the supported commands. The textual user

interface can also be completely disabled and all receiver parameters

given on the command line, e.g, for invocation from a shell script.

Startup files are stored in ~/.radiostate (i.e., the directory

".radiostate") in the user's home directory). Invoking 'radio' with

the name of a startup file loads its settings. The state of the radio

can be saved to a state file with the 'w' key. On normal termination

the radio state is also stored as "default". It will be automatically

loaded on the next invocation if no explicit name is given.

The various operating modes are specified in

/usr/local/share/ka9q-radio/modes.txt.  Each entry specifies a

demodulator along with filter parameters, the post-detection frequency

shift, AGC responses and option flags. For example, "USB" selects the

"linear" demdulator, a filter passband of +100 to +3000 Hz, a zero

post-detection frequency offset, an AGC attack rate of -50 dB/sec, an

AGC recovery rate of +6 dB/sec, an AGC hang time of 1.1 seconds and

mono output.

The text mode interface provides the following status windows:

The "Tuning" window shows the radio, LO and IF frequencies. If Doppler

correction is active, it is also shown here. The user can change

frequency with the arrow keys, a Griffin PowerMate USB knob, a mouse

wheel, or with text entered through a popup menu.

The "Signal" window shows signal, noise and SNR estimates at various

points in the signal path. Levels are in dBFS, i.e., decibels below

full scale (A/D saturation) as there is no way to know the gain ahead

of the A/D converters without careful calibration, which is invariably

frequency dependent.

The "Info" window shows, if available, the name of the channel or

frequency band and, if it's a ham band, the allowed emissions and

required license class.  This information comes from

/usr/local/share/ka9q-radio/bandplan.txt.

The "Filtering" window gives the pre-detection filter and

post-detection frequency shift settings. The filter edges, frequency

shift and Kaiser window beta parameter can be adjusted in the same way

as the tuning. Smaller betas give sharper filters but poorer

stop-band attenuation; larger betas give wider transition bands

but better rejection outside the passband. [See the later discussion

about sample rates and decimation.]

The "Demodulator" window shows mode-specific information.  The AM and

linear modes include a simple fast-attack, slow decay AGC for SSB

(more work can be done here).

The "Options" window selects the various options for the "Linear"

demodulator.

The "SDR Hardware" window shows the A/D sample rate, TCXO offset,

nominal tuner frequency, analog gain settings and gain and phase

imbalance estimates.

The "Modes" window allows the operating moded to be conveniently selected

with a mouse click.

The "I/O" window gives information on the I/Q input and PCM audio

output network streams: IP addresses or host names; port numbers;

packet, sample and error counts; and a SDR timestamp useful when

playing back recorded I/Q data. It also estimates the true I/Q sample

rate against the local computer time-of-day clock.

A "Debug" area at the very bottom of the screen shows version information

and various test and debugging messages.

Textual user input entered through popup menus; for example a tuning

frequency can be directly entered as "147m435" (147.435 MHz), 760k

(760 kHz) or 1g296296 (1296.296 MHz). If no units letter is used, a

'best guess' is made to produce a valid frequency; e.g. 14313 will

give 14.313 MHz, not 14.313 kHz (which is below the Funcube's range).

Other ka9q-radio Modules

Other modules provide either miscellaneous functions or begin more

complex planned features. None of them have especially complex user

interfaces; most take only a few command line arguments and can run as

background daemons.

'Funcube'

The 'funcube' module takes I/Q data from a locally connected Funcube

Pro+, multicasts it over the local LAN and accepts unicast commands to

tune the radio and set analog gain levels. It will also automatically

change gain levels to avoid A/D saturation. Similar modules will be

written for other SDR front ends such as the SDRPlay and RTL-SDR that

will either talk directly to the hardware or through "shimware" such

as SoapySDR.

'IQrecord' and 'IQplay"

The 'iqrecord' and 'iqplay' modules perform the functions suggested by

their names. 'iqrecord' accepts multicast I/Q and PCM audio streams

and writes them to disk. Gaps in the multicast stream due to lost

packets or silence suppression are seeked over in the recordingq to
maintain the

correct sample count and playback timing.

I/Q streams are written into files named as 'iqrecord-xxxxxx-n' where

xxxxxx is the RTP SSRC (Stream Source Identifier) and 'n' is a number

incremented to avoid overwriting existing recordings. PCM streams are

written as 'pcmrecord-xxxxxx-n'. Both files are written as raw,

headerless 16-bit PCM with all meta information in extended file

attributes.

Most but not all modern file systems support extended attributes; a

notable exception is Microsoft's VFAT32 often used as a

least-common-denominator for file exchange between different operating

systems. Warning! Many file copy and archiving utilities do not

preserve extended attributes, at least not by default.

I/Q and PCM recordings have many uses. An I/Q recording preserves

everything fed to the 'radio' input regardless of modulation type or

bandwidth (provided it fits within the receiver passband). This is

useful for testing and for demonstrating the 'radio' program when an

antenna is not available.

I/Q recordings are especially useful as backups during events

difficult or impossible to reproduce, such as a satellite pass or

balloon flight. When made at or close to the SDR hardware they depend

only on the antenna, SDR hardware and I/Q multicast program

(e.g. 'funcube').  This can guard against bugs (or the outright

failure) of more complex elements in the downstream signal path, e.g.,

a digital data demodulator. After the demodulator is fixed, the

recording can be played back to recover the data.

'iqrecord' is an excellent example of multicasting's versatility; it

can just sit quietly in the corner recording the same I/Q data stream

being processed by a real-time software receiver or telemetry decoder

without impairing its function in any way.

The 'iqplay' module plays back I/Q recordings (only -- no PCM support

at present) using the metadata contained in the external file

attributes. It can also read a raw stream from standard input to

simulate SDR front end hardware.

'Opus'

The 'opus' module was described earlier as an optional "transcoder"

that accepts uncompressed PCM (either mono or stereo) and produces a

lower bit rate compressed version with the Opus codec. Options include

a range of blocksizes from 2.5 milliseconds to 120 ms. The larger

blocks are useful only to reduce packet rate and header overhead; they

provide no additional compression and are supported only by more

recent versions of the Opus library. The default is 20 ms, which is

long enough to give good compression but short enough to be

essentially unnoticeable.

The compressed output data rate is specified by a command line

parameter; it defaults to 32 kb/s which produces surprisingly good

audio even on stereo music.

The Opus codec also supports a VOX-like "discontinuous" mode well

suited to half-duplex push-to-talk amateur operation. When silence is

detected, it automatically drops to sending "comfort noise" at only 3

packets per second.

"Packet"

This module is my first digital demodulator module for the ka9q-radio

package. It accepts PCM audio from the 'radio' program and demodulates

and decodes standard 1200 bps AX.25 frames using the ancient Bell 202A

AFSK modem. The decoded AX.25 frames are multicast in UDP packets that

do not currently have RTP headers; this will probably change. The

decoded frames can also optionally be displayed on the

console. Otherwise the module can run as an unattended daemon.

"APRS"

This unfinished module accepts decoded AX.25 frames from the "packet'

module, extracts APRS position data from a selected station, computes

the azimuth, elevation and range to that station from a specified

point, and commands antenna rotors to point at the transmitting

station. This was written to automatically track high altitude

balloons.

Footnotes and Sidebars

Sample Rates and Decimation

The I/Q input sample rate must be an integer multiple of the 48 kHz

audio output rate.  Decimation is performed as a byproduct of the fast

correlator used for pre-detection filtering. Since the input FFT in a

fast correlator executes at the input sample rate, CPU loading will

increase.  Faster and simpler decimators are in the works.

By default the filter decimates the Funcube 192 kHz A/D input sample

rate by a factor of 4 to a 48 kHz audio output.  Other SDR front ends

with higher sample rates will require correspondingly higher

decimation ratios.  Although 48 kHz may seem excessive for

communications-grade audio, I don't recommend reducing it. 48 kHz is

supported by nearly every audio D/A, and it still uses only 0.154% of

a gigabit Ethernet link.

More importantly, the Opus codec strongly prefers 48 kHz stereo even

for narrowband mono voice, and there seems to be no advantage to mono

or a lower sample rate (i.e., the compressed data rate is not reduced

nor is audio quality improved at a given rate.) So if the audio bit

rate is a concern, just use Opus; don't reduce the PCM sample rate.

Digital demodulators are best run on computers with fast local network

connections to the radio program so they can be given uncompressed

PCM.  (Audio compression works by eliminating signal components

inaudible to the human ear, but which may be very important for a

digital demodulator.)

Filter Tradeoffs

Simultaneously improving both filter rolloff and stop-band attenuation

requires a longer FIR impulse response, which implies greater latency

and somewhat increased CPU loading.  Currently, the filter blocksize

and FIR length can only be specified on the command line or in the

startup file, i.e., they cannot be changed without restarting the

program. (Note: the length of the FFT executed by the filter is equal

to the sum of the blocksize and FIR length minus one. The default

blocksize of 3840 corresponds to 960 samples after 4:1 decimation to

48 kHz; this is the number of samples in a 20 ms Opus frame. 3840 +

4353 - 1 = 8192, i.e., the FFT has 8k points. While the FFTW package

can handle any number of points, a power of 2 is more efficient. I've

found the default parameters to be suitable for most purposes.)

Correcting Fractional-N Frequency Synthesizer Artifacts

Because the first LO in the Funcube dongle is an analog hardware with

the usual synthesizer tuning artifacts, it is retuned only when

necessary to contain the desired signal in the first IF, i.e., between

-Fs/2 and +Fs/2. Otherwise tuning is with the second (software) LO

because it can be smoothly and instantly tuned without any

glitches. This is especially important with open-loop Doppler

correction.

Like most modern radios, the Funcube uses fractional-N frequency

synthesis with inherent restrictions on the actual set of frequencies

that can be produced. Although its programming interface advertises 1

Hz steps, the actual tuning step size varies with frequency; in the HF

and VHF range it is 1000/2048 = 0.48828 Hz so it cannot produce exact

1 Hz frequency steps.

The 'radio' program works around this in two independent ways. First,

the hardware synthesizer programming formulas and constants from the

Funcube firmware are used to determine the actual frequency for any

requested frequency so the discrepancy can be corrected by the second

LO. Second, only frequencies that the synthesizer can exactly produce

are selected; the step size varies with frequency so the worst

(largest) value of 125 Hz is always used, with the difference again

absorbed by the second LO. Although these two fixes are specific to the

Funcube dongle, other SDRs undoubtedly have the same problem subject

to the same workarounds -- if I can get the necessary details.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openresearch.institute/pipermail/ground-station-openresearch.institute/attachments/20180421/bafa0bd2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: README.md
Type: text/markdown
Size: 22589 bytes
Desc: not available
URL: <http://lists.openresearch.institute/pipermail/ground-station-openresearch.institute/attachments/20180421/bafa0bd2/attachment.bin>