OP25 OP25 - LiquidSoap and Raspberry 3

miclass

Member
Joined
Dec 27, 2020
Messages
21
I am using OP25 (boatmod) with LiquidSoap on a Raspberry to decode and listen (locally) to a DMR signal. I have tried different audio processings with LiquidSoap but I am still not satisfied with the output volume, which is too low. It don't think it's a problem of the raspberry / loudspeakers because if I try to play a webradio with mplayer I can hear it well and loud.

This is my current configuration:
Code:
#!/usr/bin/liquidsoap

# Example liquidsoap streaming from op25 to icecast
# (c) 2019, 2020 gnorbury@bondcar.com, wllmbecks@gmail.com
#
set("log.stdout", true)
set("log.file", false)
set("log.level", 1)
# Make the native sample rate compatible with op25
set("frame.audio.samplerate", 8000)
#input = mksafe(input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -x 2 -s"))
input = input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -s -x 4")
# Consider increasing the buffer value on slow systems such as RPi3. e.g. buffer=0.25
# Longer buffer results in less choppy audio but at the expense of increased latency.

# OPTIONAL AUDIO SIGNAL PROCESSING BLOCKS
# Uncomment to enable
#
# High pass filter
input = filter.iir.butterworth.high(frequency = 200.0, order = 4, input)
# Low pass filter
input = filter.iir.butterworth.low(frequency = 3250.0, order = 4, input)
# Normalization
#input = normalize(input, gain_max = 20.0, gain_min = -6.0, target = -13.0, threshold = -60.0)
input = compress(threshold=-18.,ratio=3.,gain=3.,normalize(input))

# LOCAL AUDIO OUTPUT
# Uncomment the appropriate line below to enable local sound
#
# Default audio subsystem
out (input)
#
# PulseAudio
#output.pulseaudio(input)
#
# ALSA
#output.alsa(input)

# dump recordings to a file
time_stamp = '%Y-%m-%d_%H.%M.%S'
output.file(%mp3, "./OUT_#{time_stamp}.mp3", input , fallible=true)
I tried with both normalize and normalize + compress.
Note: as you can see I wuold like also to dump the audio as mp3 file, for this reason I removed "mksafe" from the input to avoid recording also the silence between conversation.

Any advice on how to improve volume?

Thank you!
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
"-x 4" is very high and likely to result in clipping.

Try this instead:
Code:
input = mksafe(input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -2 -x 1.5 -s"))
input = compress(input, attack = 2.0, gain = 0.0, knee = 13.0, ratio = 2.0, release = 12.3, threshold = -18.0, rms_window = 1.0)
input = normalize(input, gain_max = 6.0, gain_min = -6.0, target = -16.0, threshold = -65.0)
 

miclass

Member
Joined
Dec 27, 2020
Messages
21
"-x 4" is very high and likely to result in clipping.

Try this instead:
Code:
input = mksafe(input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -2 -x 1.5 -s"))
input = compress(input, attack = 2.0, gain = 0.0, knee = 13.0, ratio = 2.0, release = 12.3, threshold = -18.0, rms_window = 1.0)
input = normalize(input, gain_max = 6.0, gain_min = -6.0, target = -16.0, threshold = -65.0)
Thank you! Now it's better, I still have some clipping at the very begining of the transmissions, I don't know if it can be solved.

I currently have 2 streams, forking from the same source, the first one goes to the audio output following your lines of code for op25.liq (including mksafe), while the second one goes to an mp3 file, without mksafe (to avoid recording silence).

I am having a problem with the second stream, it works but it is also countinously producing with constant frequency an mp3 file of blank sound. I looked what is happening on liquidsoap and this is one of the incriminated moments:
Code:
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [mksafe:3] Switch to input.external_7218 with transition.
2021/01/04 15:24:12 [safe_blank:4] Activations changed: static=[], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:4] Activations changed: static=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:), (dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:4] End of track.
2021/01/04 15:24:12 [input.external_7218:4] Buffer emptied, buffering needed.
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [mksafe:3] Switch to safe_blank with forgetful transition.
2021/01/04 15:24:12 [input.external_7218:4] Activations changed: static=[(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [safe_blank:4] Activations changed: static=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:3] Source failed (no more tracks) stopping output...
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
Someone could explain me what is happening? Is that counter from 0 to 2000 the buffer?
How could I avoid that everytime a blank mp3 file is written?

This is what I use in the op25.liq to dump the mp3:
Code:
# dump recordings to a file
time_stamp = '%Y-%m-%d_%H.%M.%S'
output.file(%mp3, "./OUT_#{time_stamp}.mp3", input , fallible=true)
Thank you!
 
Last edited:

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
The mksafe() function is responsible for filling the buffer with silence when op25 is not outputting data. You need to fork the input stream before it passes through mksafe() so that you can use the non-safe stream to send to the mp3 file.
 

miclass

Member
Joined
Dec 27, 2020
Messages
21
The mksafe() function is responsible for filling the buffer with silence when op25 is not outputting data. You need to fork the input stream before it passes through mksafe() so that you can use the non-safe stream to send to the mp3 file.
This is indeed what I am already doing, despite this the non-safe stream produces one short mp3 blank file (just 3760 byte) every time that this counter reach 2000:
Code:
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (1920/2000).
2021/01/04 15:24:12 [mksafe:3] Switch to input.external_7218 with transition.
2021/01/04 15:24:12 [safe_blank:4] Activations changed: static=[], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:4] Activations changed: static=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:), (dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:4] End of track.
2021/01/04 15:24:12 [input.external_7218:4] Buffer emptied, buffering needed.
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [mksafe:3] Switch to safe_blank with forgetful transition.
2021/01/04 15:24:12 [input.external_7218:4] Activations changed: static=[(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [safe_blank:4] Activations changed: static=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)], dynamic=[mksafe:iir_filter_7223:iir_filter_7225:compress_7227:normalize_7230:mksafe:pulse_out(liquidsoap:):pulse_out(liquidsoap:)].
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [(dot)/OUT_%Y-%m-%d_%H(dot)%M(dot)%S(dot)mp3:3] Source failed (no more tracks) stopping output...
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
2021/01/04 15:24:12 [input.external_7218:6] Not ready: need more buffering (0/2000).
So what I get at the end are the mp3 files of only the conversations (that are my objectives, so this is fine) plus a lot of small mp3 files of just 3760 bytes, each minute, more o less. How could I avoid them?

This is now my op25.liq:
Code:
#!/usr/bin/liquidsoap
# Example liquidsoap streaming from op25 to icecast
# (c) 2019, 2020 gnorbury@bondcar.com, wllmbecks@gmail.com
#
set("log.stdout", true)
set("log.file", false)
set("log.level", 9)

# Make the native sample rate compatible with op25
set("frame.audio.samplerate", 8000)

input = input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -s")
# Consider increasing the buffer value on slow systems such as RPi3. e.g. buffer=0.25
# Longer buffer results in less choppy audio but at the expense of increased latency.

# OPTIONAL AUDIO SIGNAL PROCESSING BLOCKS
# Uncomment to enable
#
# High pass filter
input2 = mksafe(input)
input2 = filter.iir.butterworth.high(frequency = 200.0, order = 4, input2)
# Low pass filter
input2 = filter.iir.butterworth.low(frequency = 3250.0, order = 4, input2)
# Normalization
input2 = compress(input2, attack = 2.0, gain = 0.0, knee = 13.0, ratio = 2.0, release = 12.3, threshold = -18.0, rms_window = 1.0)
input2 = normalize(input2, gain_max = 6.0, gain_min = -6.0, target = -18.0, threshold = -65.0)
input2 = ladspa.tap_limiter(input2, limit_level = -1.0)
input2 = limit(input2, threshold = -0.2, attack = 2.0, release = 25.0, rms_window = 0.02)

# LOCAL AUDIO OUTPUT
# Uncomment the appropriate line below to enable local sound
#
# Default audio subsystem
out (input2)
#
# PulseAudio
#output.pulseaudio(input)
#
# ALSA
#output.alsa(input)

# dump recordings to a file
time_stamp = '%Y-%m-%d_%H.%M.%S'
output.file(%mp3, "./OUT_#{time_stamp}.mp3", input , fallible=true)
One additional note, I also added:
Code:
input2 = ladspa.tap_limiter(input2, limit_level = -1.0)
input2 = limit(input2, threshold = -0.2, attack = 2.0, release = 25.0, rms_window = 0.02)
It seems to have solved the clipping I was having sometime at the begining of the conversations.

Thanks.
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
Does the "buffer=0.25" have anything to do with 2000 counter? (8000Hz * 0.25sec = 2000 samples)
What happens if you make it smaller?
 

miclass

Member
Joined
Dec 27, 2020
Messages
21
Does the "buffer=0.25" have anything to do with 2000 counter? (8000Hz * 0.25sec = 2000 samples)
What happens if you make it smaller?
It decreases - yes it's related to the buffer.
Point is that whenever the buffer ends the output.file believes that the stream has been closed and it close the mp3 file (empty, with just some bytes, I think only the header and tags of the file, it's 3760 bytes) and creates a new one.
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
It decreases - yes it's related to the buffer.
Point is that whenever the buffer ends the output.file believes that the stream has been closed and it close the mp3 file (empty, with just some bytes, I think only the header and tags of the file, it's 3760 bytes) and creates a new one.
Sounds like a liquidsoap peculiarity. I don't think there's much if anything I can do to op25 to mitigate/change that behavior.
 

miclass

Member
Joined
Dec 27, 2020
Messages
21
Sounds like a liquidsoap peculiarity. I don't think there's much if anything I can do to op25 to mitigate/change that behavior.
Thank you for your answer anyway.
For the moment I solved in a not so elegant way (bash script in crontab that every hour delete all mp3 files smaller than 4k :) ). Perhaps someone with some more experience with Liquidsoap will answer. I tried to ask also on the Liquidsoap mailing list. If I will find a better solution I will report it here.
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
I was interested in this thread because I am using a basic setup that gives me audio either from the Pi jack, Icecast, and dump the audio file. However the mp3 files end up having a lot of silence. I would like the mp3 file to just contain audio.
Reading through this thread I see boatbod mentions forking the audio. Looking at miclass's op25.liq example. I thought the line
input2 = mksafe(input)
would accomplish this. However op25.liq terminates with an error at my line:
output.alsa(input2). It works in my current op25.liq that has output.alsa(input).

So I guess I am missing a step with the forking process. What am I missing?
Thanks
Kevin
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
I was interested in this thread because I am using a basic setup that gives me audio either from the Pi jack, Icecast, and dump the audio file. However the mp3 files end up having a lot of silence. I would like the mp3 file to just contain audio.
Reading through this thread I see boatbod mentions forking the audio. Looking at miclass's op25.liq example. I thought the line
input2 = mksafe(input)
would accomplish this. However op25.liq terminates with an error at my line:
output.alsa(input2). It works in my current op25.liq that has output.alsa(input).

So I guess I am missing a step with the forking process. What am I missing?
Thanks
Kevin
mksafe() fills a stream with silence when there is no input. It's necessary for an icecast output to keep the stream alive, but not required when sending to a file or audio output device.

Try increasing liquidsoap logging level to 3 and running the .liq script from the command line. It should then provide more info on what it doesn't like about the script. Post that here along with the full script and maybe we can figure out what's going on.
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
I found where I had one error that was just a "doh!" error. Of course it generated a different one that has me really baffled. I will admit I am kind of ignorant other than the small the LiquidSoap manual.

My op25.liq that works has this line near the top:
input = mksafe(input.external(buffer=1.0, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -x 2.0 -s"))

What I changed it to is this:
input = input.external(buffer=0.25, channels=2, samplerate=8000, restart_on_error=false, "./audio.py -x 2.0 -s")
input2 = mksafe(input)

It is failing on the first line with of the change with:

Invalid value at line 21, char 45:
Incompatible number of channels, please use a conversion operator..


What am I missing or doing wrong. All I did was remove the "mksafe" function from above to below. What I have is effectively the same as the example. Character 45 is

I did try it without the "-x 2..0" as the example didn't have that but it didn't make any difference anyway.

Really stumped now.

Any ideas?
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
boatbod - thank you for the reply. I gave it a try, I had my doubts because the original form does. I got a "input.external.raw audio undefined variable" error. Did a little digging and I am running liquidsoap 1.3.3, that function seems to be new with liquidsoap 2.0. But it was worth a shot.! Actually I don't see plain "input.external" in 2.0, so you might want to keep that in mind going forward.

I need to take a fresh look tomorrow.
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
boatbod - thank you for the reply. I gave it a try, I had my doubts because the original form does. I got a "input.external.raw audio undefined variable" error. Did a little digging and I am running liquidsoap 1.3.3, that function seems to be new with liquidsoap 2.0. But it was worth a shot.! Actually I don't see plain "input.external" in 2.0, so you might want to keep that in mind going forward.

I need to take a fresh look tomorrow.
Yes, the upgraded liquidsoap is going to give me issues because the old scripts break in several ways.
I cannot think of a reason that would make the number of channels incorrect for input.external() but ok when wrapped in mksafe(). It's not as if input.external is single channel. ?????
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
boatbod - that is the same theory as a guy I know who runs OP25 (and way smarter on this stuff) suggested. I am going to try that when I get home tonight and report back. But right - it made no sense to me either!
The only purpose is to record mp3 without silence. If there is another way that would be cool (without a second sdr).
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
Well I did finally get it to work, just by using my orig op25.liq and modifying it. However, the end result was a huge file (654 MB) of nothing, just using the raw input (except for what is done by the input.external line without the mksafe() ).

I wish I could figure out how I could remove the silence automatically, or not write it. Admittedly I am ignorant on this stuff and my old brain is just too full of junk to digest what does what and how with liquidsoap.

I do appreciate the assistance by you while I tried.

Kevin
 

krutzy

Member
Joined
Sep 17, 2004
Messages
16
Location
Culpeper, VA
I gave it the old college try. Neither "eat_blank" nor "skip_blank" like the fact that the input is "fallible", nor do they accept a fallible parameter.
I am sure there is away around it by doing a bunch of scripting, but that is beyond my abilities.
Even if I run "sox" or another audio processor, they gobble up all the cpu. But the output is still useful doing it manually.
Thanks for the idea!
 

boatbod

Member
Joined
Mar 3, 2007
Messages
2,727
Location
Talbot Co, MD
I gave it the old college try. Neither "eat_blank" nor "skip_blank" like the fact that the input is "fallible", nor do they accept a fallible parameter.
I am sure there is away around it by doing a bunch of scripting, but that is beyond my abilities.
Even if I run "sox" or another audio processor, they gobble up all the cpu. But the output is still useful doing it manually.
Thanks for the idea!
Ok, so what happens if you mksafe() the input then pass it through eat_blank()?
 
Top