Next step would be to try and do the decoding on the BBB as well but I suspect it won't do that due to its limited resources but I'll try anyway.
telive uses very little cpu and memory, so this won't be a problem. if you don't have enough cpu, then you will get hiccups in the received audio, but there is a workaround:
disable live playing in telive
modify tetrad to play the files that it processes. you can also skip the ogg encoding and save the files as wav. (you can also write some other script to play back the files once they are saved on disk).
since the files that you want to play are already on disk, playback will be much smoother. of course this will introduce a lot of delay, but should work better on slow hardware