23 Jul
2014
23 Jul
'14
10:46 p.m.
Hi,
I'm pretty new to Linux audio, but a project I'm working on is going to
need me to generate musical notes on a remote Linux server, in
(near-)realtime (i.e. changing the sound output based on realtime events).
I have basically no familiarity with MIDI etc., but the easiest interface
I've found for the starting and stopping of notes, so far, is the
FluidSynth API. That's not the issue, however: the trick is getting the
audio back to my Windows machine with low-ish latency, and in reasonable quality.
To add a little interest to the scenario..., my Windows machine is also
behind NAT and a firewall. The ping round-trip time is under 30 ms,
though, so I'm hoping latencies of around 200 ms might be possible.
Just to make life more interesting, I'd rather not have to use X Windows at
all on the remote machine, as I'm blind, and haven't even begun to think
about the ins and outs of accessibility of a remote X server (and, frankly,
would rather just not go there for now).
(The application I'm working on may interest you, BTW: I'm converting
financial data, particularly currency price charts, into soundscapes, for
"visualising" recent price movements. If it were just a one-off, I could
convert it into an audio file and download it; but, instead, I want it to
be interacctive, so I can rewind through the image, step backward or
forward frame by frame, etc. I'm going to have fun with this!)
I've been considering a few methods of getting the audio to my Windows
machine, but none of the options I've come up with so far seem to quite
work as I want. Here's what I've considered.
1. Send audio to JACK, then send it to the Asterisk PBX via the App_Jack
application; listen to the audio via a softphone on the Windows machine,
connecting to the Asterisk server via SIP. Sound quality wouldn't be ideal
- 8KHz mono - but I could live with that. Unfortunately, it looks like
App_Jack doesn't successfully connect to a JACK server: a bug report was
lodged about it over 3 years ago, marked severe, but doesn't seem to have
been fixed as yet.
2. Send audio via Jack and a streamer (probably DarkIce) to an Icecast2
server; then use a media player such as WinAmp or MPlayer. The problem
here is buffering: 10-20 seconds of latency just isn't acceptable for the
realtime aspects of the project, and I don't know if there's a way to get a
media player to do unbuffered playback (a quick Google search seemed to
indicate not).
3. Use netJACK1 to play audio direct from JACK (that's the only one of the
3 networked-JACK solutions mentioned on the JACK website that's recommended
for use over a WAN). Unfortunately, once I dug into this, it looks like
the assumption is that the audio will be played on a Linux box: there
doesn't seem to be any provision for having the Windows machine as the one
doing the audio playing. I did have a quick look at netJACK2, but that
seems to assume the two machines are on the same LAN - you don't even
specify the IP address to connect to!
4. I wondered about using PulseAudio. However, that seems quite tricky to
set up without X on the remote machine, and I'm not at all clear about the
NAT traversal issue.
5. I also wondered about NAS: I gather it's very outdated, but it sounds
like about what I'm after. However, I couldn't find a Windows download;
and FluidSynth doesn't have a NAS interface, as far as I understand it.
TiMidity does, though, so I'd just need to work out how to send it MIDI
events from software...
Tunneling through a forwarded port (using ssh) may help in some aspects,
but I'm not sure. AT least, it adds an extra set of possibilities, as do
VPNs.
As you can see, I've been wracking my brains over this for a couple of
days, and while I've felt I'm really close on more than one occasion, I've
not quite got there. Any input would be very much appreciated!
Alternatively, if there's a better place for me to be asking this question,
a pointer would be very helpful.
Cheers,
Nikhil.
24 Jul
24 Jul
12:29 a.m.
Nikhil Nair, Jul 23 2014:
Hey Nikhil,
I've only dabbled with it once or twice. FWIW:
...
2. Send audio via Jack and a streamer (probably
DarkIce) to an Icecast2
server; then use a media player such as WinAmp or MPlayer. The problem
here is buffering: 10-20 seconds of latency just isn't acceptable for the
realtime aspects of the project, and I don't know if there's a way to get a
media player to do unbuffered playback (a quick Google search seemed to
indicate not).
I managed latencies of 1-5 seconds, if I remember correctly.
Basically tweaking the encoding options. Unfortunately I can't find my personal XML
config files. But I pulled all that from the icecast2 and ices/darkice manuals. If you
choose this, search for Jörn Nettingsmeier, Linux Audio Conference and Karl Heiss in
conjunction with icecast. They've done very useful and skilled work on this!
3. Use netJACK1 to play audio direct from JACK
(that's the only one of the
3 networked-JACK solutions mentioned on the JACK website that's recommended
for use over a WAN). Unfortunately, once I dug into this, it looks like
the assumption is that the audio will be played on a Linux box: there
doesn't seem to be any provision for having the Windows machine as the one
doing the audio playing. I did have a quick look at netJACK2, but that
seems to assume the two machines are on the same LAN - you don't even
specify the IP address to connect to!
If you can find JACK2 under windows, you can
use a VPN tunnel. Openvpn isn't too difficult under Linux. I've heard, that it
isn't too bad setting it up under windows either.
My last experience is, that you will need closely matching JACK versions, since the
streaming codec had a few incompatible changes over time.
...
Tunneling through a forwarded port (using ssh) may
help in some aspects,
but I'm not sure. AT least, it adds an extra set of possibilities, as do
VPNs.
Depending on your detailed specs, you can use SSH tunneling to send MIDI data
from the Linux machine using a windows synthesizer. That removes streaming the audio and
reducs latency compared to most of the methods mentioned above. I tried it between Linux
machines on different networks using the ALSA sequencer system and a tunnelled port for
aseqnet. Latency was less than a second. There are other network MIDI frameworks, which
might be better suited to interact with foreign operating systems.
Here's an untested idea: if you have a webserver installed on your machine or another
way to access the system in realtime, you could try encoding your audio to a named FIFO or
pipe (mkfifo) and simply access that FIFO. It will look like a file and be accessible like
a file. Possible issues: you might not catch enough data at a time to receive meaningful
info. Encoded audio is sent in packages. There are important blocksizes to behold. Or
something like it. The FIFO might not yield the data in such a way, that you can read it
over the network. -- Hm, basically all the issues, for which streaming applications have
been written. :)
Good luck and better answers!
...
Ta-ta
----
Ffanci
* Internet: http://freeshell.de/~silvain
4:03 p.m.
Hi Ffanci,
Thanks for all of that - some excellent food for thought there.
Another idea that's occurred to me is to use a Linux virtual machine (under
VMware Player) on my Windows machine: that seems to open up possibilities,
and removes the whole cross-platform hassle. I'll have to see, but
generating the audio locally may even then become a possibility, and all
the latency issues would then go away.
So, I have some fiddling around to do, but it looks like I'm getting
somewhere. May well use OpenVPN somewhere in that setup, too - we'll see.
Cheers,
Nikhil.
On Thu, 24 Jul 2014, F. Silvain wrote:
Nikhil Nair, Jul 23 2014:
Hey Nikhil,
I've only dabbled with it once or twice. FWIW:
...
2. Send audio via Jack and a streamer (probably
DarkIce) to an Icecast2
server; then use a media player such as WinAmp or MPlayer. The problem
here is buffering: 10-20 seconds of latency just isn't acceptable for the
realtime aspects of the project, and I don't know if there's a way to get a
media player to do unbuffered playback (a quick Google search seemed to
indicate not).
I managed latencies of 1-5 seconds, if I remember correctly.
Basically
tweaking the encoding options. Unfortunately I can't find my personal XML
config files. But I pulled all that from the icecast2 and ices/darkice
manuals. If you choose this, search for Jörn Nettingsmeier, Linux Audio
Conference and Karl Heiss in conjunction with icecast. They've done very
useful and skilled work on this!
3. Use netJACK1 to play audio direct from JACK
(that's the only one of the
3 networked-JACK solutions mentioned on the JACK website that's recommended
for use over a WAN). Unfortunately, once I dug into this, it looks like
the assumption is that the audio will be played on a Linux box: there
doesn't seem to be any provision for having the Windows machine as the one
doing the audio playing. I did have a quick look at netJACK2, but that
seems to assume the two machines are on the same LAN - you don't even
specify the IP address to connect to!
If you can find JACK2 under windows, you can
use a VPN tunnel. Openvpn isn't
too difficult under Linux. I've heard, that it isn't too bad setting it up
under windows either.
My last experience is, that you will need closely matching JACK versions,
since the streaming codec had a few incompatible changes over time.
...
Tunneling through a forwarded port (using ssh)
may help in some aspects,
but I'm not sure. AT least, it adds an extra set of possibilities, as do
VPNs.
Depending on your detailed specs, you can use SSH tunneling to send MIDI
data
from the Linux machine using a windows synthesizer. That removes streaming
the audio and reducs latency compared to most of the methods mentioned above.
I tried it between Linux machines on different networks using the ALSA
sequencer system and a tunnelled port for aseqnet. Latency was less than a
second. There are other network MIDI frameworks, which might be better suited
to interact with foreign operating systems.
Here's an untested idea: if you have a webserver installed on your machine or
another way to access the system in realtime, you could try encoding your
audio to a named FIFO or pipe (mkfifo) and simply access that FIFO. It will
look like a file and be accessible like a file. Possible issues: you might
not catch enough data at a time to receive meaningful info. Encoded audio is
sent in packages. There are important blocksizes to behold. Or something like
it. The FIFO might not yield the data in such a way, that you can read it
over the network. -- Hm, basically all the issues, for which streaming
applications have been written. :)
Good luck and better answers!
...
Ta-ta
----
Ffanci
* Internet: http://freeshell.de/~silvain
4:10 p.m.
Hey Nikhil,
a VM is a good idea. ALSA works quite well, I've even used JACK with one of
those and I've heard that a braille display will also work, if registered from
the windows side inside VMWare. Fluidsynth has both ALSA and JACK so no
problem with that.
Good luck!
Ta-ta
----
Ffanci
* Internet: http://freeshell.de/~silvain
3574
days inactive
3575
days old
linux-audio-user@lists.linuxaudio.org
3 comments
2 participants
participants (2)
-
F. Silvain -
Nikhil Nair