9 Sep
2014
9 Sep
'14
10:58 p.m.
I thought I would move this over here. I know there is already work being
done on this hw wise. These thoughts are for a point to point raw
ethernet audio transport That still allows some normal network traffic as
well.
On Tue, 2 Sep 2014, Len Ovens wrote:
My thought is something like this:
We control all network traffic. Lets try for 4 words of audio. For sync
purposes, at each word boundry a short audio packet is sent of 10 channels.
This would be close to minimum enet packet size. Then there should be room
for one full size enet packet, in fact even at 100m the small sync packet
could contain more than 10 channels (I have basically said 10m would not be
supported, but if no network traffic was supported then 10m could do 3 or 4
channels with no word sync). So:
Word 1 - audio sync plus 10 tracks - one full network traffic packet
word 2 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 1 and 2
wors 3 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 2 and 3
word 4 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 3 and 4
Nobody commented that this could not work :) 4 samples on a 100mbit link
is still less than one full 1500byte data packet. The reason I am thinking
about this right now, is that my studio has been flooded :P and so I
have no access to work on my control surface project right now.
So some background thinking:
- The idea is to replace FW audio interfaces with something at least as
good, maybe better.
- Really low latency available (even if a lot of uses don't need it)
- Really stable operation. On a desktop/rack computer where the
user has access to a PCIe slot, it is obvous that a second NIC would be
the best solution. Laptops should work too.
- Normal network traffic will make it through this mess without ever
disturbing the audio. A laptop may be used with only one NIC and still
need to access network traffic.
- It should "just work" on newer network hw as it is developed.
- It should handle a switch in the middle for use as a range extender, but
never as a network traffic mixer. (on a 1gbit link other traffic may not
disturb things with low enough channel count) What I have thought of so
far would tend to ignore other traffic anyway and a switch should end up
just sending our traffic through our ports. This situation may require
some user intervention such as pointing out which box they wish to connect
to.
- It should deal well with hot plugging. This feels messy, but it should
be possible to let things like netman play with things first and still be
able to detect an audio IF has been connected and reinit the interface for
this use.
In the end, the kernel module for this device should be loaded anytime a
NIC is detected. (detected and has a connection) It should create both an
eth* device and an ALSA device, but should only do so if it detects an
audio IF online. To begin with, this would mean the user would have to do
some setup to get around all the auto setup stuff already running (dhcp
etc.) But as the IF was used more, normal networking stuff might be
expected to detect an audio IF and leave it alone. Maybe the audio IF
could have a dhcp server that refused an IP so that the dhcp client gives
up and goes away. That is, use network protocols that are already
available when possible.
This whole effort assumes the ethernet device is connected by twisted
pair from the host computer to the audio device with a separate path for
each direction. This is very important as it _should_ make for a colision
free environment. This interface will control both audio and data flow.
Any network traffic will only be sent during times audio sending is not
needed. This can be done because all network data will go through the
audio driver. There are still a lot of 100mbit networks out. Lots of new
equipment still has them too. I have chosen a 4 sample frame at 48k (which
happens to be 16 samples for 192k... if you must) because it seems to be
the lowest latency with reasonable use of overhead. (on Gbit and higher
lines, this is no longer true. The internet still runs on 1500MTU and more
than one full packet will travel in one sample's time) I have done all my
calculations based on a 100mbit line because that happens to be what I
have to play with and appears to be able to handle up to 60 audio channels
with some left over for control. I understand that there are venues that
use more, but gbit links will handle lots more. (600 plus, but realize the
systems on each end have to be able to deal with the data as it comes in,
it is not just about the link capabilities)
So, My thought is that each group of packets sent will be timed by a group
of 4 samples. The driver will attempt to send a packet with 4 samples
worth of audio for all channels at the end of those 4 samples. The driver
then calculates how much time it has before another 4 sample times are up
and sends as much data as it has time for. This calculation should be able
to happen only one time for any channel number setup if the hw is not
doing anything too fancy (like waiting for more than one packet before it
sends). It assumes the hardware/driver uses standard size guard bands,
etc. SO for each 4 samples there would be two packets minimum (probably
maximum too for a 100m link). One audio and one data. On a 100m link these
packets would always have an mtu of less than 1500. It would not be
possible to use the arrival of an audio packet as a sync signal, sync as
always, would be an external line if two audio interfaces needed to be
used.
I expect to have an atom based MB to play with soon that has two NICs on
board. as well as an audio IF. This will not be a true test low latency
because the onboard AI has higher latency to begin with (runs at p64/n3
min) and so I will have 192 samples to play with at a time which I will
still try sending in 4 sample bundles. (I may try putting an older PCI AI
in to see if I can get that down a bit) My thought is to make the AI side
a jack client (I think I can do that much :) and the host side an ALSA
device (something new to learn).
All control will be MIDI-able. Because there is two NICs and one of them
expects to do real IP based networking, OSC is possible as well as web
based control. IN the end this is also a general computer running Linux
that can be SSHed into (even ssh -Y) almost anything is possible... but I
would guess the first box that has a real DIY digital CODEC, S/pdif, ADAT
or MADI IF will be pretty basic.... but looking at the R-Pi, basic seems
to be pretty powerful anymore.
In the long run this could be a very interesting device. There is no
reason this could not also be, an effects box (both with local analog
ports as well as through net), a softsynth (most of these boards have at
least one serial port or USB), A remote mixer... drop the box FOH and use
a networked control surface, Android pad... even a browser to control, a
FOH snake box or even a standalone recording device.
Price point? Concidering ethernet switches, USB AUdio devices, Ethernet
storage controllers, set top... I hesitate to call them boxes some of
them are so small? Even development boards look ok. I don't think It would
be worth while to make a two i/o box, but by the time we hit 8 or so it
begins to look good.
--
Len Ovens
www.ovenwerks.net
10 Sep
10 Sep
4:30 p.m.
New subject: [LAU] Open Source Audio Interface
On Tue, Sep 09, 2014 at 03:57:06PM -0700, Len Ovens wrote:
I thought I would move this over here. I know there is
already work
being done on this hw wise. These thoughts are for a point to point
raw ethernet audio transport That still allows some normal network
traffic as well.
Zita-njbridge will do this, point to point or one to many.
Each system can have its own sample rate and period size,
no word clock sync is assumed.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
11 Sep
11 Sep
4:12 p.m.
New subject: [LAU] Open Source Audio Interface
On Wed, 10 Sep 2014, Fons Adriaensen wrote:
On Tue, Sep 09, 2014 at 03:57:06PM -0700, Len Ovens
wrote:
Yes, this project would not replace that functionality. It is rather a
fully synced system where all audio inputs would be able to be used in the
same acoustic space. This is not a generic audio over ethernet protocol,
but a connection for an open audio IF to a computer where the performance
is extected to be as good as FW.
--
Len Ovens
www.ovenwerks.net
I thought I would move this over here. I know
there is already work
being done on this hw wise. These thoughts are for a point to point
raw ethernet audio transport That still allows some normal network
traffic as well.
Zita-njbridge will do this, point to point or one to many.
Each system can have its own sample rate and period size,
no word clock sync is assumed.
9:17 p.m.
New subject: [LAU] Open Source Audio Interface
On Thu, Sep 11, 2014 at 09:10:36AM -0700, Len Ovens wrote:
On Wed, 10 Sep 2014, Fons Adriaensen wrote:
If any audio HW would transmit the packet format expected by
zita-n2j (which is fairly simple), it would just work. You
could even have many such interfaces connected at the same
time (all it takes is a network switch) and they wouldn't
need a common clock.
And of course it is possible to create a Jack backend that
accepts the same format but without the resampling (that's
in the pipeline actually).
Re. the format used by njbridge: for IPv4 the IP and UDP
headers together take 28 bytes. That is less than 2 percent
of 1500, and is a small price for having packets that can be
handled by switches and routers. There is really no point in
trying to reduce that sort of overhead. The njbridge format
itself adds a 20 byte header to sample packets. This data is
used to identify the packet, to improve the timing and handle
skipped cycles, xruns, lost packets and the like. All together
the overhead is less than 3.5%.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
On Tue, Sep 09, 2014 at 03:57:06PM -0700, Len
Ovens wrote:
Yes, this project would not replace that functionality. I thought I would move this over here. I know
there is already work
being done on this hw wise. These thoughts are for a point to point
raw ethernet audio transport That still allows some normal network
traffic as well.
Zita-njbridge will do this, point to point or one to many.
Each system can have its own sample rate and period size,
no word clock sync is assumed. 
9:29 p.m.
New subject: [LAU] Open Source Audio Interface
On Thu, Sep 11, 2014 at 09:17:26PM +0000, Fons Adriaensen wrote:
Re. the format used by njbridge: for IPv4 the IP and
UDP
headers together take 28 bytes. That is less than 2 percent
of 1500, and is a small price for having packets that can be
handled by switches and routers. There is really no point in
trying to reduce that sort of overhead. The njbridge format
itself adds a 20 byte header to sample packets. This data is
used to identify the packet, to improve the timing and handle
skipped cycles, xruns, lost packets and the like. All together
the overhead is less than 3.5%.
But then you *must* take care of timing, since you have no idea if packets passed through
a router will arrive in order, how long they'll take to traverse it, or even if
they'll arrive unmangled at all.
If you are going to strip it down, just go with bare Ethernet frames :-)
--
Gordonjcp MM0YEQ
9:45 p.m.
New subject: [LAU] Open Source Audio Interface
On Thu, Sep 11, 2014 at 10:29:24PM +0100, gordonjcp(a)gjcp.net wrote:
On Thu, Sep 11, 2014 at 09:17:26PM +0000, Fons
Adriaensen wrote:
The format used by njbridge takes can handle all of that. The current
receiver (n2j) discards packets that arrive out of order (they will
have been replaced by silence before they arrive) but it would be
possible to re-insert them. But being able to do that implies more
latency - a packet can't arrive out of order without being late as
well.
Re. the format used by njbridge: for IPv4 the IP
and UDP
headers together take 28 bytes. That is less than 2 percent
of 1500, and is a small price for having packets that can be
handled by switches and routers. There is really no point in
trying to reduce that sort of overhead. The njbridge format
itself adds a 20 byte header to sample packets. This data is
used to identify the packet, to improve the timing and handle
skipped cycles, xruns, lost packets and the like. All together
the overhead is less than 3.5%.
But then you *must* take care of timing, since you have no ideai
if packets passed through a router will arrive in order, how long
they'll take to traverse it, or even if they'll arrive unmangled
at all. If you are going to strip it down, just go with bare
Ethernet frames :-)
What I wanted to make clear is that there is really no point
in doing that.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
10:27 p.m.
New subject: [LAU] Open Source Audio Interface
On Thu, 11 Sep 2014, Fons Adriaensen wrote:
If any audio HW would transmit the packet format
expected by
zita-n2j (which is fairly simple), it would just work. You
Yes it does.
could even have many such interfaces connected at the
same
time (all it takes is a network switch) and they wouldn't
need a common clock.
Ok, Could I have two mics, one left and one right each going through two
of these interfaces without a common clock and sound right? Or even both
mono but fairly close to each other? Assuming all our audio channels are
in the same packet stream would resampling still allow them to at least be
syncronised to each other?
Switch: What kind of latency is required? Packet A from AI A would then be
delayed by packet B from AI B at the switch. The switch does not know the
difference between audio and data, so common data may delay it further. At
least the switch should prevent collisions though.
And of course it is possible to create a Jack backend
that
accepts the same format but without the resampling (that's
in the pipeline actually).
Ah, I may wait then. This seems to answer my biggest doubt.
Re. the format used by njbridge: for IPv4 the IP and
UDP
headers together take 28 bytes. That is less than 2 percent
of 1500, and is a small price for having packets that can be
handled by switches and routers. There is really no point in
Raw ethernet packages should have no problem with a switch. I would
question the use of a router, but already installed infrastructure might
have it sitting there anyway. For our use case (an audio interface) there
would be no router. Possibly a switch may be used if the AI was far enough
from the host.
trying to reduce that sort of overhead. The njbridge
format
itself adds a 20 byte header to sample packets. This data is
used to identify the packet, to improve the timing and handle
skipped cycles, xruns, lost packets and the like. All together
the overhead is less than 3.5%.
To be honest the only thing that has really been a stopper for njbridge in
this application has been the resampling. Making it work as well when sync
is available would fix that.
reducing overhead only really effects channel count and then only on a
100m link. I personally can't imagine 32 plus channels (not ascribing any
maximum channel count to njbridge, I don't know) for anything I
would do... and where I can imagine lots of channels I also see lots of
money where whatever I need could just be bought, open or no. I was merely
thinking expandability.
Using 1500 byte packets would seem to me to be adding unneeded latency, I
would rather pay the price in overhead I think. At a low channel count it
would take a number of sample times to fill 1500 bytes. (I should be more
specific, I am basing all my calc on 48k and a 100m link where the 1500
byte packet would already be ~6 samples long) Assuming the same 32 bits
per sample/channel, 8 channels is only 32 bytes per sample. we would be
looking at around 45 samples worth of time to fill the buffer and then add
6 more samples assuming there is already some network data in the que..
about 1ms. Perhaps that is not too bad then. Higher channel counts would
have lower possible latency. I really can't speak for the cpu time
involved in sending/receiving (making/breaking packets) but expect it
could be minimal.
--
Len Ovens
www.ovenwerks.net
7:54 a.m.
New subject: [LAU] Open Source Audio Interface
Hi Len,
This concerns me greatly so I thought I'd just comment.
2014-09-10 0:57 GMT+02:00 Len Ovens <len(a)ovenwerks.net>et>:
<...>
So some background thinking:
- The idea is to replace FW audio interfaces with something at least as
good, maybe better.
Good stuff!
- Really low latency available (even if a lot of uses
don't need it)
- Really stable operation. On a desktop/rack computer where the user has
access to a PCIe slot, it is obvous that a second NIC would be the best
solution. Laptops should work too.
- Normal network traffic will make it through this mess without ever
disturbing the audio. A laptop may be used with only one NIC and still need
to access network traffic.
If it complicates stuff I would be all for removing this requirement,
instead have a dedicated NIC. Laptops have wireless and potentially
having an additional NIC in a desktop should be low cost.
<..>
All control will be MIDI-able. Because there is two NICs and one of them
expects to do real IP based networking, OSC is possible as well as web based
control. IN the end this is also a general computer running Linux that can
be SSHed into (even ssh -Y) almost anything is possible... but I would guess
the first box that has a real DIY digital CODEC, S/pdif, ADAT or MADI IF
will be pretty basic.... but looking at the R-Pi, basic seems to be pretty
powerful anymore.
I see, so the "box" will be a dedicated computer. There would need to
be some custom hardware to wire in AD/DA?
<..>
Price point? Concidering ethernet switches, USB AUdio
devices, Ethernet
storage controllers, set top... I hesitate to call them boxes some of them
are so small? Even development boards look ok. I don't think It would be
worth while to make a two i/o box, but by the time we hit 8 or so it begins
to look good.
8 or more is where it starts to get interesting, below that I think
USB suffices.
I'm guessing this is early days for this project, is there anything
you know you want help with?
Regards,
Robert
--
Len Ovens
www.ovenwerks.net
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
3:35 p.m.
New subject: [LAU] Open Source Audio Interface
On Thu, 11 Sep 2014, Robert Jonsson wrote:
Any development will start with a dedicated link (NIC). However, leaving
the posibility of network traffic as well would be worthwhile and I
suspect in the end, not that difficult. The hard part of the kernel net
driver, dealing with the HW, has already been done. Adding another net
device is a matter of accepting and dispensing data as if it was another
audio channel. Because we have complete control of the link, any data can
be sent as is, even chopped up packets, so long as they get put back
together at the end... yes I make it sound more trivial than it is :)
All USB and FW AUdio IFs already are a dedicated computer with custom
codec wiring. The thing is, these dedicated computers have gotten much
more powerful at the price we are looking at. There is actually already
work being done on this kind of HW. and there are many audio projects of
the standalone variety as well (notably with the RP). embeded linux is
quite popular and often is already available for much of this lowcost HW.
- Really low
latency available (even if a lot of uses don't need it)
- Really stable operation. On a desktop/rack computer where the user has
access to a PCIe slot, it is obvous that a second NIC would be the best
solution. Laptops should work too.
- Normal network traffic will make it through this mess without ever
disturbing the audio. A laptop may be used with only one NIC and still need
to access network traffic.
If it complicates stuff I would be all for removing this requirement,
instead have a dedicated NIC. Laptops have wireless and potentially
having an additional NIC in a desktop should be low cost. All control
will be MIDI-able. Because there is two NICs and one of them
expects to do real IP based networking, OSC is possible as well as web based
control. IN the end this is also a general computer running Linux that can
be SSHed into (even ssh -Y) almost anything is possible... but I would guess
the first box that has a real DIY digital CODEC, S/pdif, ADAT or MADI IF
will be pretty basic.... but looking at the R-Pi, basic seems to be pretty
powerful anymore.
I see, so the "box" will be a dedicated computer. There would need to
be some custom hardware to wire in AD/DA? 8 or more is where it starts to get interesting, below
that I think
USB suffices.
I'm guessing this is early days for this project, is there anything
you know you want help with?
Karl has already started working on HW. He has set up a project for it.
Mostly getting to know the HW. I do not know if he has started on the
CODEC IF yet. I think He is planning on running a jack server on it. The
plan being to use netjack for his first trial. However, If I can have a
jack client ready by the time he has other things done, raw enet could be
used from the start. I am pretty sure I can do the jack client to raw
ethernet driver. The other end, I am sure of the ethernet part, but less
sure of the ALSA part. The other way to go would be a jack backend... but
the problem with that is the whole jack1/2 thing. I do not know if a jack
client can be a sync master (in free run mode maybe?).
Very much early days. Because of the changes in computer hardware, Linux
audio is very much drying up in the (semi)pro area. ALSA has done a great
job keeping up with internal audio, but the PCIe cards that are about are
mostly not supported and the USB AIs are having problems that are really
not related to the alsa drivers anyway. I have seen one USB3 device:
https://www.blackmagicdesign.com/ca/products/ultrastudiousb3
Which has 4 analog i/o, 1 pair aes3 and 8 more from audio embedded on the
video for a total of 14 i/o. It also accepts 8 channels of HDMI audio
input, but I do not know if they replace the video lines or not. I also do
not know if there is any hardware that takes 8 audio lines and drops them
onto HDMI. I am pretty sure there are no Linux drivers (or OSx for that
matter).
My thought is that if there is a very high quality way of using the NIC
for this, and if it was open and works with any OS, that Audio IF
manufacturers might pick it up and use it because it doesn't cost them
anything to do so. Much the same as many small boxes use linux as an OS.
My thought is that if the comtrol method uses an open protocol like midi,
even if the AI has a mixer in it, it would not be hard to figgure out what
midi command does what inside the box even if it was not published and if
a sysex midi was used to load plugins to the AI (many AIs use plugins now)
it would not matter if windows or linux was used to transfer the plugin...
even if it is a binary blob. This has been the dream paragraph BTW :)
Anyway, there is a very alive DYI community. The audio IF seems to be a
hole. There are CODEC projects (audio in or out, to from s/pdif), but not
the computer IF. This could fill that hole. I think there are two possible
ways to forward with an Audio IF. USB3 might be a possibility if there was
a standard for audio... thing is there will be and whatever I could come
up with will not be it. PCIe is not out of the question either, but would
limit the use to desktop. ethernet is wide open.... though what happens in
the next gen laptops if they decide that ethernet port isn't needed? How
good would ethernet through USB3 be? Or through bluetooth? The comment
about people can use wireless for networking is a thought that is not lost
on laptop makers trying to cut costs I am sure. Are there enough people who
demand wired net to keep it there?
USB3 network adapters: This is not so bad as it seems. If there is a USB3
client port on a project, the ethernet over USB3 stuff should be pretty
standard anyway and could be used straight over for audio in the same way
as USB MIDI has. This would sidestep the USB3 audio standard thing.
In the long run, I could see most of the audio work being done on the
"box" itself and the host computer being more of a display/control thing.
Again, the easiest thing to point to is the R-pi projects which are
already doing this. It is a move from using the desk/laptop for audio to a
specialized box. This idea is not new either, many of the digital FOH
mixers are exactly that (including multi track recording). The real
difference in what we are trying to do is "open". It used to be that
anything electronic came with a schematic... we are lucky any more to get
a block diagram. This is an audio IF that come with the schematic, both
hard and soft.
--
Len Ovens
www.ovenwerks.net
7:46 p.m.
New subject: Open Source Audio Interface
What I had in mind for ethernet transport is this:
Raw EN packets are very simple:
dest-MAC,send-MAC,type,data
There is more, but that is part of the HW driver and the linux raw EN
interface just requires those sets of data. If I could be sure the HW
would support promiscuous mode, I would tend to use a MAC of 1 and 2, but
I think some hw will not allow that (is this true?). However, EN like IP
has broadcast addresses that can be used to discover the MAC at both ends.
Each MAC is 6 bytes (octets in netspeak), The type is two bytes and is
either the data size if 1500 or less or a type if more. Having size is
kind of nice, but probably not needed as the hw level keeps track anyway.
Anyway, in the end we are left with a data packet from 46 to 1500 bytes.
At 100mbit we will not be using 1500 :) 1000 is about it for 4 samples of
48k audio ... 8 samples at 96k and 16 samples at 192k ... So the minimum
latency does not change with sample rate in this case. As happens the
minimum packet of 46 bytes is enough for 46/4/4=2.975 Channels. This gives
us some extra space to send MIDI bytes (9 in fact) if we keep the MIDI
info in the place where the audio data would be. These channels would be
marked as valid but non-audio. This assumes a MADI like encoding where
each sample is stored in sequence as if it was aes 3. That is 32 bits per
sample.
Unlike MADI, empty channels would not be filled with null data, but rather
just not sent. In MADI, 64 channels are always sent even for a payload of
1. In this case we should send multiples of two. There is no reason that
the channel count should not change from frame to frame, but of course the
receiving sw would have to have time (non-real time) to reset itself and
audio sw that doesn't know how to deal with more channels all the sudden
might need restarting too :) In Jack's case, if the first two were set up
as the sound card, the rest could be added as jack clients. On the AI end
they would all be jack clients anyway and jack would see the whole
complement of codecs all the time. I feel it is worth while having jack in
the AI because this would allow routing. There would be no having the
outputs be channels 9 and 10 because that happens to be how s/pdif appears
because those could look to the host like 1 and 2.
I don't know if there is any (psuedo)standard for midi commands for
routing. I also do not know if 128 channels is enough. That is probably
not a problem though as there are 16 channels, so 16 x 128 is lots.
Something like noteoff value where note is the input and value is the
output... hmm, would that be a toggle or two commands, one to connect and
one to release.
MIDI channels: obviously I have set aside one midi channel for device
control. No matter what standard I set up, it is open and changable. Midi
channels can be set up at the user's whim basically. If there are any
physical Midi ports on the AI, they would be available. But if there is a
software synth of some sort on the AI, there could be another line for
that. This is where Jack starts to look really nice as the host end of
things, but ALSA would lend universalness to things. The nice thing about
this project is that problems can be fixed from both ends, The host driver
and the AI driver.
Still a very rough sketch, I know. It probably needs to be written up with
things defined properly etc. However, making sure things work first would
be a good idea.
--
Len Ovens
www.ovenwerks.net
8:19 p.m.
New subject: Open Source Audio Interface
Just some cents...
On Thu, 11 Sep 2014 12:44:45 -0700 (PDT) Len Ovens <len(a)ovenwerks.net>
wrote:
Unlike MADI, empty channels would not be filled with
null data, but
rather just not sent. In MADI, 64 channels are always sent even for a
payload of 1. In this case we should send multiples of two. There is
no reason that the channel count should not change from frame to
frame, but of course the receiving sw would have to have time
(non-real time) to reset itself and audio sw that doesn't know how to
deal with more channels all the sudden might need restarting too :)
In Jack's case, if the first two were set up as the sound card, the
rest could be added as jack clients. On the AI end they would all be
jack clients anyway and jack would see the whole complement of codecs
all the time. I feel it is worth while having jack in the AI because
this would allow routing. There would be no having the outputs be
channels 9 and 10 because that happens to be how s/pdif appears
because those could look to the host like 1 and 2.
Varying channel count per packet/frame means a varying number
of malloc/free per packet -> bad for realtime. Its probably better to
stay with one channel-count once the stream is set up.
Thinking about it some more, I think one of the reasons why several
vendors use dedicated network-devices with dedicated drivers is to
reduce the need for malloc/free in the kernel/driver as much as
possible and just use fixed buffers once the channel-count (and
word-size) are known.
Have fun,
Arnold
9:27 p.m.
New subject: Open Source Audio Interface
On Thu, 11 Sep 2014, Arnold Krille wrote:
Just some cents...
On Thu, 11 Sep 2014 12:44:45 -0700 (PDT) Len Ovens <len(a)ovenwerks.net>
wrote:
I have worded things badly. Audio packet size will always be the same for
a number of channels if they are all audio or some midi. What I meant (if
I did not really say it) is that the receiving end would detect a change
in packet size and reset its buffers etc. to deal with it. Perhaps an
example would help.
the user wishes to record a scratch track - rough voice and guitar (or
keys) to a beat box (click). They set the AI to send only tracks 3 and 7
as tracks one and two and record. Then they want to record an 8 track drum
track so they route the 8 mics to 8 channels. The AI starts sending 8
tracks. The host sets up larger buffers and starts seeing 8 tracks.
That is really a bad example though, because why? It makes more sense to
use the same number of tracks through a session so that mic pres stay with
the same tracks. A better example would be the user plugs in a second AI
in to the first AIs second NIC. Now the first unit looks like it has 16
channels rather than 8, so the packet size increases and this change is
recogized by the host at that time. Or perhaps the user plugs a s/pdif
input to the AI and we only want to send the channel if it has data.
Where I see your info as being useful though, is if I choose to send data
(which could be varying sized packets) I may still wish to set the data
packet size to a fixed size depending on what is left from audio. The
whole network end of things needs to be decoupled from real time.
To be honest, I have spent a lot less time thinking about the network
traffic part of things than the audio part of things. The reason for this
is that the use cases for allowing network through our line is small. As
has already been commented, a desktop can have a second NIC and the laptop
generally has wireless. The main usecase for network through the same IF
might be radio work where one might have a SIP connection (phone line)
going through net. (anyone see others?) Because the AI has a second NIC
and an OS, it should be possible to set up the SIP session on that box.
--
Len Ovens
www.ovenwerks.net
Unlike MADI, empty channels would not be filled
with null data, but
rather just not sent. In MADI, 64 channels are always sent even for a
payload of 1. In this case we should send multiples of two. There is
no reason that the channel count should not change from frame to
frame, but of course the receiving sw would have to have time
(non-real time) to reset itself and audio sw that doesn't know how to
deal with more channels all the sudden might need restarting too :)
In Jack's case, if the first two were set up as the sound card, the
rest could be added as jack clients. On the AI end they would all be
jack clients anyway and jack would see the whole complement of codecs
all the time. I feel it is worth while having jack in the AI because
this would allow routing. There would be no having the outputs be
channels 9 and 10 because that happens to be how s/pdif appears
because those could look to the host like 1 and 2.
Varying channel count per packet/frame means a varying number
of malloc/free per packet -> bad for realtime. Its probably better to
stay with one channel-count once the stream is set up.
Thinking about it some more, I think one of the reasons why several
vendors use dedicated network-devices with dedicated drivers is to
reduce the need for malloc/free in the kernel/driver as much as
possible and just use fixed buffers once the channel-count (and
word-size) are known.
10:10 p.m.
New subject: Open Source Audio Interface
On Thu, 11 Sep 2014 14:24:56 -0700 (PDT) Len Ovens <len(a)ovenwerks.net>
wrote:
Where I see your info as being useful though, is if I
choose to send
data (which could be varying sized packets) I may still wish to set
the data packet size to a fixed size depending on what is left from
audio. The whole network end of things needs to be decoupled from
real time.
That last sentence is a bit contradicting. You want to send audio over
network, you want the audio to be near realtime. But you want to
decouple network from realtime... (In math: A==B && B==C && A!=C)
To be honest, I have spent a lot less time thinking
about the network
traffic part of things than the audio part of things. The reason for
this is that the use cases for allowing network through our line is
small. As has already been commented, a desktop can have a second NIC
and the laptop generally has wireless. The main usecase for network
through the same IF might be radio work where one might have a SIP
connection (phone line) going through net. (anyone see others?)
Because the AI has a second NIC and an OS, it should be possible to
set up the SIP session on that box.
There is a widespread use-case for audio and non-audio network traffic
on the same segment: audio and control/midi messages on the same
segment. And you don't want your fader moves to interrupt your audio.
And your current thinking seems to be about using a dedicated
network-link between PC and audio-interface with a possible chain-link
to other interfaces. But what about a dedicated network (with switch)
to connect multiple PCs and audio-interfaces? Then you have to deal
with lots more channels, thus lots more realtime data and also lots
more control-messages.
Have fun,
Arnold
PS: Maybe this question sound foolish: Why not AVB / open AVB?
10:52 p.m.
New subject: Open Source Audio Interface
On Fri, 12 Sep 2014, Arnold Krille wrote:
Yes. I have included midi links in real time as it happens as part of the
audio packet. network data has other uses and that was why I wanted both.
On Thu, 11 Sep 2014 14:24:56 -0700 (PDT) Len Ovens
<len(a)ovenwerks.net>
wrote:
Shouldn't be, The network traffic does not need to be real time, but the
network does. In other words the way the host computer deals with network
traffic does not need to be real time. The computer puts a packet into a
buffer, this part does not need to be real time. The network sends it
while there is no audio traffic, this part does need to be real time.
Where I see your info as being useful though, is
if I choose to send
data (which could be varying sized packets) I may still wish to set
the data packet size to a fixed size depending on what is left from
audio. The whole network end of things needs to be decoupled from
real time.
That last sentence is a bit contradicting. You want to send audio over
network, you want the audio to be near realtime. But you want to
decouple network from realtime... (In math: A==B && B==C && A!=C) To be honest,
I have spent a lot less time thinking about the network
traffic part of things than the audio part of things. The reason for
this is that the use cases for allowing network through our line is
small. As has already been commented, a desktop can have a second NIC
and the laptop generally has wireless. The main usecase for network
through the same IF might be radio work where one might have a SIP
connection (phone line) going through net. (anyone see others?)
Because the AI has a second NIC and an OS, it should be possible to
set up the SIP session on that box.
There is a widespread use-case for audio and non-audio network traffic
on the same segment: audio and control/midi messages on the same
segment. And you don't want your fader moves to interrupt your audio. And your current thinking seems to be about using a
dedicated
network-link between PC and audio-interface with a possible chain-link
to other interfaces. But what about a dedicated network (with switch)
to connect multiple PCs and audio-interfaces? Then you have to deal
with lots more channels, thus lots more realtime data and also lots
more control-messages.
Why? there is already stuff out there that does that (with higher latency
of course) The main idea is still to replace a FW audio interface up to
around 32 channels, not provide a generic network audio protocol.
PS: Maybe this question sound foolish: Why not AVB /
open AVB?
What is the latency? Or what is the minimum latency I could get on a link
with say 50% audio/network traffic and a 100m link? The real show stopper
though is that the end points require dedicated HW. That is it would not
just work with the average laptop or desktop NIC. AT least with the
desktop a new NIC could be added. The question as to the NIC driver
actually using the required capability is something else.
--
Len Ovens
www.ovenwerks.net
11:56 p.m.
New subject: Open Source Audio Interface
On Thu, Sep 11, 2014 at 6:50 PM, Len Ovens <len(a)ovenwerks.net> wrote:
Shouldn't be, The network traffic does not need to
be real time, but the
network does. In other words the way the host computer deals with network
traffic does not need to be real time. The computer puts a packet into a
buffer, this part does not need to be real time.
this is not correct. there's no difference between this operation and
putting audio into the h/w buffer associated with a PCI audio interface.
the entire transport chain must be RT.
The network sends it while there is no audio traffic, this part does need
to be real time.
also wrong. it must be able to send it on demand, or at least within a
given (realtime) window.
but this then gets to the clocking part ... not word clock per se (although
that is related) but the question of how the entire transport mechanism is
driven.
12 Sep
12 Sep
3:18 a.m.
New subject: Open Source Audio Interface
On Thu, 11 Sep 2014, Paul Davis wrote:
On Thu, Sep 11, 2014 at 6:50 PM, Len Ovens <len(a)ovenwerks.net> wrote:
Shouldn't be, The network traffic does not need to be real time,
but the network does. In other words the way the host computer
deals with network traffic does not need to be real time. The
computer puts a packet into a buffer, this part does not need to
be real time.
this is not correct. there's no difference between this operation and
putting audio into the h/w buffer associated with a PCI audio interface. the
entire transport chain must be RT.
Ok, first off network traffic does not equal audio traffic. Perhaps I
should say ethernet traffic and IP traffic. The audio is sent as raw
ethernet packets. This call must be realtime. The whole audio chain is
real time. The network or IP traffic is fed to our driver through normal
channels. Our driver looks to the system like an eth* device, but in fact
it is not a device, but a socket that puts a packet into a buffer. That
part does not have to be real time... but once it is in the buffer the
feeding of that data as a raw ethernet packet does have to be real time
because this operation must be completed in time for the next packet of
audio data because, whatever the interval the audio is sent at, the audio
data has to be sent in time with that.
So in the case I was supposing with 4 samples of data per audio packet,
as soon as the 4th sample has been collected by the ADC, An audio packet
is sent with all 4 samples for all channels. Then the time left till next
4th sample is calculated/read and the size of data allowed to fit is sent
(some leeway is left). The ethernet interface should be left open by the
time the next audio frame is ready. The audio driver may have to test the
actual time required from packet send to receive to know how big to allow
the IP data packet to be. The guard band time is supposed to be a certain
length (the MACs plus type plus guard band _should_ be 304 bits long). If
the IP data in the buffer is not completely written by the time the audio
packet has been sent, it waits for the next audio packet to go first. The
network (IP) traffic will be slowed down more than would be expected by
just looking at percentages, but the audio should be able to be sent on
time every time. There is no other traffic but what goes through the audio
driver.
In a home situation where even with wide band internet the max speed is 6
to 15 mbits The network speed from the computer to the gateway does not
need to be a full 100mbit. In a more critical situation, a second NIC
would be better.
Len is not very good at explaining things...
--
Len Ovens
www.ovenwerks.net
11 Sep
11 Sep
9:44 a.m.
New subject: [LAU] Open Source Audio Interface
Len Ovens:
I thought I would move this over here. I know there is
already work being
done on this hw wise. These thoughts are for a point to point raw
ethernet audio transport That still allows some normal network traffic as
well.
On Tue, 2 Sep 2014, Len Ovens wrote:
...
Why don't you prototype it with netcat:
Terminal 1:
$ nc -u -l -p 4000 localhost
hej
asdf
^C
$
Terminal 2:
$ nc -u localhost 4000
hej
asdf
^C
$
Instead of two terminals, use two computers and write a sender and
a receiver program, and start debugging.
Regards,
/Karl Hammar
-----------------------------------------------------------------------
Aspö Data
Lilla Aspö 148
S-742 94 Östhammar
Sweden
+46 173 140 57
My thought is something like this:
We control all network traffic. Lets try for 4 words of audio. For sync
purposes, at each word boundry a short audio packet is sent of 10 channels.
This would be close to minimum enet packet size. Then there should be room
for one full size enet packet, in fact even at 100m the small sync packet
could contain more than 10 channels (I have basically said 10m would not be
supported, but if no network traffic was supported then 10m could do 3 or 4
channels with no word sync). So:
Word 1 - audio sync plus 10 tracks - one full network traffic packet
word 2 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 1 and 2
wors 3 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 2 and 3
word 4 - audio sync plus 10 tracks - one full audio packet 40 tracks
split between word 3 and 4
Nobody commented that this could not work :) 4 samples on a 100mbit link
is still less than one full 1500byte data packet. The reason I am thinking
about this right now, is that my studio has been flooded :P and so I
have no access to work on my control surface project right now.
11:15 a.m.
New subject: [LAU] Open Source Audio Interface
On Thu, Sep 11, 2014 at 5:44 AM, <karl(a)aspodata.se> wrote:
Why don't you prototype it with netcat:
Terminal 1:
$ nc -u -l -p 4000 localhost
hej
asdf
^C
$
Terminal 2:
$ nc -u localhost 4000
hej
asdf
^C
$
Instead of two terminals, use two computers and write a sender and
a receiver program, and start debugging.
as powerful as some unix prototyping tools can be, just stop for a second.
although i'd be the first person to suspect stupid financial motivations
behind the proliferation of audio-over-(ethernet|IP) solutions, do you
really think that if delivery low latency audio streams was even remotely
achievable with a tool like netcat that we would have seen the emergence of
special network interfaces and completely new ethernet-level protocols?
audio-via-cat5 cable isn't a new problem, waiting for the right solution.
it is an old problem with at least a half dozen solutions each with
different pros and cons.
1:58 p.m.
New subject: Open Source Audio Interface
On Thu, 11 Sep 2014, Paul Davis wrote:
On Thu, Sep 11, 2014 at 5:44 AM,
<karl(a)aspodata.se> wrote:
Why don't you prototype it with netcat:
as powerful as some unix prototyping tools can be, just stop for a second.
although i'd be the first person to suspect stupid financial motivations
behind the proliferation of audio-over-(ethernet|IP) solutions, do you
really think that if delivery low latency audio streams was even remotely
achievable with a tool like netcat that we would have seen the emergence of
special network interfaces and completely new ethernet-level protocols?
audio-via-cat5 cable isn't a new problem, waiting for the right solution. it
is an old problem with at least a half dozen solutions each with different
pros and cons.
Not new at all. There are a lot of solutions. Many of them closed. AES3
and AES10 are open standards. AES3 suffers from low channel count and
aes10 suffers from expecive HW... and the need for an interface a laptop
may not have. Laptops have become a desktop replacement for many people.
(in a few years a smartphone may become a desktop replacement for some
people too)
I think netcat is not needed for this project, It is still higher level
than what would be the best use of ethernet. Raw ethernet at level2 (the
MAC level) is the lowest standard level. But it is easy to use for
prototyping. The main drawback for using a higher level (ease of use being
the main plus), is that it assumes collisions and other delays and that it
will work on a LAN where it will have to deal with other traffic. So it is
easy, yes, but making it reliable is hard. It can be very reliable in a
point to point configuration, but then the sw config tends to be less
automated and not so easy for the end user. The other problem with high
level network is the use of CPU and bandwidth for overhead. Netjack works
well for me, I can set it up to be very good, but it does use much more
CPU than a PCI or USB audio IF would use. I don't think there should be
any reason for that in a point to point setup.
To rephrase, it will work on any LAN including one hooked up to the wire.
Therefore it will get used that way and has to be able to deal with going
through a switch that is dealing with downloading a movie in the living
room and feeding a wireless AP. There will be complaints about it not
working in those situations. Making it strickly P2P, level2 avoids all of
that.
Latency is additive. This net transport is only a small part of the path
and making it much lower than what is needed means that all the other
things that add latency are still ok. So when I talk about link latency, I
am talking about raw ethernet send to raw ethernet receive only at this
point.
--
Len Ovens
www.ovenwerks.net
3606
days inactive
3609
days old
linux-audio-dev@lists.linuxaudio.org
18 comments
7 participants
participants (7)
-
Arnold Krille -
Fons Adriaensen -
gordonjcp＠gjcp.net -
karl＠aspodata.se -
Len Ovens -
Paul Davis -
Robert Jonsson