10 Mar
2009
10 Mar
'09
8:52 p.m.
Hello all,
Today I had a strange problem when using the
WFS system here.
The 'master' PC sends a multicast message every
1024 samples (21.333 ms) to all 'render' PCs.
This has to arrive on time, and when it's late
the renderers will mute their output and report
the error in their status messages.
Today I used some ssh -X logins from my laptop
in the WFS room to the WFS master to run Ardour
and some other apps for a demo. All this worked
well all the time, as it has done before.
I left everything running when going for lunch
with our visitors, and when I returned restarted
Ardour to listen again. I got a lots 'of 'message
too late' errors from the rendering machines, and
interrupted sound. Strange enough this seemed to
be related to the _volume_ of the sounds...
In other words to Ardour's level meters.
Restarting Ardour in a new ssh login did not
help, but running it directly on the WFS master
solved the problem. So apparently the network
traffic required to update Ardour's meters was
causing the delays. And clearly the whole remote
X session was slower than normal. CPU loads
looked normal.
Now all this should be peanuts for a Gbit
network that has no other traffic at all,
and it worked perfectly before. I just never
left it running for such a long time.
Anyone an idea as to what is happening here,
and how it could be cured ?
TIA,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Be quiet, Master Land; and you, Professor,
will you be so good as to listen to me ?
11 Mar
11 Mar
7:35 a.m.
Den Wednesday 11 March 2009 00.40.44 skrev Fons Adriaensen:
Hello all,
Today I had a strange problem when using the
WFS system here.
The 'master' PC sends a multicast message every
1024 samples (21.333 ms) to all 'render' PCs.
This has to arrive on time, and when it's late
the renderers will mute their output and report
the error in their status messages.
Today I used some ssh -X logins from my laptop
in the WFS room to the WFS master to run Ardour
and some other apps for a demo. All this worked
well all the time, as it has done before.
I left everything running when going for lunch
with our visitors, and when I returned restarted
Ardour to listen again. I got a lots 'of 'message
too late' errors from the rendering machines, and
interrupted sound. Strange enough this seemed to
be related to the _volume_ of the sounds...
In other words to Ardour's level meters.
Restarting Ardour in a new ssh login did not
help, but running it directly on the WFS master
solved the problem. So apparently the network
traffic required to update Ardour's meters was
causing the delays. And clearly the whole remote
X session was slower than normal. CPU loads
looked normal.
Now all this should be peanuts for a Gbit
network that has no other traffic at all,
and it worked perfectly before. I just never
left it running for such a long time.
Anyone an idea as to what is happening here,
and how it could be cured ?
I can't say that I can help you but 19 years as a network engineer can come in
handy.
The most obvious is that the network is overloaded in some way. Doesn't need
to be in terms of bandwidth. It can be things like spanning-tree going
heywire in switches. Redirects is another thing that comes to my mind. A
sophisticated (there are things like slow pings of IPv6 addresses that fills
up the table in a switch) DoS might also be the case.
If we assume it's the network that causes the problem you obviously have the
switches and the routers that can cause the problems not to say firewalls.
What network set up (equipment and such) do you have?
Is the network protected in some way?
Have you made some performance tests? iperf is handy.
TCP:
server:> iperf -s
client:> iperf -c server
UDP:
server:> iperf -su
client:> iperf -c server -u
/bengan
5:52 p.m.
On Wed, Mar 11, 2009 at 11:35:15AM +0100, Bengt Gördén wrote:
I'll make a complete list of all the equipment
next monday (I don't return there before).
The network is completely isolated, there are just
the four WFS computers (and a fifth which was not
being used), and my Thinkpad R51.
Apart from the ssh -X connection the only traffic
are the messages I mentioned before (around 50/s)
and the status reports from the slave computers,
around 60/s for all three added. All messages are
less than an MTU.
Anyone an idea
as to what is happening here,
and how it could be cured ?
I can't say that I can help you but 19 years as a network engineer can come in
handy.
The most obvious is that the network is overloaded in some way. Doesn't need
to be in terms of bandwidth. It can be things like spanning-tree going
heywire in switches. Redirects is another thing that comes to my mind. A
sophisticated (there are things like slow pings of IPv6 addresses that fills
up the table in a switch) DoS might also be the case.
If we assume it's the network that causes the problem you obviously have the
switches and the routers that can cause the problems not to say firewalls.
What network set up (equipment and such) do you have?
Is the network protected in some way? Have you made some performance tests? iperf is handy.
TCP:
server:> iperf -s
client:> iperf -c server
UDP:
server:> iperf -su
client:> iperf -c server -u
Good tip, will do.
Now you're here you may be able to comment on
another problem: the network adapter in the
master does not receive the multicast messages
it is sending (even if the TX socket has the
IP_MULTICAST_LOOP option set). Chip is Intel
85something, driver e1000e.
Many thanks,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Be quiet, Master Land; and you, Professor,
will you be so good as to listen to me ?
14 Mar
14 Mar
11:33 a.m.
On Wed, 2009-03-11 at 00:40 +0100, Fons Adriaensen wrote:
Hello all,
Today I had a strange problem when using the
WFS system here.
...
In other words to Ardour's level meters.
Restarting Ardour in a new ssh login did not
help, but running it directly on the WFS master
solved the problem. So apparently the network
traffic required to update Ardour's meters was
causing the delays. And clearly the whole remote
X session was slower than normal. CPU loads
looked normal.
It would be a good test to hide the meters in question (close the mixer
window and turn off the editor mixer and track header meters). This
would prove the metering (=network and/or X) is to blame.
Sampo
12:22 p.m.
On Sat, Mar 14, 2009 at 04:33:01PM +0200, Sampo Savolainen wrote:
It would be a good test to hide the meters in question
(close the mixer
window and turn off the editor mixer and track header meters). This
would prove the metering (=network and/or X) is to blame.
Hiding the meters did not help, probably updates
are sent anyway. Isolating Ardour so it was no
longer part of the WFS audio chain did not help
either - the delays continued to appear, clearly
correlated to the levels in the Ardour session,
and not at all to the audio signals being used
by the WFS system.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
Be quiet, Master Land; and you, Professor,
will you be so good as to listen to me ?
5524
days inactive
5528
days old
linux-audio-dev@lists.linuxaudio.org
4 comments
3 participants
participants (3)
-
Bengt Gördén -
Fons Adriaensen -
Sampo Savolainen