7 Mar
2016
7 Mar
'16
2:12 a.m.
Greetings, everyone. Since I am using 85% of JACK DSP in my primary
production box, while using 14% of the CPU and 1/8 of the RAM according
to htop, it appears that I need to develop a way to move forward :-) It
is very clear from the excellent Patrick Shirkey's input that one can
use netjack to chain any combination of motherboards each running JACK;
I would like to use the power of my box to do the same internally.
I have taken a number of hours over the last few days in testing. Thus
far, my impression is that netjack1, netjack2, and jacktrip are
explicitly not set up for this. Using jack2, netjack1 and jacktrip
worked fine for one client/slave, but a second one froze both, and
netjack2 did not work at all, I found a web reference stating that
multijack explicitly does not do localhost unless explicitly set to do
so at the TCP/IP stack programming level. Using jack1, netjack1 did not
connect at all, so I decided to halt for now and ask Those Who Know :-)
So far, the only clearly workable option, has been to try the Docker
lightweight virtualizer, and run each JACK server and related
application chain in its own container. Anyone have better ideas?
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
7 Mar
7 Mar
9:49 a.m.
On Sun, Mar 06, 2016 at 07:12:25PM -0600, Jonathan E. Brickman wrote:
Greetings, everyone. Since I am using 85% of JACK DSP
in my primary
production box, while using 14% of the CPU and 1/8 of the RAM
according to htop, it appears that I need to develop a way to move
forward :-) It is very clear from the excellent Patrick Shirkey's
input that one can use netjack to chain any combination of
motherboards each running JACK; I would like to use the power of my
box to do the same internally.
I so don't get what you're trying to achieve. Use multiple soundcards?
Use multiple CPUs? Parallelize the client graph?
Cheers
--
mail: adi(a)thur.de http://adi.thur.de PGP/GPG: key via keyserver
3:27 p.m.
On 03/07/2016 02:12 AM, Jonathan E. Brickman wrote:
Greetings, everyone. Since I am using 85% of JACK DSP
in my primary
production box, while using 14% of the CPU and 1/8 of the RAM according
to htop, it appears that I need to develop a way to move forward :-)
14% of what? One physical core? One virtual core? All cores? Chicken?
I would like to use the power of my box to do the same
internally.
Jack should be doing this already in a single instance.
I have taken a number of hours over the last few days in testing. Thus
far, my impression is that netjack1, netjack2, and jacktrip are
explicitly not set up for this. Using jack2, netjack1 and jacktrip
worked fine for one client/slave, but a second one froze both, and
netjack2 did not work at all, I found a web reference stating that
multijack explicitly does not do localhost unless explicitly set to do
so at the TCP/IP stack programming level. Using jack1, netjack1 did not
connect at all, so I decided to halt for now and ask Those Who Know :-)
So far, the only clearly workable option, has been to try the Docker
lightweight virtualizer, and run each JACK server and related
application chain in its own container. Anyone have better ideas?
Yes, maybe take a step back and try to find the problem before finding a
solution.
From what I just read:
* You noticed, that there is a high DSP load for JACK
* You noticed, that there is a low CPU load on you box, which has most
likely multiple physical cores?
So I assume you think the following:
Ok, my Hardware still has free CPU resources, and since JACK is supposed
to be a smart thing, it should be able to utilize all of my CPU cores.
One could think, that JACK just needs to put your clients on different
CPUs so they can run in _parallel_, using up more CPU Cycles and increasing.
Sadly it is not that easy, since this assumes, that your processing
chain actually _can_ be parallelized. Can it? Depends...
If your processing chain looks like a daisy chain, you are out of luck:
mic -> client1 -> client2 -> client3 -> speaker
(fig 1)
Since each client must wait for the previous one, only one client can do
its work at a time, which means only one of your cores can be utilized
_at a time_, the others will just sit there idling around.
On the other hand, if your processing chain looks like this:
mic --> client1 ---> speaker
\-> client2 -----^ ^ ^
\-> client3 ------/ /
\-> client4 -------/
(fig2)
the clients 1,2,3,4 can run at the same time. JACK _can_ schedule them
to run on 4 different CPUs, which means you can reach a higher average
CPU load while keeping the DPS load lower.
There are many more details, but the most important thing is the question:
"Can my processing chain be parallel?"
"Can I do anything to my processing chain to make it (more) parallel?"
I stop here trying to interpret what you were trying to achieve and
maybe the best thing you could do to get some tips is to describe your
processing chain, maybe something can be worked out. A picture in form
of a graph, says more than a thousand words in this case.
Best Regards
Markus
8:38 p.m.
On 03/07/2016 08:19 AM, Markus Seeber wrote:
No, it should not, as you explained! :-)
On 03/07/2016 02:12 AM, Jonathan E. Brickman wrote:
Eight physical cores.
Greetings, everyone. Since I am using 85% of
JACK DSP in my primary
production box, while using 14% of the CPU and 1/8 of the RAM according
to htop, it appears that I need to develop a way to move forward :-)
14% of what?
One physical core? One virtual core? All cores? I would like
to use the power of my box to do the same internally.
Jack should be doing this
already in a single instance. From what I just read:
* You noticed, that there is a high DSP load for JACK
* You noticed, that there is a low CPU load on you box, which has most
likely multiple physical cores?
So I assume you think the following:
Ok, my Hardware still has free CPU resources, and since JACK is supposed
to be a smart thing, it should be able to utilize all of my CPU cores.
Best to not
assume. :-)
What is happening right now, is I have seven synth+filter chains, all
run through the single JACK server, all feeding eventually into the one
sound card. I have more than ample CPU to run them all, but as you and
others have explained, one JACK server is reaching its limits to handle
them all because of the limits of the synchronous nature of everything.
So what I intend to do, is to run all of the chains independently,
asynchronously, on their own JACK servers, and then combine them all
into a separate final which will connect to the sound card. This is
being done already with as many motherboards as desired, but I would
like to do it within one very powerful box.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
8:49 p.m.
On Mon, Mar 7, 2016 at 2:37 PM, Jonathan E. Brickman <jeb(a)ponderworthy.com>
wrote:
What is happening right now, is I have seven synth+filter chains, all run
through the single JACK server, all feeding eventually into the one sound
card.
if the synths are all independent clients and they do not feed other, then
they will automatically be parallelized by JACK2.
if they are not, then there is a bug that should be fixed, rather than
finding complex workarounds.
I have more than ample CPU to run them all, but as you and others have
explained, one JACK server is reaching its limits to
handle them all
because of the limits of the synchronous nature of everything. So what I
intend to do, is to run all of the chains independently, asynchronously, on
their own JACK servers, and then combine them all into a separate final
which will connect to the sound card. T
this isn't going to work the way you want it to.
9:53 p.m.
On Mon, Mar 07, 2016 at 01:37:48PM -0600, Jonathan E. Brickman wrote:
What is happening right now, is I have seven
synth+filter chains,
all run through the single JACK server, all feeding eventually into
the one sound card. I have more than ample CPU to run them all, but
as you and others have explained, one JACK server is reaching its
limits to handle them all because of the limits of the synchronous
nature of everything.
Nobody has explained that, you just don't seem to understand,
or want to understand, what *has* been explained.
If your single system is able to do what you require, it
can do it using a single Jack server (you may need Jack2
instead of Jack1).
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
8 Mar
8 Mar
2:05 p.m.
On 03/07/2016 08:37 PM, Jonathan E. Brickman wrote:
[...]
What is happening right now, is I have seven synth+filter chains, all
run through the single JACK server, all feeding eventually into the one
sound card. I have more than ample CPU to run them all, but as you and
others have explained, one JACK server is reaching its limits to handle
them all because of the limits of the synchronous nature of everything.
So what I intend to do, is to run all of the chains independently,
asynchronously, on their own JACK servers, and then combine them all
into a separate final which will connect to the sound card. This is
being done already with as many motherboards as desired, but I would
like to do it within one very powerful box.
Since you did not give much case specific information about the topology
of your processing chain, it is hard to tell where the problem is in
your case and how to solve it. Not sure what's up about all this
asynchronous talk, maybe we talk past each other? (pun intended)
Maybe some visualisation of your jack graph could help, I think patchage
can export the structure of that into a dot/graphviz file, you could
attach that. Information about the strain each of these filters puts on
the CPU would be helpful as a hint too. That would not be the number at
the top of htop, but next to the process of each of these filters.
Except from that, Paul and Fons already made the point about how this is
supposed to work.
Best regards
Markus
2:12 p.m.
On 03/08/2016 06:57 AM, Markus Seeber wrote:
On 03/07/2016 08:37 PM, Jonathan E. Brickman wrote:
[...]
OK. I should be able to have the box off-PA in a few days, and
will
work out producing a Patchage export and a ps aux export at load.
In the meantime, we are talking about this:
http://lsn.ponderworthy.com/doku.php/concurrent_patch_management
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
What is happening right now, is I have seven
synth+filter chains, all
run through the single JACK server, all feeding eventually into the one
sound card. I have more than ample CPU to run them all, but as you and
others have explained, one JACK server is reaching its limits to handle
them all because of the limits of the synchronous nature of everything.
So what I intend to do, is to run all of the chains independently,
asynchronously, on their own JACK servers, and then combine them all
into a separate final which will connect to the sound card. This is
being done already with as many motherboards as desired, but I would
like to do it within one very powerful box.
Since you did not give much case
specific information about the topology
of your processing chain, it is hard to tell where the problem is in
your case and how to solve it. Not sure what's up about all this
asynchronous talk, maybe we talk past each other? (pun intended)
Maybe some visualisation of your jack graph could help, I think patchage
can export the structure of that into a dot/graphviz file, you could
attach that. Information about the strain each of these filters puts on
the CPU would be helpful as a hint too. That would not be the number at
the top of htop, but next to the process of each of these filters.
Except from that, Paul and Fons already made the point about how this is
supposed to work.
11 Mar
11 Mar
3:29 a.m.
>> What is happening right now, is I have seven
synth+filter chains, all
>> run through the single JACK server, all feeding eventually into the one
>> sound card. I have more than ample CPU to run them all, but as you and
>> others have explained, one JACK server is reaching its limits to handle
>> them all because of the limits of the synchronous nature of everything.
>> So what I intend to do, is to run all of the chains independently,
>> asynchronously, on their own JACK servers, and then combine them all
>> into a separate final which will connect to the sound card. This is
>> being done already with as many motherboards as desired, but I would
>> like to do it within one very powerful box.
> Maybe some visualisation of your jack graph could help, I think patchage
> can export the structure of that into a dot/graphviz file, you could
> attach that. Information about the strain each of these filters puts on
> the CPU would be helpful as a hint too. That would not be the number at
> the top of htop, but next to the process of each of these filters.
The DOT is
attached. At max load, the only CPU being stressed more than
5% is running just one of the Yoshimi processes, one taking high ranges
in patch SRO; this one CPU is kept at a steady 14% when SRO is sounding
with maximum notes. There is no very significant CPU stress, just
maxing-out of JACK DSP.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
4:13 a.m.
so, you've got 6 non-parallelizable stages. the first stage (3 instances of
yoshimi, plus stringbassacid plus stringsSSO) has 5 clients that can be in
parallel. The second stage (Mixer/*) has 6 clients that can be run in
parallel. The third stage has 3 clients (Mixer/* and 1 yoshimi) that can be
run in parallel. the 4th stage has 3 clients that be run in parallel. the
5th stage has 1 client, as does the 6th.
basically, this is a picture perfect demonstration to me of why the
process-level parallelism that JACK enables is just a bad idea.
Distributing this amount of processing across 19 JACK clients some of which
are parallelizable and some are not is, to my mind (as JACK's original
author) precisely how the program should never be used.
maybe someone will find a way for you to do what you want, but I personally
think that this whole workflow is ill-conceived. i'm sorry that JACK's
capabilities led you to this, because I think you're not well served with
this tool configuration.
On Thu, Mar 10, 2016 at 9:28 PM, Jonathan E. Brickman <jeb(a)ponderworthy.com>
wrote:
What is happening right now, is I have seven synth+filter chains, all
run through the single JACK server, all feeding eventually into the one
sound card. I have more than ample CPU to run them all, but as you and
others have explained, one JACK server is reaching its limits to handle
them all because of the limits of the synchronous nature of everything.
So what I intend to do, is to run all of the chains independently,
asynchronously, on their own JACK servers, and then combine them all
into a separate final which will connect to the sound card. This is
being done already with as many motherboards as desired, but I would
like to do it within one very powerful box.
Maybe some visualisation of your jack graph could help, I think patchage
can export the structure of that into a dot/graphviz file, you could
attach that. Information about the strain each of these filters puts on
the CPU would be helpful as a hint too. That would not be the number at
the top of htop, but next to the process of each of these filters.
The DOT is attached. At max load, the only CPU being stressed more than
5% is running just one of the Yoshimi processes, one taking high ranges in
patch SRO; this one CPU is kept at a steady 14% when SRO is sounding with
maximum notes. There is no very significant CPU stress, just maxing-out of
JACK DSP.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
8:03 a.m.
On Fri, March 11, 2016 2:13 pm, Paul Davis wrote:
so, you've got 6 non-parallelizable stages. the
first stage (3 instances
of
yoshimi, plus stringbassacid plus stringsSSO) has 5 clients that can be in
parallel. The second stage (Mixer/*) has 6 clients that can be run in
parallel. The third stage has 3 clients (Mixer/* and 1 yoshimi) that can
be
run in parallel. the 4th stage has 3 clients that be run in parallel. the
5th stage has 1 client, as does the 6th.
basically, this is a picture perfect demonstration to me of why the
process-level parallelism that JACK enables is just a bad idea.
Distributing this amount of processing across 19 JACK clients some of
which
are parallelizable and some are not is, to my mind (as JACK's original
author) precisely how the program should never be used.
maybe someone will find a way for you to do what you want, but I
personally
think that this whole workflow is ill-conceived. i'm sorry that JACK's
capabilities led you to this, because I think you're not well served with
this tool configuration.
If this was a hardware issue where one physical module had becoming a
bottleneck the obvious solution is to add additional modules to expand the
pipeline.
What is the specific problem with having multiple JACK instances running
on the same desktop. Why can't a modern parallel CPU with 8 hyperthreads
run multiple instances of JACK? It also offers some interesting potential
for session management.
If a user has a 64 core desktop it would be absurd if we cannot enable
s/he to use it to the full potential because one instance of JACK is
getting overloaded when the vast majority of processing cores are not even
being used.
If this cannot be fixed in JACK directly we should be able to spin up
multiple instances on the same machine and have them play nice with each
other.
On Thu, Mar 10, 2016 at 9:28 PM, Jonathan E. Brickman
<jeb(a)ponderworthy.com>
wrote:
--
Patrick Shirkey
Boost Hardware Ltd
What is happening right now, is I have seven synth+filter chains, all
run through the single JACK server, all feeding eventually into the one
sound card. I have more than ample CPU to run them all, but as you and
others have explained, one JACK server is reaching its limits to handle
them all because of the limits of the synchronous nature of everything.
So what I intend to do, is to run all of the chains independently,
asynchronously, on their own JACK servers, and then combine them all
into a separate final which will connect to the sound card. This is
being done already with as many motherboards as desired, but I would
like to do it within one very powerful box.
Maybe some visualisation of your jack graph could help, I think patchage
can export the structure of that into a dot/graphviz file, you could
attach that. Information about the strain each of these filters puts on
the CPU would be helpful as a hint too. That would not be the number at
the top of htop, but next to the process of each of these filters.
The DOT is attached. At max load, the only CPU being stressed more than
5% is running just one of the Yoshimi processes, one taking high ranges
in
patch SRO; this one CPU is kept at a steady 14% when SRO is sounding
with
maximum notes. There is no very significant CPU stress, just maxing-out
of
JACK DSP.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
8:58 a.m.
On 03/11/2016 08:03 AM, Patrick Shirkey wrote:
If this cannot be fixed in JACK directly we should be
able to spin up
multiple instances on the same machine and have them play nice with each
other.
and how would that be different from splitting the current graph in JACK
and not preform worse?
1:17 p.m.
On Fri, March 11, 2016 6:58 pm, Robin Gareus wrote:
On 03/11/2016 08:03 AM, Patrick Shirkey wrote:
Currently it seems that we cant do either so which method is preferable?
According to Jonathan's results he is finding a bottle neck with JACK DSP
with a single server. In the absense of a fix for JACK so that it is not a
bottle neck his solution is to run multiple servers on the same machine.
However it seems that it is not possible to have more than 2 instances of
JACK running on the same machine without using a virtual
machine/environment.
According to Paul the issue is that we should not rely on JACK to create a
processing graph like Jonathans.
I don't see much difference between a single server with multiple graphs
or multiple servers However if JACK is getting overloaded with a single
server and we can't split the graph up internally then it makes sense to
me that we enable more than two instances of JACK on the same machine
without forcing people to use docker or some other virtual environment.
Is the issue with multiple servers a priority problem or is it just that
no one has ever bothered to test more than two instances on the same
machine before?
--
Patrick Shirkey
Boost Hardware Ltd
If this cannot be fixed in JACK directly we
should be able to spin up
multiple instances on the same machine and have them play nice with each
other.
and how would that be different from splitting the current graph in JACK
and not preform worse?
1:59 p.m.
On Fri, Mar 11, 2016 at 7:17 AM, Patrick Shirkey <pshirkey(a)boosthardware.com
wrote:
On Fri, March 11, 2016 6:58 pm, Robin Gareus wrote:
Not quite what I said, but close enough.
20 context switches minimum per process() cycle. This isn't dramatic, but
it is notable. Some of them might not be context switches if the "Mixer"
stuff is actually an example of a multi-client process - I don't know.
On 03/11/2016 08:03 AM, Patrick Shirkey wrote:
Currently it seems that we cant do either so which method is preferable?
According to Jonathan's results he is finding a bottle neck with JACK DSP
with a single server. In the absense of a fix for JACK so that it is not a
bottle neck his solution is to run multiple servers on the same machine.
However it seems that it is not possible to have more than 2 instances of
JACK running on the same machine without using a virtual
machine/environment.
According to Paul the issue is that we should not rely on JACK to create a
processing graph like Jonathans.
If this cannot be fixed in JACK directly we
should be able to spin up
multiple instances on the same machine and have them play nice with each
other.
and how would that be different from splitting the current graph in JACK
and not preform worse?
I don't see much difference between a single server with multiple graphs
or multiple servers
That isn't the choice. The choice is threeway:
- reduce overhead caused by context switching between programs
- reduce DSP load by running more in parallel (this is dependent on the
graph; JACK2 will do
the best possible already, so if Jonathan is already using JACK2,
maximum parallelism
is already in use)
- reduce the amount of work done by each client.
2:24 p.m.
On Fri, March 11, 2016 11:59 pm, Paul Davis wrote:
On Fri, Mar 11, 2016 at 7:17 AM, Patrick Shirkey
<pshirkey(a)boosthardware.com
Are we absolutely sure this is the case? That Jonathan has not found a
"bug" in JACK2 or the DSP load algorithm?
wrote:
On Fri, March 11, 2016 6:58 pm, Robin Gareus wrote:
JACK
Not quite what I said, but close enough.
20 context switches minimum per process() cycle. This isn't dramatic, but
it is notable. Some of them might not be context switches if the "Mixer"
stuff is actually an example of a multi-client process - I don't know.
On 03/11/2016 08:03 AM, Patrick Shirkey wrote:
> If this cannot be fixed in JACK directly we should be able to spin up
> multiple instances on the same machine and have them play nice with
each
other.
and how would that be different from splitting the current graph in and not preform worse?
Currently it seems that we cant do either so which method is preferable?
According to Jonathan's results he is finding a bottle neck with JACK
DSP
with a single server. In the absense of a fix for JACK so that it is not
a
bottle neck his solution is to run multiple servers on the same machine.
However it seems that it is not possible to have more than 2 instances
of
JACK running on the same machine without using a virtual
machine/environment.
According to Paul the issue is that we should not rely on JACK to create
a
processing graph like Jonathans.
I don't see much difference between a single server with multiple graphs
or multiple servers
That isn't the choice. The choice is threeway:
- reduce overhead caused by context switching between programs
- reduce DSP load by running more in parallel (this is dependent on the
graph; JACK2 will do
the best possible already, so if Jonathan is already using JACK2,
maximum parallelism
is already in use) - reduce the amount of work done by each client.
According to Jonathan his multiple cores are barely reaching 5% usage. How
can JACK_DSP be so high when there is so much room left to play with if
JACK2 is handling the parallelism correctly?
It seems similar to my car telling me that I am red lighting when I am
only going 20km/hr in second.
--
Patrick Shirkey
Boost Hardware Ltd
2:34 p.m.
On Fri, Mar 11, 2016 at 8:24 AM, Patrick Shirkey <pshirkey(a)boosthardware.com
He doesn't have *that* much parallelism. He has 6 non-parallelizable
stages, each one with varying levels of parallelism.
wrote:
Are we absolutely sure this is the case? That Jonathan has not found a
"bug" in JACK2 or the DSP load algorithm?
the dataflow algorithm doesn't have a lot of room for bugs. but sure, yes,
it is possible. however, this is easy to determine with a debug build
(maybe even just a --verbose run) of JACK.
- reduce the amount of work done by each
client.
According to Jonathan his multiple cores are barely reaching 5% usage. How
can JACK_DSP be so high when there is so much room left to play with if
JACK2 is handling the parallelism correctly?
8:13 p.m.
On Fri, Mar 11, 2016 at 08:34:30AM -0500, Paul Davis wrote:
On Fri, Mar 11, 2016 at 8:24 AM, Patrick Shirkey
<pshirkey(a)boosthardware.com> wrote:
IIRC Jack's DSP load measurement indicates peak load over the
recent history. A client that generates a very uneven load (e.g.
doing big FFTs every Nth cycle) can easily make it indicate
much more than the average CPU load. Of course clients doing
that are just badly implemented, but they do exist.
The complexity of the graph doesn't look like an issue to
me, I'm routinely running such things without any problem.
Ciao,
--
FA
A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)
Are we absolutely sure this is the case? That
Jonathan has not found a
"bug" in JACK2 or the DSP load algorithm?
the dataflow algorithm doesn't have a lot of room for bugs. but sure, yes,
it is possible. however, this is easy to determine with a debug build
(maybe even just a --verbose run) of JACK. 
9:17 p.m.
On Fri, Mar 11, 2016, at 07:13 PM, Fons Adriaensen wrote:
A client that generates a very uneven load (e.g.
doing big FFTs every Nth cycle) can easily make it indicate
much more than the average CPU load. Of course clients doing
that are just badly implemented, but they do exist.
Badly implemented? For a convolver perhaps, but I think (as I would)
that it's perfectly fair to implement some methods that way. A phase
vocoder at a short period size will exhibit exactly that behaviour.
Chris
2:41 p.m.
On 03/11/2016 02:24 PM, Patrick Shirkey wrote:
According to Jonathan his multiple cores are barely
reaching 5% usage. How
can JACK_DSP be so high when there is so much room left to play with if
JACK2 is handling the parallelism correctly?
It seems similar to my car telling me that I am red lighting when I am
only going 20km/hr in second.
The proper analogy here would be a Formula-1 car in Downtown Manhattan
during rush hour :)
Cheers!
robin
3:10 p.m.
On 03/11/2016 02:24 PM, Patrick Shirkey wrote:
Or perhaps Jack2 already 'are' the bike couriers in Downtown Manhattan
during rush hour. Jet pack couriers or a trail bike couriers would be
faster than the bike courier, but also more prone to accidents. Perhaps
the OP is asking for bike couriers, who when ever it is possible
temporarily use a jet pack. OTOH carrying a jet pack on the back, would
slow down riding the bike.
According to Jonathan his multiple cores are
barely reaching 5%
usage. How can JACK_DSP be so high when there is so much room left
to play with if JACK2 is handling the parallelism correctly?
It seems similar to my car telling me that I am red lighting when I
am only going 20km/hr in second.
The proper analogy here would be a Formula-1 car in Downtown Manhattan
during rush hour :)
4:48 p.m.
On 3/11/2016 7:41 AM, Robin Gareus wrote:
On 03/11/2016 02:24 PM, Patrick Shirkey wrote:
Indeed -- except that cars in Manhattan are restricted to using wheels
:-) I have rocket engines which don't give off exhaust at all, lots and
lots of fuel, no skyscrapers in the way, and no one else in the air; I
am going to either learn or help build a way to use those engines :-)
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
According to Jonathan his multiple cores are
barely reaching 5% usage. How
can JACK_DSP be so high when there is so much room left to play with if
JACK2 is handling the parallelism correctly?
It seems similar to my car telling me that I am red lighting when I am
only going 20km/hr in second.
The proper analogy here would be a Formula-1 car in
Downtown Manhattan
during rush hour :)
Cheers!
robin
4:57 p.m.
On Fri, Mar 11, 2016 at 10:48 AM, Jonathan Brickman <jeb(a)ponderworthy.com>
wrote:
Indeed -- except that cars in Manhattan are restricted to using wheels
:-) I have rocket engines which don't give off exhaust at all, lots and
lots of fuel, no skyscrapers in the way, and no one else in the air; I am
going to either learn or help build a way to use those engines :-)
although it isn't proven yet .. i think that your problem may come from the
fact that you want to have 19 different engines, and you keep flicking
switches to go from one to the other.
i'm not even sure that we've confirmed that you are using jack2 yet. are
you?
5 p.m.
On 3/11/2016 9:57 AM, Paul Davis wrote:
On Fri, Mar 11, 2016 at 10:48 AM, Jonathan Brickman
<jeb(a)ponderworthy.com <mailto:jeb@ponderworthy.com>> wrote:
Indeed -- except that cars in Manhattan are restricted to using
wheels :-) I have rocket engines which don't give off exhaust at
all, lots and lots of fuel, no skyscrapers in the way, and no one
else in the air; I am going to either learn or help build a way to
use those engines :-)
although it isn't proven yet .. i think that your problem may come
from the fact that you want to have 19 different engines, and you keep
flicking switches to go from one to the other.
Nope, I don't want to switch
engines. Everything runs at once, and runs
very well by the way. I just want to take more advantage of what I
have, by running some things asynchronously, exactly the way some are
already doing using multiple motherboards.
i'm not even sure that we've confirmed that you are using jack2 yet.
are you?
Yep. I try JACK1 two or three times a year, briefly, but JACK1 can't
come close to what JACK2 is giving me now.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
5:17 p.m.
On Fri, Mar 11, 2016 at 11:00 AM, Jonathan Brickman <jeb(a)ponderworthy.com>
wrote:
On 3/11/2016 9:57 AM, Paul Davis wrote:
On Fri, Mar 11, 2016 at 10:48 AM, Jonathan Brickman <
<jeb@ponderworthy.com>jeb@ponderworthy.com> wrote:
the "engines" i'm referring to are your many multiple clients (19 or so).
i think you're still confused by terminology here, in a way that doesn't
advance your situation. Nothing in a JACK system runs "asynchronously".
Everything is "synchronous", which means that within a given process cycle,
all clients will be processing audio samples taken from, or to be written
to, the same locations in the hardware buffer of the audio interface in
use. They work on the same "time slice". Changing this would break the
fundamental assumptions and design of JACK (all versions).
what you are talking about is parallel vs. serialized execution of clients
(which some might term "sequential execution"). parallel execution means
that clients can be distributed across cores or cpus or whatever;
serialized execution means that only 1 client can execute at a time, so
unless that client has its own internal processor-level parallelism (e.g.
ardour), it will make no difference how many cores/cpus/mobos you have -
only 1 of them will be used at a time.
your workflow has some parallelizable elements and some elements that must
be run serially/sequentially. it is an odd design, significantly outside
the intended scope of what JACK was intended to be used for. some people
think it can be made to work for a flow like this.
Indeed -- except that cars in Manhattan are
restricted to using wheels
:-) I have rocket engines which don't give off exhaust at all, lots and
lots of fuel, no skyscrapers in the way, and no one else in the air; I am
going to either learn or help build a way to use those engines :-)
although it isn't proven yet .. i think that your problem may come from
the fact that you want to have 19 different engines, and you keep flicking
switches to go from one to the other.
Nope, I don't want to switch engines. Everything runs at once, and runs
very well by the way. I just want to take more advantage of what I have,
by running some things asynchronously, exactly the way some are already
doing using multiple motherboards.
i'm not even sure that we've confirmed that you are using jack2 yet. are
you?
Yep. I try JACK1 two or three times a year, briefly, but JACK1 can't come
close to what JACK2 is giving me now.
as long as you have parallelizable data flows, that's true. JACK1 doesn't
pay any attention to parallelization: all clients are run sequentially.
5:48 p.m.
although it isn't proven yet .. i think that your problem may
come from the fact that you want to have 19 different engines,
and you keep flicking switches to go from one to the other.
Nope, I
don't want to switch engines. Everything runs at once, and
runs very well by the way. I just want to take more advantage of
what I have, by running some things asynchronously, exactly the
way some are already doing using multiple motherboards.
the "engines" i'm referring to are your many multiple clients (19 or so).
i think you're still confused by terminology here,
in a way that
doesn't advance your situation. Nothing in a JACK system runs
"asynchronously". Everything is "synchronous", which means that
within
a given process cycle, all clients will be processing audio samples
taken from, or to be written to, the same locations in the hardware
buffer of the audio interface in use. They work on the same "time
slice". Changing this would break the fundamental assumptions and
design of JACK (all versions).
No, I know very well that nothing in a single JACK
system runs
asynchronously. The point is that if a single JACK system cannot be
flexible enough to use most of the computing power I have, because of
the limitations of any synchronous design, multiple JACK systems will be
employed within the one box, just as others already employ multiple JACK
systems on multiple motherboards to serve the same purpose. I am hoping
to avoid having to run each JACK system in its own Docker container, and
at least in theory, it should be possible to do this using netjack1,
netjack2, or jacktrip, but it appears that either localhost may not have
been tested very much as a network for these, or there may be another
limitation somewhere of which I'm not aware which prevents that from
working.
what you are talking about is parallel vs. serialized
execution of
clients (which some might term "sequential execution"). parallel
execution means that clients can be distributed across cores or cpus
or whatever; serialized execution means that only 1 client can execute
at a time, so unless that client has its own internal processor-level
parallelism (e.g. ardour), it will make no difference how many
cores/cpus/mobos you have - only 1 of them will be used at a time.
No, I am not. I
am talking about expanding the paradigm at large, to
use available resources, however it can be done.
your workflow has some parallelizable elements and
some elements that
must be run serially/sequentially. it is an odd design, significantly
outside the intended scope of what JACK was intended to be used for.
some people think it can be made to work for a flow like this.
In point of fact it
works very nicely right now, as far as it can. I
have to admit that I don't care how JACK was intended to be used; I care
merely what it can do. Certainly the tools which the Wright brothers
used in 1903 were never designed to build airplanes :-)
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
6:23 p.m.
On Fri, 11 Mar 2016, Jonathan Brickman wrote:
No, I know very well that nothing in a single JACK
system runs asynchronously.
The point is that if a single JACK system cannot be flexible enough to use most
of the computing power I have, because of the limitations of any synchronous
design, multiple JACK systems will be employed within the one box, just as others
already employ multiple JACK systems on multiple motherboards to serve the same
purpose. I am hoping to avoid having to run each JACK system in its own Docker
container, and at least in theory, it should be possible to do this using
netjack1, netjack2, or jacktrip, but it appears that either localhost may not
have been tested very much as a network for these, or there may be another
limitation somewhere of which I'm not aware which prevents that from working.
Using network to transfer final audio sounds "OK"ish. Using a net backend
would allow syncing media clock which would be the main problem where only
one of your jack servers has a "real" audio device. However, these net
backends do add latency. That is they tend to skip a buffer or defer their
use of the incoming audio data. You should be able to do this already
within an application.
Assuming you are using the same set of outputs for all of your chains,you
must be using some sort of mixer. I think I recall nonmixer. That
application may be forcing sync opperation on all your other apps/plugins.
(Your URL in your sig does not point to a web page that explains your
setup) It may be that the mixer/plugin host you are using does not lend
itself to async operation.
In point of fact it works very nicely right now, as
far as it can. I have to
admit that I don't care how JACK was intended to be used; I care merely what it
can do. Certainly the tools which the Wright brothers used in 1903 were never
designed to build airplanes :-)
The Wright brothers did create their own tools as needed though. You may
need to do the same. The Wrights redesigned the airfoil and the tools to
test them, same with the propeller and the engine for that matter. It is
interesting to note that they already had a history of making their own
tools to build bicycles. So your assertion is not right, the tools the
wright bothers used were in fact made exactly for the creation of
aircraft.
--
Len Ovens
www.ovenwerks.net
6:26 p.m.
On Fri, Mar 11, 2016 at 12:23 PM, Len Ovens <len(a)ovenwerks.net> wrote:
Assuming you are using the same set of outputs for all
of your chains,you
must be using some sort of mixer. I think I recall nonmixer. That
application may be forcing sync opperation on all your other apps/plugins.
(Your URL in your sig does not point to a web page that explains your
setup) It may be that the mixer/plugin host you are using does not lend
itself to async operation.
please, no more "async". JACK is not asynchronous.
he is using nonmixer which i believe creates multiple JACK clients in order
to exploit possible parallelism in the JACK server rather than implementing
itself. that is why his .dot graph shows multiple "mixer" clients. they are
likely all inside one process, but created independently to process non's
own view of its internal graph.
10:17 p.m.
On Fri, 11 Mar 2016, Jonathan Brickman wrote:
Nope, I don't want to switch engines. Everything
runs at once, and
runs very well by the way. I just want to take more advantage of
what I have, by running some things asynchronously, exactly the way
some are already doing using multiple motherboards.
Ok, I am not seeing any sign of this at all. I am obviously not echoing
your setup exactly... in fact my setup should be more prone forcing
everything to one core. I have 4 cores. I started just adding synths...
examples include yoshimi because you mention it, setbfree, calf fluid,
hexter, synthv1. I have them all set up in separate chains either inside a
carla box or just jack strings. They feed into one nonmixer, some of the
channels have aux to reverb. All of the synths are fed from the same midi
input.
Jack dsp goes up to about 10.5% max, The four cores all bounce around from
about 7% to 15%. The important word here is all. There is not one that is
higher than the rest.
Now true, I am running only 6 synths, but I would expect to start to see
some indication of uneven load by now if there was a problem. Three times
more synths does not look like a problem. Is there one particular
application you have that just takes a big chunk of dsp/cpu?
(Yoshimi does not like changes in buffer size BTW)
(nonmixer does not have solo/pfl/afl or mute groups, so listening to just
one channel is more work than I like)
The Carla boxes seem to have been the biggest CPU users here. (not
surprising really and may reflect the plugins more than the host anyway)
I notice you use velocity to adjust levels. In the case of some synths
that may not make a lot of difference, but many of them have a timbre
change with velocity. I suspect the mutes in nonmixer could be controlled
by midi/osc which would allow using a change of output level for even
timbre. On the other hand maybe a synth alone should be softer so when
mixed with another it is still within range. This would be closer to the
natural (acoustic) mix. No worries though, it is all artistic preferences
after all. Maybe in an acoustic situation a player would hold back when
playing with others and not when soloing...
--
Len Ovens
www.ovenwerks.net
5:53 p.m.
the "engines" i'm referring to are your
many multiple clients (19 or so).
OK, I think I see what you are referring to: the
switching nature of the
client list, where the JACK server has to switch between. And this is
entirely why it helps to run multiple JACK servers on multiple
motherboards, and why it will help to run multiple JACK servers on my
box, because the client list reduces tremendously per JACK server.
--
Jonathan E. Brickman jeb(a)ponderworthy.com (785)233-9977
Hear us at http://ponderworthy.com -- CDs and MP3 now available!
<http://ponderworthy.com/ad-astra/ad-astra.html>
Music of compassion; fire, and life!!!
6:21 p.m.
On Fri, Mar 11, 2016 at 11:53 AM, Jonathan Brickman <jeb(a)ponderworthy.com>
wrote:
the "engines" i'm referring to are your many multiple clients (19 or so).
OK, I think I see what you are referring to: the switching nature of the
client list, where the JACK server has to switch between. And this is
entirely why it helps to run multiple JACK servers on multiple
motherboards, and why it will help to run multiple JACK servers on my box,
because the client list reduces tremendously per JACK server
No, it will not. The entire graph is driven by a single time/clock source
(your audio interface).
From the moment that clock ticks (generally via an
interrupt to the CPU)
until it ticks again, the entire JACK graph (regardless of how
clients
and/or network-connected other JACK servers are involved) MUST complete
execution.
If you use sequential execution, then everything will happen on 1 core. If
the dataflow is parallel (which your case does include, partially) then
this may waste computational resources that could be used to drop the DSP
load.
So you can parallelize it to the extent possible, which JACK2 already does.
In your case, there are times when it could be using up to
But you get NOTHING from using additional JACK servers except for more time
spent on context switches and communication overhead. The clients are
already being parallelized as much as they can be.
You are not maximising the CPU resources you have available because you
don't have a fully parallelizable data flow. There's no way that using
multiple computers, multiple JACK servers or anything can get around the
fact that clients D, E and F cannot be executed until clients A, B and C
are done. This happens multiple times in your data flow.
12 Mar
12 Mar
1:30 a.m.
On 03/11/2016 05:53 PM, Jonathan Brickman wrote:
OK, I
think I see what you are referring to: the switching nature of the
client list, where the JACK server has to switch between. And this is
entirely why it helps to run multiple JACK servers on multiple
motherboards, and why it will help to run multiple JACK servers on my
box, because the client list reduces tremendously per JACK server.
I'm curious how did you reach that conclusion?
You either need to synchronize the multiple jacks or run them all
independently each jack instance with its own soundcard. In either case
the CPU will need to do at least as many context switches as running the
whole thing in a single graph.
In any case, the I don't believe the context switch overhead with only
19 clients is much of an issue here.
First thing I'd check here is the scheduler:
cyclictest -m -S -d0 -p80
is a good start. Leave it running for a while, the "Max:" at the end is
relevant, those are usec. Ideally it'll not exceed 50 or so long term.
If you want to dig deeper, compile jack2 with --profile it can print
nice diagrams like
http://www.grame.fr/ressources/publications/Timing.pdf including
wake-up times for each of the clients; then again this is fairly
advanced and the issue in your case is likely simpler.
I didn't follow recent yoshimi development, but last I looked it was
neither rt-safe nor very efficient. Some mutex/locks in the process
callback could well explain why your CPUs idles a lot and jack does not
work properly. Did you try recent zynaddsubfx?
2c,
robin
13 Mar
13 Mar
5:37 p.m.
On 03/12/2016 01:30 AM, Robin Gareus wrote:
If you want to dig deeper, compile jack2 with --profile it can print
nice diagrams like
http://www.grame.fr/ressources/publications/Timing.pdf including
wake-up times for each of the clients; then again this is fairly
advanced and the issue in your case is likely simpler.
Obtaining timig information the clients would have been the next step to
identify then critical path. I think something can be learned from this
data.
11 Mar
11 Mar
6:51 p.m.
Indeed -- except that cars in Manhattan are restricted
to using wheels
:-) I have rocket engines which don't give off exhaust at all, lots
and lots of fuel, no skyscrapers in the way, and no one else in the
air; I am going to either learn or help build a way to use those
engines :-)
The analogy shouldn't be that the computer is a Formular-1 car or
a rocket, the analogy should be that jack is a courier and the streets
are the software and the computer. I still suspect that jack2 is (are)
several bike couriers, wich nowadays seem to be the best solution in
big cities, resp. nowadays soft- and hardware. However, in the future
couriers might prefer "Back to the Future" hoverboards. A hoverboard
ollie (skateboarding trick where the rider and board leap into the air
without the use of the rider's hands) would allow to exploit additional
transport infrastructure. IOW the next generation hardware might ask
for jack3.
2:33 p.m.
On 03/11/2016 01:17 PM, Patrick Shirkey wrote:
According to Jonathan's results he is finding a
bottle neck with JACK DSP
with a single server.
He reports that the bottleneck is his complete setup, I saw no evidence
that JACK is the bottleneck nor that the reason is the "single-server".
Maybe I missed some analysis related to that?
In the vast majority of cases raw CPU power is the least to worry about
in low-latency audio realtime graphs.
ciao,
robin
3230
days inactive
3236
days old
linux-audio-dev@lists.linuxaudio.org
33 comments
11 participants
participants (11)
-
Adrian Knoth -
Chris Cannam -
Fons Adriaensen -
Jonathan Brickman -
Jonathan E. Brickman
-
Len Ovens -
Markus Seeber -
Patrick Shirkey -
Paul Davis -
Ralf Mardorf -
Robin Gareus