[LAU] multiJACK patch management: the first glimmerings of success

[Jörn Nettingsmeier]

On 04/02/2016 06:33 PM, Jonathan E. Brickman wrote:
> For those who may be interested, I am seeing the first small glimmering
> of what looks like real success, in running multiple JACK servers on one
> box for the purpose of distributing DSP load.  I can now reliably have
> two netjack1 slaves connect to one netjack1 master, the master running
> ALSA to hardware, all within one Linux box under one kernel and one
> filesystem.
>
> The way that this works is "containerization" or "sandboxing", which can
> give an IP to anything ranging from one running binary to a large
> collection.  I started the current effort using Docker containerization,
> but its standard setups require a lot of filesystem recreation, each
> Docker container has an entire working OS filesystem apart from the
> kernel; but during the studies for Docker, I blundered into something
> else called firejail, and firejail appears to be just what the doctor
> ordered, it seems to do just about everything while using the existing
> filesystem, and one can very easily turn off the features one doesn't need.
>
> To cut to the chase, after boot and system prep, first one creates an IP
> bridge:
>
>     sudo brctl addbr br0
>     sudo ifconfig br0 10.99.99.1/24
>
> and then starts JACK in the sandboxes:
>
>     # The master hardware sandbox is named MASTER, the QJackCTL profile
>     is named MASTER,
>     # the JACK server is named MASTER.
>     nohup firejail --name=MASTER --noprofile --net=br0 --ip=10.99.99.10 \
>          /usr/bin/jackd -n MASTER -m -dalsa -r48000 -p256 -n2 -Xseq -D
>     -Chw:NVidia -Phw:NVidia \
>          > ~/LOGS/jack-master.log &
>
>
>     # The SRO hardware sandbox is named SRO, the QJackCTL profile for it
>     is named SRO,
>     # the JACK server is named SRO.
>     nohup firejail --name=SRO --noprofile --net=br0 --ip=10.99.99.15 \
>          /usr/bin/jackd -n SRO -dnetone \
>          > ~/LOGS/jack-sro.log &
>
>
>     # Ditto for STRINGS.
>     nohup firejail --name=STRINGS --noprofile --net=br0 --ip=10.99.99.20 \
>          /usr/bin/jackd -n STRINGS -dnetone \
>          > ~/LOGS/jack-strings.log &
>
> And then connects the JACK servers, by adding jack_netsource processes
> into the existing MASTER sandbox:
>
>     #!/bin/bash
>
>     # SRO
>     nohup firejail --join=MASTER jack_netsource -s MASTER -H 10.99.99.15
>      > ~/LOGS/netsource-sro.log &
>
>     # STRINGS
>     nohup firejail --join=MASTER jack_netsource -s MASTER -H 10.99.99.20
>      > ~/LOGS/netsource-strings.log &
>
> I have not tested further yet, no clients -- am working on something
> closer to production now -- but this is looking very good, the
> connections are reported successful, zero xruns, and on this prototyping
> box -- 2.6GHz AMD quad, ten-plus years old -- 0.8% DSP in use on all
> three JACK servers and very low memory usage.  This is considerably
> better than I had hoped.
>
> The sandboxed binaries above, cannot reach outside the one box as
> written above, but the excellent and very responsive developer of
> Firejail has provided a method
> <https://github.com/netblue30/firejail/issues/372>. As a result this
> could all be done  box-independently -- for instance, if one's "monolith
> is the building" (going to have to remember that, Patrick), one could
> have all indicated motherboards NFS or SMB to one file server, and then
> use a central control GUI machine to run all firejails on whichever
> hardware was proven most appropriate, and testing could become much
> easier.  Along the way I also found 'xpra' likely to be a very good way
> to setup such a central control, am going to test that as a side project.
>
> I tried using netjack2 first, but ran into mystery behavior, xruns
> started piling huge when the second slave connected.  So I went with
> netjack1, especially because Patrick Shirkey already proved the above
> paradigm in multibox mode using netjack1.  I am currently using netjack1
> under jackd2, but will change to netjack under jackd1 if a reason to do
> so appears.
>
> I am now working on a working dual-JACK prototype, in a near-production
> design, as a next step towards a generally transportable MultiJACK Patch
> Management methodology and the next big build of my Box of No Return :-)
>
> Cheers, and thanks everyone!!!!


Hmmm... I wonder: this is either a fundamental bug in JACK2 that none of 
the developers seem to be able to reproduce, a complete misunderstanding 
on the part of the original poster, or something completely different 
that is not sufficiently understood. In any case, I'd say it would be 
useful to come up with at least a working hypothesis of why this should 
improve things.

I have never used a graph as complex as Jonathan's, but I don't see how 
adding virtual machine and networking overhead could improve throughput 
when JACK2 should already parallelize everything that can be 
parallelized... Wouldn't the time be better spent to analyze JACK2's 
scheduling behaviour and catch bugs if there are any?

When exporting, I have seen something that might be similar to what 
Jonathan might be seeing: a freewheeling, 100% CPU-bound process only 
seems to use a fraction of the available CPU even on a single core. Now 
this might indicate a real problem, or it might indicate that the CPU 
load tools I'm using do not accurately report what's going on under 
tight real-time conditions with loads of context switches.


-- 
Jörn Nettingsmeier
Lortzingstr. 11, 45128 Essen, Tel. +49 177 7937487

Meister für Veranstaltungstechnik (Bühne/Studio)
Tonmeister VDT

http://stackingdwarves.net