[LAD] PipeWire

[Len Ovens]

On Mon, 19 Feb 2018, Wim Taymans wrote:

> PipeWire started as a way to share arbirary multimedia, wich requires vastly
> different requirements regarding format support, device and memory management
> than JACK. It wasn't until I started experimenting with audio processing that
> the design started to gravitate to JACK. And then some of JACKs features became
> a requirement for PipeWire.
>
> The end goal of PipeWire is to interconnect applications and devices through
> a shared graph in a secure and efficient way. Some of the first applications
> will be wayland screen sharing and camera sharing with access control for
> sandboxed applications. It would be great if we could also use this to connect
> audio apps and devices, possibly unifying the pulseaudio/JACK audio stack.

By unifying I think you mean both things in one server rather than both 
making jack work like pulse or pulse work like jack. I have been using 
jackdbus as my audio server/backend with pulse as a desktop compatability 
layer for 3 to 4 years now with reasonable success. Jackdbus takes care of 
all physical audio devices (I have no bluetooth audio devices) with my 
multitrack audio device (an older ice1712 based delta66) as jack's master 
and any other devices as clients via zita-ajbridge (with SRC). In general, 
I don't use devices cannected through SRC for recording but many beginner 
users have bought "pro USB mics" to start recording and so SRC is "a 
thing".

I run pulse without the alsa, udev and jackdbus-detect modules but do load 
jacksink/source via script as needed. I use my own script because it 
allows me to name my pulse ports so that pulse sees a device name rather 
than just jackd. I do not know the internals of pulseaudio, but have found 
that pulse will sync to any real device it happens to have access to even 
though no stream is using that device. This ends meaning that data is 
transfered to jackd on that devices time schedule rather than jack's with 
the result of xruns in jackd and even crashes when jackd is put in freerun 
mode. By running pulse with no alsa modules and no udev module (which auto 
loads alsa modules when a new device is detected), both of these problems 
are solved.

The one problem I have left is that pulse then has to follow jackd's 
latency model. This is probably because jackd-sink/source are in the 
sample category rather than well thought out and finished. As jack's 
latency goes down (it can be changed while jackd is running), jack's cpu 
load goes up as expected, but it stays in reasonable limits. However, 
pulse is forced to follow along and pulse uses more than double the cpu as 
jack does. Along with this some desktop applications start to fail 
noticably. Skype is a good example of this because it does actually see 
some use in the profesional audio world in broadcast application where 
skype is sometimes used for live remote contribution. (think phone in talk 
show or even news) In such a case, the local studio may be running totally 
in jack using something like idjc with skype linked in using pulse 
bridging. (thankfully asterisk can deal with jack directly and already 
expects low latency operation so normal phone calls just work) Low latency 
jack operation is important in an announcer application as monitoring as 
often done with headphones where a delay of one's own voice may be 
annoying. So jack needs to run at 5ms or so while skype seems to think 
30ms is quite normal (and uses echo cancelation so the talker can't hear 
their own delayed voice).

What this points out is that there are two different requirements that 
sometimes need to be met at the same time. Pipewire has the advantage of 
knowing about both uses and being able to deal with them somewhat more 
gracefully if it chooses to. Desktop needs it's own buffering it seems.

Certainly most people who use jack much would have liked to see jack 
become standard with a pulse like wrapper for desktop. The development 
energy just wasn't there.

> Because the general design is, what I think, now very similar to JACK, many
> people have been asking me if I'm collaborating with the linux pro-audio
> community on this in any way at all. I have not but I really want to change

It does not really matter if pipewire is similar to jack in operation. 
Jack allows things that some applications require and there are users who 
do not have pulse on their system at all. so even if pipewire did not 
allow jack clients to directly connect, jack is still around, still in use 
and will be for some time. (do not be disapointed when some people choose 
to remove pipewire in their new installs and replace it with jackd1, they 
may be vocal, but a small number of people)

> that. In this mail I hope to start a conversation about what I'm doing and I
> hope to get some help and experience from the broader professional audio
> developers community on how we can make this into something useful for
> everybody.

While I have done some development using the jack API, you will have 
noticed that most of my points above are from a user POV.

> sink queue to be picked up in the next pull cycle of the sink. This is somewhat
> similar to the JACK async scheduling model. In the generic case, PipeWire has to

There will be some people who will say jack async is not good enough, but 
they will likely also be those commented on above who will use jackd1 (and 
only LADSPA plugins). This is not in any way a put down of these people, I 
think there are uses where a jack only system will remain the best 
approach. Just as there are still many headless servers with no X or 
wayland.

> The idea is to make a separate part of the graph dedicated to pro-audio. This

Thank you, that is absolutely a requirement if you wish to avoid the 
situation we have now of so many people either hacking pulse to work with 
jackd, removing pulse, complaining desktop audio is blocked when an 
application uses alsa directly, etc. What it comes down to, is that 
profesional audio users will continue to use jackd unless pipewire 
properly takes care of their use case. Because of where pulse has gone, do 
expect a "wait and see" from the pro community. There are still a number 
of people who very vocally tell new proaudio users that the first thing 
they should do is to remove pulse when in most systems this is not needed. 
These poor new users are then left with a broken system because they are 
not able to do all the workarounds needed to get desktop audio to work 
again. Having people who use proaudio working with you from the start 
should help keep this from happening. There will still be people against 
it, but also people for, who are also vocal.

A request:
it is hard to know exactly how pipewire will work, but one of the requests 
I hear quite often is being able to deal with pulse clients separately. 
That is being able to take the output of one pulse client and feed it to a 
second one. This could be expanded to the jack world. Right now, jack sees 
pulse as one input and one output by default. This is both good and bad. 
It is good because, most pulse clients only open a pulse port when they 
need it. This makes routing connections difficult to make manually. The 
pulse-jack bridge provides a constant connection a jack client can connect 
to. This is bad because it is only one connection that combines all pulse 
audio including desktop alerts etc. Some way of allowing an application on 
the desktop to request a jack client as if it was an audio device would be 
a wonderful addition. Also, a way of choosing which port(s) in the jack 
end of things should be default would be nice. Right now, when pulse auto 
connects to jack it select system_1 and system_2 for stereo out. On a 
multi-track card system_9 and system_10 (or any other pair) may be the 
main audio out for studio monitoring. Ports 9 and 10 just so happen to 
s/pdif on my audio interface.

I have also been overly long, but a replacement audio server affects a lot 
of things. It is worth while taking the time to get it right.

--
Len Ovens
www.ovenwerks.net