[LAD] Meego pulseaudio "compliance" and "enforcement" (was Re: [Meego-handset] Enabling Speakerphone)

Kai Vehmanen kvehmanen at eca.cx
Sun Dec 19 11:28:23 UTC 2010


On Sat, 18 Dec 2010, Niels Mayer wrote:

> It has decent performance on maemo and appears to use pulseaudio,
> which eats 1/3 of the CPU of the 'sunvox' process. The sunvox
> 1916  1162 user     S     6388  2.5 35.1 /usr/bin/sunvox
>  825     1 pulse    R <   3812  1.5 12.1 /usr/bin/pulseaudio --system

so as mentioned in other replies, this is not really pulseaudio to blame. 
On N900 there is some fixed processing that must be applied to all streams 
(and there are different pipelines for different uses and routes, so it's 
not always constant). With N900 this code is now in PA (as it's the system 
mixer, there's no hw mixer under the hood). The load is smaller e.g. when 
you use earpiece/headset (versus speaker), and also it should be lower if 
you have the apps use the hw rate (48kHz in N900 case). You can challenge 
the N900 design, but none of this is really specific to Pulseaudio.

The N900 audio design in fact gives quite a lot of responsibility to Linux 
side (perhaps an interesting comparison is the Palm Pre as it also uses 
Pulseaudio). This has some cost, but also a great deal of flexibility (in 
terms of what can be implemented, but also for work flows as you can 
devel&debug the whole audio system with same Linux development tools). The 
more traditional approach is to put all of this logic to a separate HW 
block, but then the Linux side has even less control and access to what 
happens to the audio streams. In these cases, Pulseaudio becomes just a 
frontend mixer for streams originating from/to Linux apps. In this sense 
N900 allows you much closer access to the HW, and there is more potential 
for really low-latency uses with this architecture. I'm personally 
thrilled of the potential of this approach (while not claiming the 
implementation is perfect yet).

> Doing an "ls -lR /" in a remote xterm (over SSH) results in some audio
> glitching, but no "desynchronization" where the audio just stops
> playing.

This is a real problem for sure.

I believe the cause for the glitches is that sunvox's audio thread is not 
run under SCHED_FIFO/RR. The real/nonrealtime sync point really should be 
in application's hands (like it is by default in JACK apps). If Pulseaudio 
alone tries to handle this (trying to buffer audio from/to nonrealtime 
apps), you will get huge latencies (and huge utilization of 
unswappable locked memory).

But the problem is not really Pulseaudio, but instead the mechanisms to 
allow/manage FIFO/RR. JACK of course provides a good system for setting 
the scheduling for clients, but the core problem is not really setting the 
scheduling policy, but how to manage and coordinate the system-wide impact 
of multiple real-time apps, and their possible conflicts.

When you build a JACK system, jackd and related apps take priority over 
everything else on the system (in real-time sense). Avoiding glitches for 
them is of utmost priority (UI can lag as long as audio runs without 
xruns). On a mobile device, there are various other important functions 
that need real-time (the exact list varies from product to product but 
there are always some), and which must work. So even if you use apps like 
Sunvox, you still want to be able to receive/make calls reliably, be 
certain that the device wakes you up in the morning, and is ready for 
quick camera shooting when the situation arises. So overall system 
reliability and responsiveness is really important. Having lots of 
uncoordinated (e.g. developed separately, not referring to quality or 
cases of possible malicious intent here) real-time apps is a tricky 
problem. It's not a new problem, agreed, but still a difficult to solve in 
the generic case.

Ideally we'd have more scheduling policy options in Linux for this (there 
have been no shortage of proposals, also from Nokia), but currently 
FIFO/RR are still the best options on the table for low-latency audio.

Now there are some building blocks in place already in MeeGo/Maemo (e.g. 
cgroups scheduling is used and there is a system for managing resources), 
but there are still gaps that cause trouble to apps. Currently the only 
way to achieve JACK-style low-latency is to write your app's audio thread 
as a PA module. Aside cgroups, real-time watchdogs could be used, and even 
with the risk of restarting past flamewars ;), PolicyKit is obviously one 
option as well to improve this area.

But to summarize, the core problems that still need addressing w.r.t. 
JACK-style low latency, are not really related to Pulseaudio (nor solved 
simply by switching to JACK). If MeeGo/Maemo would have used JACK, but had 
no managed system in place for passing FIFO/RR to apps, the only option to 
implement low-latency engines would be to write them as inprocess JACK 
apps. So you'd have the same exact problem as with Pulseaudio (and have 
many apps running nonrealtime despite their need for low-latency access to 
the mixer -> glitches in cases where most certainly there should be none).

Disclaimer: while I have a daytime job working for MeeGo, I'm not exactly 
working in this area in MeeGo, so some of the above might be obsolete 
info, but hopefully helpful still to this thread.

More information about the Linux-audio-dev mailing list