[LAD] Meego pulseaudio "compliance" and "enforcement" (was Re: [Meego-handset] Enabling Speakerphone)

Niels Mayer nielsmayer at gmail.com
Sat Dec 18 18:16:56 UTC 2010


Marco -- many thanks for your reply -- I've taken the liberty of
CC'ing  this informative reply to the Linux-Audio-Developers list
(your CC was not posted as you're not a subscriber:
http://lists.linuxaudio.org/pipermail/linux-audio-dev/2010-December/thread.html
).

The list comprises a good number of people with expertise in both
pulseaudio; hopefully the Jack sound server authors, including Paul
Davis, will be willing to publicly share their perspectives on the
issues raised regarding the role of pulseaudio on a handset and  Linux
audio performance/latency/efficiency issues.

Here's a link to Marco's original  post to meego-handset mailing list,
forwarded below:
http://lists.meego.com/pipermail/meego-handset/2010-December/000090.html

---------- Forwarded message ----------
From: Marco Ballesio <gibrovacco>
Date: Sat, Dec 18, 2010 at 4:42 AM
Subject: Re: Meego pulseaudio "compliance" and "enforcement" (was Re:
[Meego-handset] Enabling Speakerphone)
To: Niels Mayer <nielsmayer>
Cc: meego-handset at lists.meego.com, Linux Audio Developers
<linux-audio-dev at lists.linuxaudio.org> [...]

Hi,

sorry for the late reply but, whew, this was really long, definitely
more than what my poor brain can handle in a single bunch ;). Maybe we
could split this into a "Resource Policy Framework" and "Pulseaudio on
Embedded" couple of threads.

As a general comment on PA, it may appear odd but when I started
working with it I shared 100% of your considerations. Working with it
(not ON it), I started realising that, ok, it may not be perfect (and
which software actually is?) but it brings definitely more benefits
than troubles to the system. See my notes below for more details.

..snip..

>
> Hopefully there's more than just source-code to describe the policies
> and enforcement.

Maybe you've read in some other of my posts here and there, I'm trying
to get a Resource Policy Framework wiki somewhere. In the meanwhile,
documents are scarce but present:

- The MeeGo conference presentation:

http://conference2010.meego.com/session/policy-framework-flexible-way-orchestrate-multiple-functionalities-meego-devices

Does not say too much, but it comes with some slides and many "mmmmh"
(keepalive signals).

- The resource policy application developer's manual:
http://wiki.meego.com/images/Meego-policy-framework-developer-guide.pdf

Mysteriously, it's not linked or referred to anywhere.. at least, now
it's on this thread.

- The source code: it may twist more than one nose but, at the end, I
always like to check in the sources before/after Googling. Documents
may be obsolete (who knows?) or expose a partial view. Code can't ;).
Not that I want to say with this that we don't need more documents
about the Resource Policy Framework...

> Is there a design document stating the
> "organizational" or "legal" or "human" reasons for said policies and
> enforcements?

My pointers of above may begin to satiate your hunger (appetisers?).

>
> And, as a developer, where is the documentation for the appropriate
> layer to use, the appropriate API, etc -- e.g. for the original
> question that started this thread -- how to switch on/off speakers,
> FM-radio transmitter,  etc on handset. ( cf my original reply
> http://lists.meego.com/pipermail/meego-handset/2010-December/000066.html
> )

Unfortunately, this is still in the "dark side" of the documentation,
which is currently application-centric. Hopefully one day we'll have
an "Adaptation developer's guide", in the meanwhile, a good starting
point may be checking the sources, the N900/Meego ruleset and posting
questions here. If it's not enough, we may set up an IRC channel (if
there's enough interest)...

>
> Something high-level like the following Maemo document would be very
> helpful -- but I have been unsuccessful finding such documentation for
> Meego: http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Architecture/Multimedia_Domain

See my notes above about the project wiki. Your pointers are
definitely good hints, thanks.

>
> Also, why not use ALSA use-case-manager:
> http://opensource.wolfsonmicro.com/node/14
> UCM appears to be scheduled for other distros "handset" work:
> https://wiki.ubuntu.com/Specs/N/ARM/AlsaScenarioManager

this project covers the audio-specific cases: for what I understood it
might very well be used as an ALSA enforcement point in the Framework
so, more than compete with Resource Policy it would be complementary
in case you don't really want/need to use a sound server. BTW, I could
not find so much info, maybe you can help me to retrieve:

- more than just source-code to describe how it works.
- the developer's documentation for the appropriate layer to use, etc.
- something like:
http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Architecture/Multimedia_Domain

..snip..

now, the pulseaudio cases. First of all, please note I'm not the best
advocate for it, btw my 0.05 €.

Posting some extracts of your comments to the PA mailing list will
definitely give you a better insight for the internals of the server.

> Regarding my old nemesis, pulseaudio ( http://tinyurl.com/2976vu6
> == http://old.nabble.com/uninstall-pulseaudio-to-increase-audio-app-stability-across-updates-(was-Re:-yum-update)-td27759501.html#a27759501
> ), I think the only "elegant" thing about pulseaudio is that it's
> bluetooth handling works for those that care about bluetooth;  given
> the number of bug-reports and incompatibilities pulseaudio generates
> in different distros, I'm not sure "elegant" is the right word....

And actually "elegant" was not referred to pulseaudio, but to the way
its ports can handle audio routing instead of using ALSA (they bring
better synchronisation).

Yes, the sound server has shown instabilities in some cases and I
personally used to uninstall it on my Ubuntu box immediately after the
upgrades. Recently but many improvements have been made at a point
that, on a PC, I'm nowadays not seeing it anymore jumping to a steady
30% of the CPU or completely jamming any audio output in the middle of
a movie.

Another _personal_ feeling I had is that pulseaudio was pretty hard to
configure for a generic HW (like distros tend to do), but works pretty
well after being tuned for a given one (like 95% of embedded products
are). Very often the root issue may have been a wrong configuration
for your system.

>
> From my position, as a multimedia/ALSA/linux-audio developer, having
> to go through pulseaudio sounds like an all-around bad idea, and to
> have "enforcement" or "compliance"  attached makes it sound even
> worse. Tell me it ain't so!

I can agree from a kernel/ALSA pov (everything in userland appears to
be less efficient than any kernel drivers ;) ), I strongly disagree
from the system architecture's one. Some points you should consider:

- Audio outputs frequency tuning: in the real word a crappy speaker
costs less than a good one (strange?) so, unless you've a very high
target price, you'll usually have to take the maximum out of garbage
by "massaging" the output spectrum in order to get an adequate overall
transfer function. And this must work with all of the applications
-even the ones coming from third parties on which you don't have the
bare minimum control on- and different analog outputs (front, rear,
bt, ...). Not that pulseaudio comes with these features
out-of-the-box, but it's quite easy to plug them in.

- Audio inputs: the dual of audio outputs.

- Mixing: it's possible that users will want more than one application
accessing the audio output at the same time (e.g. navigator and car
radio) or that they'll want to switch bw two different applications
all of a sudden (e.g. radio and ringtone). In all these cases they'll
want a good mixing and no sudden power steps when the ringtone app
does not know about the volume level used for the radio.

- Acoustic delay estimation: sitting in the middle of pulseaudio
(running as a real-time thread) it's easier to write a proper AEC.
Back to 2004 I've tried to grant, with a TI aic31, constant latencies
for both LEC and AEC by just using ALSA and the application... It took
a really long time to work.

- It may not appear so, but it's energy-efficient (see my comments
below if you just jumped out of your chair).

Now, I don't mean that those things can be handled only by pulseaudio.
Actually developers could write their own sound server, or use an
already available one, to do all of this, but..

- How long would it take to be better than pulseaudio? (e.g. wrt my
points above).
- Can it grant the same solid user base and portability across embedded HW?
- Could it grant the same level of contribution from companies with
solid experience on the subject?
- Are there architectural flaws for which pulseaudio couldn't
effectively handle these cases? If yes, has anybody tried to discuss
about them on the PA mailing list?

>
> There is a growing class of applications that do not want or need
> pulseaudio around -- those using http://jackaudio.org/ .  When the
> jack audio server launches, the first thing it does it use dbus to
> disable pulseaudio. Is that also non compliant?

I've no religious issues against jack and actually I don't have that
much experience with it, so I'd like to know more about jack on
embedded. I'd like to know, for instance, how many embedded, low-power
devices are already using it and with which degree of success. Also it
would be great to know if anybody has interfaced it with a cellular
modem to handle Circuit-switched cellular calls, and if the device has
actually been certified for such a service in any countries.

>
> It seems inappropriate to preclude an entire class of application --
> real-time "pro" audio and video apps that utilize the Jack audio
> server to provide low-latency, tightly synchronized audio -- as needed
> for modern multimedia creation and playback. Perhaps such applications
> are a stretch for an OMAP3-class device, but given the many
> audio/media apps listed in http://omappedia.org/wiki/PEAP_Projects ,
> clearly OMAP4 and beyond might not be, even on a puny "handset." Of
> course, those making such audio apps might sidestep pulseaudio
> compliance/latency/inefficiency issues by using
> http://opensoundcontrol.org/ and an external DAC (
> http://gregsurges.com/tag/dac/ ).
>

please correct me if I'm wrong, but if I understood well most of the
apps using _exclusively_ jack are aimed to audio/video
composition/editing. Now, as we are on the IVI ML I think it's quite a
strange use case (even though I must admit I don't know which
requirements you have).

> Finally, it seems odd that in a "handset" environment, pulseaudio is
> an absolute requirement. To me, it is just a waste of batteries,

see below (and above).

> and a
> terrible source of unnecessary context switching and interrupts during
> audio playback.

well, it runs as real-time priority, so it's executed when it's
scheduled. Context switches would not be different for any process
(e.g. jack) running with the same scheduling.

> It's sort of like being forced to drive around in a
> gas-guzzling, oversized sport-utility vehicle with 10 foot tires and 5
> feet of ground clearance -- just to drive to the market or work on a
> well-paved freeway on a summer's day -- even when one might prefer a
> bicycle, motorcycle, sports-car, subway,  or whatever tool is best for
> the job.

funnily, this is EXACTLY the same thing I thought the first time I
profiled pulseaudio on the N900, then I found some reasonable
explanation (scroll a little below).

..snip..

> Take for example HD audio/video playback -- something where you need a
> "sportscar" to get the latency and sample rate associated with the
> audio stream while also performing HD decoding (e.g. 16bit/96K audio
> is supported on omap3430 per
> http://and-developers.com/device_information ). Pulseaudio typically
> operates at a fixed rate and forces resampling to that rate, causing
> an oft perceptible loss in fidelity.

Yes, this is a bad aspect, but it's actually possible to change the
output bitrate of the server (it depends on the port used). Maybe you
could query on the PA mailing list for more knowledge on the subject.

>
> IMHO what is needed is not a "digital mixer" and resampler and extra
> user-space processing before going into the kernel and out to the
> sound hardware via ALSA drivers. We certainly need "use case
> management" and the notion of a digital patch bay, and some way of
> smoothly mixing between sounds, glitch free switching of sample rates
> at the ALSA level, and then choosing the direct path to hardware
> that'll best handle the "main audio task" we're doing -- e.g. playing
> music, watching a movie, making a phone call, or just using the screen
> UI.  What isn't needed is "desktop networked audio" capabilities of
> pulseaudio, or any extra user-space streaming/mixing/resampling of
> signals.

so we'll end up writing our own sound server or using a different one,
won't we? As an alternative, we may have to heavily modify ALSA to
suit our needs, and we'll not have yet covered all of the needed
features.

>
> The inefficiencies introduced by pulseaudio is evidenced by the
> n900/meego-1.1 handset, which cannot maintain audio synchronization
> during audio or video playback if even the slightest trace of
> something else is going on, including a wireless network scan, or even
> just cursoring around in a networked terminal (which goes through the
> wireless stack in my setup).

Personally, I would not say that MultiMedia with MeeGo on the N900 is
already at the same quality of Maemo 5. Lots of tuning is imho still
missing.

>
> On Maemo/n900 note what happens when playing back an Mp3 -- pulseaudio
> consumes twice the resources at "high priority" of the decoding
> process:

You must consider that here there many algorithms involved that
users/applications don't even know about, like the spectrum
optimisations I've been mentioning before. If you could run an
oprofile session over your tests you'd see that the actual CPU
attributable to PA is pretty lower than your figures.

..snip..

> I'm sure a simple experiment could determine exactly the "effect" of pulseaudio:
>
> Play the same playlist until the battery wears out using the same
> software player outputting first to pulseaudio (pref not through
> ALSA's pulseaudio driver because that wouldn't be "fair" to go through
> the kernel twice) then play the same through ALSA "dmix" interface,
> just to emulate pulseaudio's mixing/resampling functionality.

As written above, PA is not only resampling and mixing here.. imho if
the user/the developer didn't have to adjust the system to work with
PA in order to achieve this, it's a success. The CPU usage comes from
the fact we're actually running something we need.

> I would
> imagine the pure-ALSA solution would pass the "energizer bunny' test
> for more hours and far fewer kernel/userspace context switches.

Sure it would, but with a lower set of features (and an higher cost
for the audio system). It's up to the device price target/customer's
pockets and needs to decide what's better.

> Although a realtime userspace process like pulseaudio can help deliver
> stable audio in a multicore  environment -- it may end starving out
> other processes on a slower uniprocessor. Which is why I believe a
> pulse-audio-free solution should be available and still be "compliant"
> on low-end systems.

I would agree in some cases, that is when the system conditions don't
require it to run multi-application use cases and the price tag for
the audio system is irrelevant wrt the audio quality.

Note for the reader: if you've reached this point it means you're
really interested about pulseaudio on embedded devices :D.

Regards

>
> The one place where pulseaudio is currently helpful is in hooking up
> bluetooth devices. But ALSA has it's own bluetooth layer as well, and
> other architectures for audio shouldn't be precluded:
> http://bluetooth-alsa.sourceforge.net/future.html  or Phonon
> (http://pulseaudio.org/wiki/KDE ).
>
> -- Niels
> http://nielsmayer.com
>



More information about the Linux-audio-dev mailing list