12 Dec
2010
12 Dec
'10
6:03 p.m.
On 12/12/2010 05:17 PM, Robin Gareus wrote:
long-answer and brainstorm:
The question is whether it is worth the time to make it, and what can be
learned from the information.
End-users won't learn much from those statistics. Unless you have
exactly the same hardware, it is close to impossible to draw
conclusions. One would need to
a) isolate the correlating factors and there are certainly much more
than just the kernel-revision, flavor, CPU-speed and audio-interface.
b) find values that can be measured reliably.
eg. what would you learn from information such as:
With kernel A on system B using sound-card C and jack-settings D
one can run for>=X seconds without an x-run.
where D is chosen to maximize X while f.i. minimizing latency.
Besides there is no common goal: some end-users require huge I/O (read
128 audio tracks from disk). Others only need one channel but low
latency, yet others only CPU power for effect-processing.
As for making something that is useful for developers: compare different
versions, do regression tests on the same system. It will take huge
effort to pull it off. Maybe the Phoronix can be used as basis: They
already laid a solid basis for statistical analysis and are working on a
system that allows to cherry-pick revisions, change-sets and compare
those. However AFAIK it runs in virtual-machine which makes it useless
for testing rt performance, but there may be options to use it on
bare-metal.
IMHO low-latency is quite overrated. There are few use-cases that
actually require it. If one really needs reliable low-latency (lets say
<20ms) - s/he needs a realtime-kernel (and for live-performances also
some other tweaks e.g. disable updated).
This is what I am thinking too. Which is why I think rt-kernels should
only be recommended to those users who need them, considering that
rt-kernels often cause non audio-related issues (problems installing
graphic drivers, etc).
Then there is the developers point of view. To whom and for what end are
they maintaining rt-kernels?
A RT-linux system either works or it does not. The
performance
differences between different working rt-kernel revisions are usually
quite subtle, and it is always possible to overload the system because
of hardware limitations.
For benchmarking different kernel revisions/systems: I suggest to stick
to the rt-test tools such as hwlatdetect, cyclictest, pi_stress etc.
Something that would be useful to have is a jack-audio stress-test suite!
After installing a new system or getting new hardware: automatically run
some tests to check the limits of the system.
ardour's jacktest.c is a first step in that direction (testing CPU/DSP
load). It could be supplemented by an I/O test tool (read audio-files
from disk). An extended version of latentor, that uses jack_delay to
find the lowest possible latency with no x-runs may be a 3rd, etc
But that information would only be useful for you, not for others.
2c,
robin
--
ailo