[LAU] Kxstudio RT kernel vs low latency

[Joakim Hernberg]

On Thu, 24 Jul 2014 11:18:16 -0700
Russell Hanaghan <hanaghan.osaudio at gmail.com> wrote:

> Is anyone achieving significant audio latency gains or xrun stability
> using a RT kernel with Kxstudio or similar Ubuntu studio distro's? 

Disclaimer: This is very simplified and not 100% correct in all aspects,
but in the interest of brevity and due to yours truly's limited
understanding of such things :)

Another disclaimer: you might not need deterministic "hard" low
latency..  Maybe you don't mind high latency or the occasional xrun, if
so you might not need a -rt patched kernel.

The most important difference between a lowlatency (PREEMPT) and
an -rt pathed (PREEMPT RT) kernel is that the maximum kernel scheduling
latency is a lot lower. Why is this important?  Because if the kernel
can't execute the soundcard interrupt handler and the audio threads
promptly enough it will produce an xrun.

There is a deadline on when audio processing has to be finished, given
by the sample rate and the buffer size used.  This imposes a limit on
the latency of the audio processing, one would also have to take into
account scheduling latency and other overhead.

rt-tools is a package of tools to test and trouble shoot the rt kernel
(or any kernel for that matter), it contains among other programs one
called cyclictest, that can be used to get a good idea of the maximum
kernel scheduling latency.

Assuming the system is configured to let your user run SCHED_FIFO
threads at a priority up to 99, run something like: "sudo cyclictest -m
-n -Sp99 -i100 -d0" (Running it as root will allow it to disable C
states powersaving).

With these parameters it will try to run as many threads as you have
cpus and it will check how long it takes the scheduler to execute them,
thus giving a good indication of the scheduling latency. Note that the
values given are in microseconds, so 1,000 represents 1 millisecond.

Another important point is, that to ascertain max kernel scheduling
latencies, the system needs to be put under load. rt-tools also
contains the program hackbench, which can be used to put load on
the system.  Try something like "hackbench -l100000" (to make it run a
little longer).  You might also want to stress the disk io system
(and others) with bonnie++, etc.  Another good idea is to leave it
running for a few days.

For instance on my own i7-2600k Archlinux system I see max latencies of
about 100 usecs with the rt kernel, with the vanilla distro kernel
(which is lowlatency) I see peaks into the tens of milliseconds... Each
such peak means xruns due to the kernel not scheduling the needed
threads promptly enough...

There are many other reasons for xruns..  A few examples (in no
particular order of importance): Non Maskable Interrupts, SMT
(hyperthreading), certain hardware/kernel modules combinations (nvidia,
wifi, bt, etc come to mind), power saving, a badly configured system,
"badly" programmed software, etc.

But, if the kernel isn't able to schedule your audio work fast enough,
it simply won't be finished on time and you will get xruns!

Another important topic is setting the right rt (SCHED_FIFO) priority
of the threads needed for audio (in order for them to be
processed before anything else).

On the rt kernel, part of the interrupt handlers are exposed as threads
to user space, the same can be achieved on lowlatency by using the
"threadirqs" kernel boot flag.

By default most everything on the system runs as SCHED_NORMAL, that
means as normal non rt threads, with no particular priority.  The
interrupt handlers run as SCHED_FIFO at priority 50, this means that
they will be scheduled with preference to all other threads.

What is needed for xrun free audio, is for the audio processing to run
SCHED_FIFO at a priority higher than that used to process the other
hardware interrupts and the rest of the running threads, thus being able
to finish processing the audio buffers before the deadline.

There is a script called rtirq that will configure the priority of the
interrupt handlers for low latency audio, or it can be done manually.

This is what I do on my own system to set the priority of the interrupt
handler for my soundcard to 98: "chrt -f -p 98 `pgrep
irq/18-snd_hdsp`" (be carful to use the ` character instead of the '
one.  The name of the interrupt handling thread might be divined by
doing cat /proc/interrupts.  Note that it might change on reboot, so
possibly the rtirq script is a good idea.

I run jackd realtime at priority 80.

This has the effect that the soundcard interrupt handler will
executed at the earliest possible moment, then jack will run the jack
client's audio processing callback (at priority 70). Only after that is
done will the systen start scheduling the interrupt handlers for other
hardware and all the other threads running on the system.

> If so, which kernel / where /do I gotta roll my own?

This is a question I can't answer for you, but IMO any linux audio
oriented distribution ought to make a realtime patched kernel available
to it's users.

-- 

   Joakim