On Tuesday 09 May 2006 01:08, Lee Revell wrote:
On Tue, 2006-05-09 at 00:47 +0200, Florian Paul
Schmidt wrote:
> On Mon, 8 May 2006 21:54:53 +0200
>
> Wolfgang Hoffmann <woho(a)woho.de> wrote:
> > Apropos: on your page on -rt setup (excellent page, btw., many thanks!
> > :-), you suggest raising "softirq-timer/0" to prio 99, to make
sleep()
> > function right (
http://tapas.affenbande.org/?page_id=40 sleep()
> > based/system timer).
> >
> > I did so, and got strange latencies (> 40 ms) exactly once every 10
> > minutes, caused by some routing-related action (rt_secret_rebuild)
> > being run by the softirq-timer/0 thread. Don't you get bit by that,
> > too? Kernel is 2.6.16-rt16
>
> I haven't been bitten by that. Do you also get xruns with [i suppose so,
> just asking to make sure]? I haven't had as much time as before to play
> around and test things, so maybe it has crept into the kernel recently
> or maybe i just always had high-res timers enabled.
To be more precise, I get xruns, not scheduling latencies. Scheduling
latency
is low (~50us), but softirq-timer/0 preempts all other threads when boosted
to prio 99, leading to xruns in jackd.
It's not a scheduler or -rt latency problem, it's a priority problem, since
softirq-timer/0 runs different jobs that would require different priorities:
wakeup sleep() at highest prio, rt_secret_rebuild et. al. at low prio.
With CONFIG_HIGH_RES_TIMERS=y it seems thatt softirq-timer/0 is no longer
needed for waking up sleep()ing threads, so it can be left at it's default
low prio (or even demoted to SCHED_OTHER?).
> > My solution is to configure with
CONFIG_HIGH_RES_TIMERS=y. Then,
> > sleep() wakes up correctly even with softirq-timer/0 being
low-priority
> > (SCHED_FIFO 1 or even SCHED_OTHER).
> >
> > In general I find adjusting priorities of the various softirq kernel
> > threads a bit of secret art. I can't find much documentation about
> > "what kernel thread runs which job" that would help making some
proper
> > decisions here. I found my desktop "feels" most responsive when
> > demoting all softirq thread to SCHED_OTHER. I did so after seeing that
> > with a non-rt kernel, bottom-half handler don't run SCHED_FIFO/_RR at
> > all. So -rt now gives me robust low latencies for jackd, and still
> > proper desktop feeling.
> >
> > Well, maybe this is getting off-topic for this list. But it seems to
me
trimming priorities between kernel and userland
threads is a bit like
no man's land.
I agree. Maybe Lee Revell knows more [CC'ing him]. Lee, you know
something about all these softirq threads? What exactly do they do?
Not really. The problem with making the softirq timer thread high
priority is that lots of random code gets run from softirq timer
context, including rt_secret_rebuild() which is a well known latency
killer.
Ah, ok. Now well known to me, too ;-)
Lately I've been focused on getting mainline
usable for audio - I have
not run -rt in a while.
It is a bit of a black art because you need lots of knowledge of the
kernel to know how to set the priorities. But that's the nature of hard
realtime - you really need to know a lot about the system to get the
priorities right.
I've never had to alter the priority of the softirq timer thread myself
- does it work OK if you leave the softirq thread alone?
With CONFIG_HIGH_RES_TIMERS=y, yes.
With CONFIG_HIGH_RES_TIMERS=n, no. Then, high-prio theads don't wake up from
sleep() if a mid-prio CPU hog is running. Typical scenario is a high-prio
watchdog like Florians rt_watchdog
(
http://tapas.affenbande.org/?page_id=38),
which fails to work unless softirq-timer/0 runs with higher prio than the
CPU
hog. It seems that softirq-timer/0 has to be scheduled for any thread to
wake
up. It's a bit like priority inversion.
If not try setting the softirq thread to a lower
RT priority than your
soundcard, JACK, and all other interrupt threads.
Yes, that should cure the xruns. But I wonder if jackd's watchdog can do
it's
job when a jack client is running wild? Probably not. softirq-timer/0 would
not get to run then, so the watchdog wouldn't wake up.
I've added Ingo to the cc: list, maybe he has
some input.
That would be helpful. Also adding Steve to cc: as he mentioned writing a
section in an O'Reilly book about how to use the -rt patch in
http://www.ussg.iu.edu/hypermail/linux/kernel/0604.2/0340.html
Wolfgang
Has anyone got a working x86_64 RT system? I can't boot rt kernels
here, hangs after loading the schedulers.
Loki