30 Mar
2004
30 Mar
'04
3:56 a.m.
Hi
Could anyone explain how I should go about killing a potensially
unresponsive thread? Of course there's pthread_cancel, but I don't wish to
wait for the other thread to reach a cancel point. Basically I'm trying to
implement a watchdog for a realtime audio thread. Any suggestions are
welcome, thanks.
Arve Knudsen
30 Mar
30 Mar
6:43 a.m.
On tis, 2004-03-30 at 08:51, Arve Knudsen wrote:
Hi
Could anyone explain how I should go about killing a potensially
unresponsive thread? Of course there's pthread_cancel, but I don't wish to
wait for the other thread to reach a cancel point. Basically I'm trying to
implement a watchdog for a realtime audio thread. Any suggestions are
welcome, thanks.
man pthread_cancel:
...
...
pthread_setcanceltype changes the type of responses to cancellation
requests for the calling thread: asynchronous (immediate) or deferred.
The type argument is the new cancellation type: either
PTHREAD_CANCEL_ASYNCHRONOUS
to cancel the calling thread as soon as the cancellation request is
received, or
PTHREAD_CANCEL_DEFERRED
to keep the cancellation request pending until the next cancellation
point.
/jens
1:16 p.m.
On Tue, 30 Mar 2004 11:38:54 +0200, Jens M Andreasen
<jens.andreasen(a)chello.se> wrote:
On tis, 2004-03-30 at 08:51, Arve Knudsen wrote:
I was thinking PTHREAD_CANCEL_ASYNCHRONOUS would be the option, but I
already rely on the default behaviour (deferred) for the regular cases,
where I politely ask the thread to exit. So I was thinking of some other
alternative. The older code I'm using as a reference, makes use of
pthread_kill. At first I thought the NPTL lib simply lacked this function
(not in pthread.h), but I see now its included through signal.h :p
Thanks
Arve Knudsen
Hi
Could anyone explain how I should go about killing a potensially
unresponsive thread? Of course there's pthread_cancel, but I don't wish
to
wait for the other thread to reach a cancel point. Basically I'm trying
to
implement a watchdog for a realtime audio thread. Any suggestions are
welcome, thanks.
man pthread_cancel:
...
...
pthread_setcanceltype changes the type of responses to cancellation
requests for the calling thread: asynchronous (immediate) or deferred.
The type argument is the new cancellation type: either
PTHREAD_CANCEL_ASYNCHRONOUS
to cancel the calling thread as soon as the cancellation request is
received, or
PTHREAD_CANCEL_DEFERRED
to keep the cancellation request pending until the next cancellation
point.
31 Mar
31 Mar
6:06 a.m.
On tis, 2004-03-30 at 18:11, Arve Knudsen wrote:
...
...
I was
thinking PTHREAD_CANCEL_ASYNCHRONOUS would be the option, but I
already rely on the default behaviour (deferred) for the regular cases,
where I politely ask the thread to exit. So I was thinking of some other
alternative. The older code I'm using as a reference, makes use of
pthread_kill. At first I thought the NPTL lib simply lacked this function
(not in pthread.h), but I see now its included through signal.h :p
There is another strategy though. Your thread will not get stuck without
any good reason, right? So you can implement just enough logic so that
your thread "knows" that it still has a chance to complete its task or
else deallocate any resources held and return. The supervising thread
you suggest will need some similar logic, so it is not about writing
more code but rather moving the code to where the action is going on.
Killing a thread from the outside is fairly simple.
Keeping track of open files/malloc/whatever from the outside is a PITA.
So go for autonomous threads. Your code will be easier to read and
maintain, and therefore also more likely to be correct.
/jens
7:28 a.m.
On ons, 2004-03-31 at 11:02, Jens M Andreasen wrote:
So go for autonomous threads. Your code will be easier
to read and
maintain, and therefore also more likely to be correct.
Just for the record. You probably already know this ...
I forgot to mention the pthread_cleanup_push/pop pair which can make
life somewhat easier when the reason for termination is not known at
compile time.
This of course implies use of pthread_exit instead of return.
Last time I checked they were defined as macros, so gcc -Wall might bark
at you when you are cascading several instances in the same function.
/jens
1:20 p.m.
On Wed, 31 Mar 2004 12:24:31 +0200, Jens M Andreasen
<jens.andreasen(a)chello.se> wrote:
On ons, 2004-03-31 at 11:02, Jens M Andreasen wrote:
> So go for autonomous threads. Your code will be easier to read and
> maintain, and therefore also more likely to be correct.
>
The thing is I don't have complete control over the audio thread, a user
defined callback is involved. In working on the ALSA implementation of the
PortAudio library there has been a request for realtime scheduling of the
audio thread, similar to the older OSS implementation. The OSS
implementation starts both a watchdog and a canary thread; the watchdog
will kill the audio thread (with SIGKILL) if it doesn't respond within
3-second intervals, or lower it's priority if the (normal priority) canary
thread is starved.
Just for the record. You probably already know this
...
I forgot to mention the pthread_cleanup_push/pop pair which can make
life somewhat easier when the reason for termination is not known at
compile time.
This of course implies use of pthread_exit instead of return.
Last time I checked they were defined as macros, so gcc -Wall might bark
at you when you are cascading several instances in the same function.
I love the push/pop combination :] It leads to easy cleanup when the audio
thread exits, I don't think this will be invoked however if I kill it with
SIGKILL however. Maybe a user defined signal, where the thread will issue
pthread_exit in the handling function? Either that or I could perhaps go
for the asynchronous pthread_cancel, and use a different mechanism for
requesting shutdown of the audio thread.
Arve
1:50 p.m.
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
The thing is I don't have complete control over
the audio thread, a
user defined callback is involved. In working on the ALSA
implementation of the PortAudio library there has been a request for
realtime scheduling of the audio thread, similar to the older OSS
implementation. The OSS implementation starts both a watchdog and a
canary thread; the watchdog will kill the audio thread (with SIGKILL)
if it doesn't respond within 3-second intervals, or lower it's
priority if the (normal priority) canary thread is starved.
You can do something like this...
int oldtype;
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &oldtype);
/* invoke the callback */
framesProcessed = PaUtil_EndBufferProcessing( &stream->bufferProcessor,
&callbackResult );
/* restore original canceltype setting */
pthread_setcanceltype(oldtype, NULL);
--
joq
2:23 p.m.
On 31 Mar 2004 10:44:15 -0600, Jack O'Quin <joq(a)io.com> wrote:
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
True .. That was one approach I considered originally while sketching up
solutions, I guess it slipped my mind in the meantime :| I was thinking it
could possibly be an expensive operation though as NPTL sources seem to
indicate, maybe best avoided if memory locks are involved (I'm no
optimization guru, I'm sure you can tell). Anyway, do you think it would
be good to keep a canary around to act on CPU starvation?
Thanks
Arve Knudsen
The thing is I don't have complete control
over the audio thread, a
user defined callback is involved. In working on the ALSA
implementation of the PortAudio library there has been a request for
realtime scheduling of the audio thread, similar to the older OSS
implementation. The OSS implementation starts both a watchdog and a
canary thread; the watchdog will kill the audio thread (with SIGKILL)
if it doesn't respond within 3-second intervals, or lower it's
priority if the (normal priority) canary thread is starved.
You can do something like this...
int oldtype;
pthread_setcanceltype(PTHREAD_CANCEL_ASYNCHRONOUS, &oldtype);
/* invoke the callback */
framesProcessed = PaUtil_EndBufferProcessing( &stream->bufferProcessor,
&callbackResult );
/* restore original canceltype setting */
pthread_setcanceltype(oldtype, NULL);
2:45 p.m.
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
True .. That was one approach I considered originally
while sketching
up solutions, I guess it slipped my mind in the meantime :| I was
thinking it could possibly be an expensive operation though as NPTL
sources seem to indicate, maybe best avoided if memory locks are
involved (I'm no optimization guru, I'm sure you can tell).
It doesn't look all that expensive. The magic is done by a platform-
dependent compare-and-swap operation. On some SMP machines that can
be slow, but generally only in high-contention situations (AFAIK).
Anyway, do you think it would be good to keep a canary
around to act
on CPU starvation?
Personally, I don't see much need for a canary thread. Others may
disagree. But, a watchdog is quite helpful for debugging. In some
cases, the application will provide its own watchdog. Is it possible
for that thread to be optional?
--
joq
3:18 p.m.
On 31 Mar 2004 11:39:36 -0600, Jack O'Quin <joq(a)io.com> wrote:
I can see why people might want a canary (a FIFO
process eating all CPU
leads to a not so responsive system :/), but it would almost certainly be
the client code hogging CPU. Maybe we could introduce a compile time
conditional, and/or an environment variable (default would be on)? Perhaps
the canary could be conditional as well.
Regards
Arve Knudsen
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
According to the
comments it could involve a memory lock though, wouldnt
this be expensive either way?
True .. That was one approach I considered
originally while sketching
up solutions, I guess it slipped my mind in the meantime :| I was
thinking it could possibly be an expensive operation though as NPTL
sources seem to indicate, maybe best avoided if memory locks are
involved (I'm no optimization guru, I'm sure you can tell).
It doesn't look all that expensive. The magic is done by a platform-
dependent compare-and-swap operation. On some SMP machines that can
be slow, but generally only in high-contention situations (AFAIK). Anyway, do you
think it would be good to keep a canary around to act
on CPU starvation?
Personally, I don't see much need for a canary thread. Others may
disagree. But, a watchdog is quite helpful for debugging. In some
cases, the application will provide its own watchdog. Is it possible
for that thread to be optional?
3:57 p.m.
On 31 Mar 2004 11:39:36 -0600, Jack O'Quin
<joq(a)io.com> wrote:
> It doesn't look all that expensive. The magic is done by a platform-
> dependent compare-and-swap operation. On some SMP machines that can
> be slow, but generally only in high-contention situations (AFAIK).
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
According to the comments it could involve a memory
lock though,
wouldnt this be expensive either way?
Dunno. Depends on the machine implementation.
On the SMP machines I worked with a while back, the cost of a compare
and swap was about 15 or 20 cycles (IIRC) when the cache line was
available in the current CPU, but maybe ten times that if there was a
dirty copy in some other CPU. These things tend to scale with memory
and cache latencies more than with CPU speed, so the effect has likely
become even more pronounced.
For realtime, we need to look at the worst-case, so I guess that can
be pretty bad. Still, even the worst case might be acceptable if it
only happens twice per buffer.
Maybe we could introduce a compile time conditional,
and/or an
environment variable (default would be on)? Perhaps the canary could
be conditional as well.
I don't see a compile-time option being much use. PortAudio is a
shared library, its users will generally all run the same binary.
An environment variable is clumsy, but would work. Is there any
mechanism in PA for setting host-dependent options?
--
joq
4:11 p.m.
On 31 Mar 2004 12:50:53 -0600, Jack O'Quin <joq(a)io.com> wrote:
What about pthread_kill with a user defined signal
(SIGRTMIN <= sig <=
SIGRTMAX), where the handler will call pthread_exit? Then the registered
cleanup functions should be invoked as usual.
Of course I could provide
extension functions accepting a PaStream *
instead, I think it would be a better idea. You would have to include
pa_linux_alsa.h. Eventually I'm thinking this stuff should be part of the
common pa_unix utility functions, so an extension for the UNIX
implementations could be introduced.
Regards
Arve Knudsen
On 31 Mar 2004 11:39:36 -0600, Jack O'Quin
<joq(a)io.com> wrote:
> It doesn't look all that expensive. The magic is done by a platform-
> dependent compare-and-swap operation. On some SMP machines that can
> be slow, but generally only in high-contention situations (AFAIK).
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
According to the comments it could involve a
memory lock though,
wouldnt this be expensive either way?
Dunno. Depends on the machine implementation.
On the SMP machines I worked with a while back, the cost of a compare
and swap was about 15 or 20 cycles (IIRC) when the cache line was
available in the current CPU, but maybe ten times that if there was a
dirty copy in some other CPU. These things tend to scale with memory
and cache latencies more than with CPU speed, so the effect has likely
become even more pronounced.
For realtime, we need to look at the worst-case, so I guess that can
be pretty bad. Still, even the worst case might be acceptable if it
only happens twice per buffer. Maybe we could
introduce a compile time conditional, and/or an
environment variable (default would be on)? Perhaps the canary could
be conditional as well.
I don't see a compile-time option being much use. PortAudio is a
shared library, its users will generally all run the same binary.
An environment variable is clumsy, but would work. Is there any
mechanism in PA for setting host-dependent options?
11:27 p.m.
Arve Knudsen <aknuds-1(a)broadpark.no> writes:
What about pthread_kill with a user defined signal
(SIGRTMIN <= sig <=
SIGRTMAX), where the handler will call pthread_exit? Then the
registered cleanup functions should be invoked as usual.
That might be better. I haven't tried doing it that way, myself. For
a watchdog timeout you might want to use SIGABRT. It can be useful to
dump core in these cases.
Of course I could provide extension functions
accepting a PaStream *
instead, I think it would be a better idea. You would have to include
pa_linux_alsa.h. Eventually I'm thinking this stuff should be part of
the common pa_unix utility functions, so an extension for the UNIX
implementations could be introduced.
That sounds fine.
Ross and the other PA developers have thought carefully about what to
make generic and what should be system-dependent, AFAICT.
--
joq
7331
days inactive
7332
days old
linux-audio-dev@lists.linuxaudio.org
12 comments
3 participants
participants (3)
-
Arve Knudsen -
Jack O'Quin -
Jens M Andreasen