6 Jul
2007
6 Jul
'07
7:57 p.m.
Hi all,
The long title says it all... =) I want to pass some data generated
from real-time audio analysis to the main program (only in that
direction, i.e. only the jack callback writes to the structures). Are
there synchronization issues involved? Can I use a C++ object for this,
or is that an absolute no,no?
I'm virtually a newbie in programming, so any tips, or pointers to code
is very appreciated.
Thanks very much,
Andrés
6 Jul
6 Jul
8:27 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Fri, Jul 06, 2007 at 06:48:54PM -0500, Andres Cabrera wrote:
The long title says it all... =) I want to pass some
data generated
from real-time audio analysis to the main program (only in that
direction, i.e. only the jack callback writes to the structures). Are
there synchronization issues involved? Can I use a C++ object for this,
or is that an absolute no,no?
You can use a C++ object for this (it's the cleanest way IMHO),
but there are some points to watch out for.
* Within the callback you cannot safely allocate and object.
It must exist statically, or be allocated at a safe place,
and pointer given to the callback.
* The main program has to know when it should read the object.
Either use a flag in the object, and let the main program
poll it periodically (if the delay is acceptable), or use
a semaphore. POSIX semas are simple and easy to use, and
you can signal them safely from within the callback.
* The callback has to know when it can (re)write the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
* If you use a flag and polling in both directions make
sure to use two separate flags. One is read-only for
the main program and write-only for the callback and
vice versa.
Things get a bit (but not much) more complex if you need
more than one object, but this should get you started.
--
FA
Follie! Follie! Delirio vano è questo !
8:43 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
Hi Fons,
Thanks for your ideas. This is how I've started to do it but was worried
I was going down the wrong path.
Thanks for the flag tips, I'll do that.
Cheers,
Andrés
Fons Adriaensen wrote:
On Fri, Jul 06, 2007 at 06:48:54PM -0500, Andres
Cabrera wrote:
The long title says it all... =) I want to pass
some data generated
from real-time audio analysis to the main program (only in that
direction, i.e. only the jack callback writes to the structures). Are
there synchronization issues involved? Can I use a C++ object for this,
or is that an absolute no,no?
You can use a C++ object for this (it's the cleanest way IMHO),
but there are some points to watch out for.
* Within the callback you cannot safely allocate and object.
It must exist statically, or be allocated at a safe place,
and pointer given to the callback.
* The main program has to know when it should read the object.
Either use a flag in the object, and let the main program
poll it periodically (if the delay is acceptable), or use
a semaphore. POSIX semas are simple and easy to use, and
you can signal them safely from within the callback.
* The callback has to know when it can (re)write the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
* If you use a flag and polling in both directions make
sure to use two separate flags. One is read-only for
the main program and write-only for the callback and
vice versa.
Things get a bit (but not much) more complex if you need
more than one object, but this should get you started.
8:45 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Sat, 2007-07-07 at 02:34 +0200, Fons Adriaensen wrote:
* The callback has to know when it can (re)write the
object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
Is there any advantage in using a simple flag instead of a semaphore and
sem_trywait()?
--ll
11:13 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On 7/6/07, Fons Adriaensen <fons(a)kokkinizita.net> wrote:
On Fri, Jul 06, 2007 at 06:48:54PM -0500, Andres
Cabrera wrote:
Beware! Flags are not enough to guarantee that you get
a coherent copy of the structure. The safest technique is
using "guard" words at the start and end of the structure.
The process thread updates the guard word at the beginning
to a unique value (time of day, or increment an integer).
Then, it performs the update. Last it copies the first guard
word to the one at the end.
The code in the other thread must copy the entire struct
to another memory area, then check that the guard words
are equal. If not, it loops back and tries the copy again.
After several tries, it should usleep() a while before trying
some more. Something like this (see libjack/transclient.c)...
/* copy a JACK transport position structure (thread-safe) */
void
jack_transport_copy_position (jack_position_t *from, jack_position_t *to)
{
int tries = 0;
long timeout = 1000;
do {
/* throttle the busy wait if we don't get the answer
* very quickly. See comment above about this
* design.
*/
if (tries > 10) {
usleep (20);
tries = 0;
/* debug code to avoid system hangs... */
if (--timeout == 0) {
jack_error ("hung in loop copying position B");
abort();
}
}
*to = *from;
tries++;
} while (to->unique_1 != to->unique_2);
}
This is adequate for normal SMP Intel and AMD machines with strong
memory ordering. On some other platforms, a memory fence instruction
is probably needed after updating the struct.. The spin wait is mainly for
SMP. On a UP it does not help, so you need to sleep.
The reason for doing something comp;icated like this rather than using
a mutex is that there are realtime priority inversion problems if the proces
thread has to wait on a mutex, because the other thread is not running
with SCHED_FIFO priority.
--
joq
The long title says it all... =) I want to pass
some data generated
from real-time audio analysis to the main program (only in that
direction, i.e. only the jack callback writes to the structures). Are
there synchronization issues involved? Can I use a C++ object for this,
or is that an absolute no,no?
You can use a C++ object for this (it's the cleanest way IMHO),
but there are some points to watch out for.
* Within the callback you cannot safely allocate and object.
It must exist statically, or be allocated at a safe place,
and pointer given to the callback.
* The main program has to know when it should read the object.
Either use a flag in the object, and let the main program
poll it periodically (if the delay is acceptable), or use
a semaphore. POSIX semas are simple and easy to use, and
you can signal them safely from within the callback.
* The callback has to know when it can (re)write the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
* If you use a flag and polling in both directions make
sure to use two separate flags. One is read-only for
the main program and write-only for the callback and
vice versa.
Things get a bit (but not much) more complex if you need
more than one object, but this should get you started.
7 Jul
7 Jul
5:18 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Fri, Jul 06, 2007 at 10:13:36PM -0500, Jack O'Quin wrote:
Beware! Flags are not enough to guarantee that you
get
a coherent copy of the structure. The safest technique is
using "guard" words at the start and end of the structure.
...
Depends on how they are used.
If the callback unconditionally updates the data each
time it runs then you are right.
If the system is that ownership of the struct is trans-
ferred between the two parties and only the owner ever
writes, then a simple flag will do.
Ciao,
--
FA
Follie! Follie! Delirio vano è questo !
5:24 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Sat, Jul 07, 2007 at 02:44:31AM +0200, Lars Luthman wrote:
On Sat, 2007-07-07 at 02:34 +0200, Fons Adriaensen
wrote:
It even simpler, and polling may be easier than finding a way to
wait both on the sema and some other condition.
If you never use wait() and always trywait(), then the sema just
is a counter.
--
FA
Follie! Follie! Delirio vano è questo !
* The callback has to know when it can (re)write
the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
Is there any advantage in using a simple flag instead of a semaphore and
sem_trywait()?
5:53 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Fri, Jul 06, 2007 at 07:40:10PM -0500, Andres Cabrera wrote:
Thanks for the flag tips, I'll do that.
One more tip: make sure to declare the flags 'volatile'.
You can't imagine to what troubles g++ will go to avoid
checking them otherwise :-)
--
FA
Follie! Follie! Delirio vano è questo !
6:04 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
Am Samstag, 7. Juli 2007 schrieb Lars Luthman:
On Sat, 2007-07-07 at 02:34 +0200, Fons Adriaensen
wrote:
A semaphore might mislead you to use wait()/lock() in the wrong thread...
And if you only sync two threads and each of them has its own flag (for
write), there shouldn't be any race-conditions and its easier to use a
bool-variable than semaphore->bla().
Arnold
--
visit http://www.arnoldarts.de/
---
Hi, I am a .signature virus. Please copy me into your ~/.signature and send me
to all your contacts.
After a month or so log in as root and do a rm / -rf. Or ask your
administrator to do so...
* The callback has to know when it can (re)write
the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
Is there any advantage in
using a simple flag instead of a semaphore and
sem_trywait()?
6:09 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On Sat, 2007-07-07 at 12:03 +0200, Arnold Krille wrote:
Am Samstag, 7. Juli 2007 schrieb Lars Luthman:
Seriously now, I'm not _that_ stupid. =)
On Sat, 2007-07-07 at 02:34 +0200, Fons
Adriaensen wrote:
A semaphore might mislead you to use wait()/lock() in the wrong thread... * The callback has to know when it can (re)write
the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
Is there any advantage in
using a simple flag instead of a semaphore and
sem_trywait()? And if you only sync two threads and each of them has
its own flag (for
write), there shouldn't be any race-conditions and its easier to use a
bool-variable than semaphore->bla().
I guess so. Can you always be sure that writing to word-sized variables
is atomic on SMP systems?
Although if you only check for 0 or not 0 I guess it doesn't matter.
--ll
6:12 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On 7/7/07, Fons Adriaensen <fons(a)kokkinizita.net> wrote:
On Fri, Jul 06, 2007 at 10:13:36PM -0500, Jack
O'Quin wrote:
And on what platform. Beware of SMP hardware with non-
coherent caches. In general, it is not portable to assume that
stores done by one thread are seen in the same order in
another thread. On most PC-class Intel or AMD systems
that does work. That is called "strong storage ordering".
PowerPC, however, only guarantees "weak storage ordering".
Some NUMA machines my not guarantee any at all. This is
a tradeoff between hardware performance and programming
convenience. Some system designs opt for the former.
Beware! Flags are not enough to guarantee that
you get
a coherent copy of the structure. The safest technique is
using "guard" words at the start and end of the structure.
...
Depends on how they are used. If the callback unconditionally updates the data each
time it runs then you are right.
If the system is that ownership of the struct is trans-
ferred between the two parties and only the owner ever
writes, then a simple flag will do.
The OP specified that "only the jack callback writes to the
structures". My suggestion would not work in the general
case of multiple update threads. For that, you need
some kind of atomic compare-and-swap or increment
operation or a very high resolution timer.
--
joq
6:22 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On 7/7/07, Lars Luthman <lars.luthman(a)gmail.com> wrote:
Mostly yes, assuming the storage location is word-aligned which it
normally will be (due to the compiler).
I vaguely recall that some now old Sun SPARC and Motorola 68K machines
may have presented problems in that area. But, as a practical constraint
it should work OK (AFAIK). It is generally necessary for an SMP implementation
to guarantee coherency on a cache line granularity. Since a word-aligned
word will generally not cross cache lines, you can assume that hardware
will ensure you at least always get either the old or the new value
when looking
at it.
On Sat, 2007-07-07 at 12:03 +0200, Arnold Krille
wrote:
And if you
only sync two threads and each of them has its own flag (for
write), there shouldn't be any race-conditions and its easier to use a
bool-variable than semaphore->bla().
I guess so. Can you always be sure that writing to word-sized variables
is atomic on SMP systems? Although if you only check for 0 or not 0 I guess it
doesn't matter.
The problem with a largish struct is that hardware often does not ensure
that another thread sees only the old or new value. It could see the new value
of the flag but the old value of some of the other fields. If so, you need a
memory fence instruction of some sort.
--
joq
6:25 a.m.
New subject: Best practice for sharing complex data structures with a jack callback function
Am Samstag, 7. Juli 2007 schrieb Lars Luthman:
Well, a flag is either in a bit-field, which I don't recommend, or a bool
variable which should be a char on most (all?) systems. Or you can force that
by using the type char. And that should be atomic.
On Sat, 2007-07-07 at 12:03 +0200, Arnold Krille
wrote:
Yeah, me too. And we will remember that next time we search for some crazy
bug, hard to reproduce, bailing out on jack... and than we realize that
people make errors...
Am Samstag, 7. Juli 2007 schrieb Lars Luthman:
Seriously now, I'm not _that_ stupid. =) On Sat, 2007-07-07 at 02:34 +0200, Fons
Adriaensen wrote:
A semaphore might mislead you to use wait()/lock() in the wrong
thread... * The callback has to know when it can (re)write
the object.
Clearly it should never wait on a sema, and it is already
periodic, so here you can use a flag in the object.
Is there any advantage in
using a simple flag instead of a semaphore
and sem_trywait()? And if you
only sync two threads and each of them has its own flag (for
write), there shouldn't be any race-conditions and its easier to use a
bool-variable than semaphore->bla().
I guess so. Can you always be sure that
writing to word-sized variables
is atomic on SMP systems? Although if you only check for 0 or not 0 I guess it
doesn't matter.
Right. One thread only writes true or false to that variable and the other one
just reads...
Arnold
--
visit http://www.arnoldarts.de/
---
Hi, I am a .signature virus. Please copy me into your ~/.signature and send me
to all your contacts.
After a month or so log in as root and do a rm / -rf. Or ask your
administrator to do so...
9:53 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
Thanks all for the thoughful discussion.
Cheers,
Andrés
Fons Adriaensen wrote:
On Fri, Jul 06, 2007 at 07:40:10PM -0500, Andres
Cabrera wrote:
Thanks for the flag tips, I'll do that.
One more tip: make sure to declare the flags 'volatile'.
You can't imagine to what troubles g++ will go to avoid
checking them otherwise :-)
8 Jul
8 Jul
5:21 p.m.
New subject: Best practice for sharing complex data structures with a jack callback function
On 7/8/07, Andres Cabrera <andres(a)geminiflux.com> wrote:
Thanks all for the thoughful discussion.
I'm late here, but like to add: take a look into jackdmp sources.
It has very clever scheme for lock-free updates of a complex structure.
For explanation of how it works, it's better to ask Stefane Letz.
File/class to look at: JackAtomicState.h
Regards,
Dmitry.
6169
days inactive
6171
days old
linux-audio-dev@lists.linuxaudio.org
14 comments
6 participants
participants (6)
-
Andres Cabrera -
Arnold Krille -
Dmitry Baikov -
Fons Adriaensen -
Jack O'Quin -
Lars Luthman