26 Aug
2008
26 Aug
'08
3:42 p.m.
Hello all,
Does anyone know of a counting semaphare class/module
in Python ? Given the lock provided by the built-in
thread module it seems impossible to implement this
(it does support multiple waiters which I don't need,
but definitely is not counting). This also means that
whatever is defined in the threading module can't be
what I want.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
O tu, che porte, correndo si ?
E guerra e morte !
26 Aug
26 Aug
3:55 p.m.
Am Dienstag, 26. August 2008 schrieb Fons Adriaensen:
Does anyone know of a counting semaphare class/module
in Python ? Given the lock provided by the built-in
thread module it seems impossible to implement this
(it does support multiple waiters which I don't need,
but definitely is not counting). This also means that
whatever is defined in the threading module can't be
what I want.
Did you look at the threading module? Looks as if there a Semaphore() in
there...
(Not as off-topic as one might think. I am currently playing around with
python too. Using gstreamer and py-gst to display videos from
still-images...)
Arnold
--
visit http://www.arnoldarts.de/
---
Hi, I am a .signature virus. Please copy me into your ~/.signature and send me
to all your contacts.
After a month or so log in as root and do a "rm -rf /". Or ask your
administrator to do so...
4:18 p.m.
On Tue, Aug 26, 2008 at 09:55:07PM +0200, Arnold Krille wrote:
Did you look two lines above yours :-) :-) :-)
Yes, of course I did. It defines a Semaphore on top of
a Condition which is pure ugly hackery on top of a RLock.
Can't be correct, since AFAICS the primitives provided
by thread are not sufficient. Apart from the fact that
a sema requiring a few hundreds of lines of high-level
code can't be healthy.
The basic problem is that the lock provided by thread
is not counting: releasing it while it is not acquired
produces and error. Of course one could create a class
that contains a counter, but then the problem is to
atomically modify the counter and access the lock.
AFAICS, for this reason, whatever happens in threading
can't be correct, or at least not pre-emption and SMP
safe.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
O tu, che porte, correndo si ?
E guerra e morte !
Am Dienstag, 26. August 2008 schrieb Fons Adriaensen:
Does anyone
know of a counting semaphare class/module
in Python ? Given the lock provided by the built-in
thread module it seems impossible to implement this
(it does support multiple waiters which I don't need,
but definitely is not counting). This also means that
whatever is defined in the threading module can't be
what I want.
Did you look at the threading module? Looks as if there a Semaphore() in
there... 
5:04 p.m.
On Tuesday 26 August 2008 22:01:18 Fons Adriaensen wrote:
AFAICS, for this reason, whatever happens in
threading
can't be correct, or at least not pre-emption and SMP
safe.
Is this really an issue in Python? The Python interpreter is not thread safe
anyway, there's a global interpreter lock that must be held by any thread
accessing Python objects.
Depending on what you're trying to do, this might be one of Python's biggest
disadvantages, but as far as I can see, you don't need to worry about
"true"
concurrency, simply because it can't happen.
Dominic
5:31 p.m.
On Tue, Aug 26, 2008 at 11:02:37PM +0200, Dominic Sacré wrote:
Is this really an issue in Python? The Python
interpreter is not thread safe
anyway, there's a global interpreter lock that must be held by any thread
accessing Python objects.
Depending on what you're trying to do, this might be one of Python's biggest
disadvantages, but as far as I can see, you don't need to worry about
"true"
concurrency, simply because it can't happen.
At first it would seem that solves the problem.
But AFAICS it doesn't.
A condition variable consists of
- one or more state variables, on wich a condition is defined,
- a mutex M to protect these variables from concurrent acceess,
- a binary sema S (or equivalent) on which to wait until the
condition is satisfied.
To make all of this work correctly in all conditions, the
essential part is that one can do [wait (S), release (M)]
atomically.
A counting sema is just one special case of this, the
condition being _count > 0, where _count is an integer
and the only state variable.
The lock provided by the built-in thread module can play
the role of both M and S, and that is indeed how Condition
in threading is implemented (it's a bit more complicated,
supporting multiple waiters, but that doesn't change the
basic way it works).
BUT: the python interpreter will release the GIL every
so many bytecode instructions. There is AFAICS nothing
that would prevent it from doing this in between the two
operations that have to be atomic - it doesn't know they
have to be.
Ciao,
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
O tu, che porte, correndo si ?
E guerra e morte !
5:55 p.m.
On Tue, Aug 26, 2008 at 11:14:05PM +0200, Fons Adriaensen wrote:
To make all of this work correctly in all conditions,
the
essential part is that one can do [wait (S), release (M)]
atomically.
And of course also [return from wait (S), acquire (M)]
--
FA
Laboratorio di Acustica ed Elettroacustica
Parma, Italia
O tu, che porte, correndo si ?
E guerra e morte !
8:53 p.m.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Fons Adriaensen wrote:
On Tue, Aug 26, 2008 at 11:02:37PM +0200, Dominic
Sacr� wrote:
I was going to suggest the twisted framework, but according to
http://mail.python.org/pipermail/python-list/2007-October/461199.html
it won't help. You can try your luck with an "extension module (written
in C) using the macros Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS to
release/acquire the GIL before/after an external blocking call".
rewrite in /perl/ is probably faster: There's a thread-safe
Thread::Semaphore :-X
#robin
Is this really an issue in Python? The Python
interpreter is not thread safe
anyway, there's a global interpreter lock that must be held by any thread
accessing Python objects.
Depending on what you're trying to do, this might be one of Python's biggest
disadvantages, but as far as I can see, you don't need to worry about
"true"
concurrency, simply because it can't happen.
At first it would seem that solves the problem.
But AFAICS it doesn't.
A condition variable consists of
- one or more state variables, on wich a condition is defined,
- a mutex M to protect these variables from concurrent acceess,
- a binary sema S (or equivalent) on which to wait until the
condition is satisfied.
To make all of this work correctly in all conditions, the
essential part is that one can do [wait (S), release (M)]
atomically.
A counting sema is just one special case of this, the
condition being _count > 0, where _count is an integer
and the only state variable.
The lock provided by the built-in thread module can play
the role of both M and S, and that is indeed how Condition
in threading is implemented (it's a bit more complicated,
supporting multiple waiters, but that doesn't change the
basic way it works).
BUT: the python interpreter will release the GIL every
so many bytecode instructions. There is AFAICS nothing
that would prevent it from doing this in between the two
operations that have to be atomic - it doesn't know they
have to be. -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAki0pV4ACgkQeVUk8U+VK0K4BwCfRmVcznGqScJ4ixbU6qOIoReo
+lwAoIc/b2F/mRmeL3PmJ7pRgP7xBq/H
=3hjS
-----END PGP SIGNATURE-----
27 Aug
27 Aug
10:35 a.m.
2008/8/26 Fons Adriaensen <fons(a)kokkinizita.net>et>:
BUT: the python interpreter will release the GIL every
so many bytecode instructions. There is AFAICS nothing
that would prevent it from doing this in between the two
operations that have to be atomic - it doesn't know they
have to be.
Maybe this will bring you further :
PyGILState_STATE gil_state
gil_state = PyGILState_Ensure()
// do anything
PyGILState_Release( gil_state )
see also:
http://docs.python.org/api/threads.html
12:10 p.m.
Hi,
On Tue, Aug 26, 2008 at 11:14:05PM +0200, Fons Adriaensen wrote:
On Tue, Aug 26, 2008 at 11:02:37PM +0200, Dominic
Sacré wrote:
It sounds like you're suggesting that the threading module is broken. You'd be
best off taking this to the Python mailing list, I suspect, so that folks that
*really* know Python can either confirm the bug, or explain what you are
missing.
-Forest
--
Forest Bond
http://www.alittletooquiet.net
http://www.pytagsfs.org
Is this really an issue in Python? The Python
interpreter is not thread safe
anyway, there's a global interpreter lock that must be held by any thread
accessing Python objects.
Depending on what you're trying to do, this might be one of Python's biggest
disadvantages, but as far as I can see, you don't need to worry about
"true"
concurrency, simply because it can't happen.
At first it would seem that solves the problem.
But AFAICS it doesn't.
A condition variable consists of
- one or more state variables, on wich a condition is defined,
- a mutex M to protect these variables from concurrent acceess,
- a binary sema S (or equivalent) on which to wait until the
condition is satisfied.
To make all of this work correctly in all conditions, the
essential part is that one can do [wait (S), release (M)]
atomically.
A counting sema is just one special case of this, the
condition being _count > 0, where _count is an integer
and the only state variable.
The lock provided by the built-in thread module can play
the role of both M and S, and that is indeed how Condition
in threading is implemented (it's a bit more complicated,
supporting multiple waiters, but that doesn't change the
basic way it works).
BUT: the python interpreter will release the GIL every
so many bytecode instructions. There is AFAICS nothing
that would prevent it from doing this in between the two
operations that have to be atomic - it doesn't know they
have to be. 
26 Aug
26 Aug
4:22 p.m.
[Fons Adriaensen]
Does anyone know of a counting semaphare class/module
in Python ? Given the lock provided by the built-in
thread module it seems impossible to implement this
(it does support multiple waiters which I don't need,
but definitely is not counting). This also means that
whatever is defined in the threading module can't be
what I want.
Perhaps I don't quite understand your needs, but the standard
Semaphore object in the threading module claims to support counting:
http://docs.python.org/lib/semaphore-objects.html
I haven't used it myself yet but the Python docs have never promised
me anything that didn't work as described (yay!). I guess I must be
missing something.
Cheers, Tim
5972
days inactive
5973
days old
linux-audio-dev@lists.linuxaudio.org
9 comments
7 participants
participants (7)
-
Arnold Krille -
Dominic Sacré -
Emanuel Rumpf -
Fons Adriaensen -
Forest Bond -
Robin Gareus -
Tim Goetze