[LAD] GIl, python, threading, and IPC

[Albert Graef]

David Robillard wrote:
> Python is nowhere even remotely near realtime safe, unless you're doing
> some clever allocation stuff etc.?

I don't know the internals of the CPython interpreter, but I'd say it's
safe to assert that Python does its own memory management. Python should
be perfectly capable of doing control rate processing, for suitable
values of "control rate" of course. :) Python works fine as a control
plugin in Pd, but that's easier since Pd always runs single-threaded.

I agree with the OP's analysis that it's the multithreading in the host
environment which kills it in this case, because CPython's GIL
effectively serializes all processing done in the interpreter. That
means that you can't take advantage of multiple cores and the context
switching overhead becomes prohibitive in a low-latency setting. OTOH,
migrating the different Python threads to different processes, to work
around the GIL, makes it hard to have the audio threads share data
structures with the corresponding Python threads.

I'm not a Python expert, but I see that Python can handle data in shared
memory, maybe that would be a solution? It would still require
synchronization on the shared data, of course, but the rest of the
processing in Python could then be done in a lockfree fashion, no?

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr.Graef at t-online.de, ag at muwiinfa.geschichte.uni-mainz.de
WWW:    http://www.musikinformatik.uni-mainz.de/ag