Re: [LAD] GIl, python, threading, and IPC

I wrote a scripting engine for a pro audio plugin by embedding a CPython interpreter. Since audio sequencing hosts use separate threads for each audio track, loading more than one instance of my plugin while running scripting code causes contention on Python's GIl and results in CPU spikes at low latencies. While CPython is more than capable of running fast enough in the audio thread for control-rate (MIDI) work, the GIL is killing us. Using process migration to move calls to CPython to daemon processes would take less code than forking python itself, but the scripting engine includes a python extension module that exposes pointers to the C++ classes in the audio engine. This complicates things quite a bit. The calls look like this: size = engine.getAudioBufferSize() engine.loadPatchFile(SOME_PATH) instrument.loadAudioFile(SOME_PATH_2) instrument.setVolume(.5) ...where 'engine' and 'instrument' are C++ classes in the audio engine that I wrapped in a python extension. I think that the biggest problem for me is tackling the complexity of managing new real time daemon processes for each audio track, finding an IPC method, and also implementing a middle-ware layer that would allow those synchronous calls to the engine be made back to the host. This is the sort of overhead that one can expect when trying to use process migration to work around the GIL. I consider myself an experienced coder, but just thinking about finding a clean, rock-solid daemon management and IPC mechanism that works on Mac and Windows freaks me out. Does anyone have any comments on this topic? cheers, -P _______________________________________________ Linux-audio-dev mailing list Linux-audio-dev(a)lists.linuxaudio.org http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev