[LAD] Communicating between python UI and C++ engine for real time audio?

I looked into this about five years ago, but didn't get too far. Wondering if anyone on here has experience splitting apps up into: - realtime low latency engine in C++ using per sample callback audio ( either RTAudio or Jack or PortAudio ) - user interfaces and algo composition routines in python that either communicate with the engine by shared memory or over a queue Basically I want to be able to do the gui and data transforming code in Python whenever possible and allow plugins to be written to work on the data in Python I'm hoping to arrive at some sort of design that ultimately lets the engine act as an audio server with multiple user interface clients, possibly not even on the same machines, but definitely not on the same cores. If anyone has tips, war stories, suggestions on reading or projects to look at, I'd love to hear them. Thanks Iain