[LAD] Realtime and helper threads

[Dan Mills]

On Sat, 2008-08-23 at 14:56 +0200, Olivier Guilyardi wrote:
> Hi Dan,
> 
> why don't you answer on the mailing list?

I thought I had!


> > That is a fairly common approach, one thread per core is good. I have
> > code that reads audio, sample rate converts it, time-stretches it if
> > appropriate and sticks it in a ring buffer. The realtime thread does not
> > pay any attention to the transport at all, it just handles the realtime
> > controls (some mixing and routing in my case).
> 
> I am unsure what you mean by "core".

Processor core on the machine, having multiple threads will allow more
then one core to be utilised if required.


> > One thing you do want to watch is that the disk IO threads should
> > probably run at a realtime priority (just lower then the one for the
> > audio thread), otherwise a make -j10 on the kernel can result in
> > ringbuffer under run.  
> 
> Disk IO is another problem to me. I would prefer to focus on computation load
> here, that is: helper threads that performs audio conversion/transformation,
> etc.... I assume that the thread priority you advise also concerns this though.

It would, in my app it is basically disk io that feeds the system but in
your case it seems to be something else, but the same principle applies.

> > 
> > If you need to be able to vary the timestrtch in real time with a file
> > playing, any ring buffer after the timestretcher will need to be short,
> > otherwise it is fairly straightforward. 
> 
> That worries me a bit. If my ringbuffer is really short, for responsive user
> control, and the time stretching performed in a non-RT thread, isn't this going
> to make things worse?

I was advocating having this done in a RT thread at lower priority then
the audio thread. 

How much control latency can you tolerate? That number will tell you
what is possible in terms of approach, quantify the problem.


> What about the following idea:
> 
> Say I make a Converter object, which performs time stretching. It doesn't know
> anything about threading.
> 
> I also have a Reader object, with an internal ringbuffer, it reads raw audio
> data, converts it using the Converter, and fills its ringbuffer with the
> converted content.
> 
> The Reader objects exposes read(double time_ratio, int nframes) to the realtime
> thread. When this function gets called, the Reader checks if it has data time
> stretched according to this time ratio in its internal ringbuffer. If it does it
> simply returns it. Otherwise, it calls the Converter on the fly, discarding the
> irrelevant data in the ringbuffer, and causing a sudden small overhead.

Then you are no longer realtime as Linux audio understands the concept.

> Once the time ratio has stopped varying (the user has stopped playing with the
> knob), the ringbuffer rapidly gets filled with data at the right time ratio, and
> the overhead vanishes.

It still does not solve your problem as clicks and pops can happen while
the user is playing with the knob. 

Regards, Dan.