On Tue, 28 Jun 2005 21:38:32 +0100 Chris Cannam <cannam(a)all-day-breakfast.com> wrote: Yeah, you definetly are right. I wonder though: Why stop at garanteeing a fixed buffer size for the whole runtime. The thing with the partitioned convolution is that, when used purely as an effect for recorded material (i.e. not playing realtime through it, in a host that can compensate for plugin delay), then large buffers definetly are desirable. Even larger than i.e. the maximum period size of my soundcard (which is 2048 frames). So it would be cool, if the plugin could use a fixed buffer size which also differs from the buffer size used by the underlying audio system (i.e. jack). For this the host would have to do some extra work (setup an extra thread to process the plugin (with a slightly smaller priority than the jack audio callback thread for example, to even the load), ringbuffers to feed/consume data to/from it), latency compensation for tracks sent through it. Or should the plugin do this internally and simply report to the host that it needs a fixed buffer size (which then corresponds to the audio system's buffer size).. Are dssi/ladspa's allowed to do threading? Without i wouldn't know how to do it. And even if it were allowed to do threading, how would the dssi know which priorities to use, etc (on a RP kernel it should have prio higher than i.e. hd and net irq's, but lower than the jack audio thread). Plus i wonder whether the (then fixed) buffer size should be user configurable in any way or would the plugin simply report "16k frames is what i want" :) Sometimes it does make sense to use it in realtime mode (with the same buffer size as the audio system), if you have the cpu power or the responses are short enough. Regards, Flo -- Palimm Palimm! http://affenbande.org/~tapas/

My suggestion is to handle buffering in the convolution plugin and accept any buffer size from the host. I'd do it without threading to ensure the lowest possible latency. For example: assume we run convolution at 512 samples. use a ringbuffer structure (eg like RingBuffer.h in LinuxSampler). http://cvs.linuxsampler.org/cgi-bin/viewcvs.cgi/*checkout*/linuxsampler/src… (the process() in the example below is just a pseudo plugin API , input and output is mono) process(float *input,float *output, int numframes) { if(numframes == 512) { convolve(input, output, 512); return; } ringbuffer->write(input, numframes); while(numframes >0) { if(ringbuffer->read_space() >= 512) { ringbuffer->read(temp_buf, 512); convolve(temp_buf, output, 512); // does convolution and writes to the output array numframes -= 512; } else { write_silence(output,512); numframes -= 512; } } } This approach has the advantage that if the host supplies the convolution plugin with 512 frames then the added latency due to buffering is zero since if(numframes == 512) then it calls convolve() and returns without messing with ringbuffers. Otherwise, with the above approach both the number of frames used in the convolver and number of frames supplied by the host can be completely arbitrary. Drawbacks of the approach: especially on high CPU usage plugins (and convolution IS cpu hungry, especially at low buffer sizes), since the host will run the process() callbacks in RT mode, CPU spikes could introduce xruns and other bad stuff. Assume a 512 frames convolution will take 80% of the CPU on a certain machine. At 44.1kHz 512 frames = 11msec. 80% of 11msec =9.2msec If we run the above code in a host enviroment that uses eg 256 frames (5.5msec buffers), the first time process() is called the >=512 condition is not satisfied and thus a 0 filled buffer is returned (silence). At the second process() call, the >=512 condition is satisfied (there are exactly 512 frames in the buffer). And the convolve() function is called, eating 9.2msec of CPU. Since 9.2msec > 5.5msec ... sh*t happens ... XRUN. If numframes supplied by the host is bigger than 512 then there are no CPU spike problems. For example if the host supplies 1024 frames, the above code would call convolve() 2 times outputting 1024 frames. (eating 2x9.2msec out of the 22msec available) It would be a bit inefficient because if the plugin knows that the host supplies at least 1024 frames then you could run the convolution at 1024 achieving greater efficiency. If the host guarantees that it always supplies the same number of frames then the convolver could adjust it's internal framesize to to achieve optimal CPU usage. If not then a scheme like the above one is unavoidable. Just for curiousity, does anyone know that's the current status of the variable/fixed buffer sizes scenarios supplied to plugins by hosts on various plugin platforms like VST, AU etc ? The above code does some memory bouncing (only when numframes supplied by the host does not match the number of frames used in the convolver): it first copies the input to the ringbuffer's own buffer and then back to temp_buf. So some memory bandwidth is wasted but I think as long as you don't run hundreds of convolution plugins (impossible on today's machines) the added overhead is negligible since convolution is so CPU heavy. I think with an approach like the above you achieve the best of both worlds, no added latency if the host calls the plugin with numframes = power of 2 (matching the internal convolver's buffer size), and some added latency if the host does not use powers of 2. Regarding the CPU spikes, if the convolver uses less than 50% of CPU then you can run the host with the half convolver's numframes without getting XRUNs. eg if the convolver uses 40% CPU at 512 frames then running it in a host with 256 frames then the convolver will still use an average of 40% CPU but it will experience 80% CPU spikes. (eg 80% 0% 80% 0% etc ...). This is not so good because if we want to add an other plugin that has a constant CPU usage of 40% which would lead to an average 80% CPU usage we can't because during the 80% CPU spike we have only 20% of CPU headroom left. Florian, since we would like to add convolution to LinuxSampler over time it would be cool if you could add the above ideas to libconvolve so that one can use the lib without worrying about supplying the right buffer sizes etc, and in plugin hosts enviroments it would be handy too since we don't always know what the host will do. cheers, Benno http://www.linuxsampler.org Florian Schmidt wrote

On Wed, Jun 29, 2005 at 03:43:39PM +0200, Florian Schmidt wrote: There's another subtle bug here in that the plugin assumes that an output buffer of 512 samples is made available by the host, even if the host is using a fixed smaller period size. This is a hidden form of buffering at the output, and in fact, when your plugin algo uses a fixed block size internally but you want to call it with another value, fixed or not, you will always need buffering at both the input and the output. I wrote a convolver library / JACK app similar to Florian's at about the same time (which is why it was never released). Main differences are that the API is a bit more general, it's C++, and it has the required I/O buffering built-in right into the data structures of the convolver engine, so there is no extra overhead in copying. The API it such that the extra delay can be easily avoided if the conditions permit it. -- FA

On Wed, Jun 29, 2005 at 05:45:12PM +0200, Benno Senoner wrote: Yes. In my lib, the two ringbuffers are part of the convolver, and the FFT and MAC operations operate directly on them. The API to write/read them is similar to ALSA's memory mapped buffer interface using *_prepare(), *_pointer() and *_commit() calls. This is AFAICT the most flexible interface you can have to circular buffers, enabling direct access. To disable the added delay when using a period size N equal to the convolver's, just call output_commit(N) once before starting the loop. Diversity is a good thing IMHO, and it's probably not bad to have both C and C++ versions. Both are (or will be in my case) GPL, so they will cross-fertilise anyway. For 'natural' reverb that will be the case. For 'effect' reverb, probably an algorithmic system wil provide much more flexibility and parameters. But you could even combine the two. The main problem with convolution based reverb is to obtain high quality impulse responses. It's not technical, but a matter of organisation, and of obtaining the permission to capture them and make them publicly available. WAVES has a number of very good ones (captured by Angelo Farina), but you first have to buy their reverb software to get access to them, even if most of them were recorded at places sponsored by public money (opera houses and churches). -- FA

On Wed, Jun 29, 2005 at 06:02:24PM +0200, Florian Schmidt wrote: Nothing special. I'd like to add a mode using multiple block sizes for minimal delay. This is not trivial and requires multiple threads as well. Mainly the I/O buffering that is included. Also it's designed to be dynamic, i.e. you can add/remove responses to/from the I/O matrix while it's running. No Xfading yet, but that would be relatively easy to add. (by 'a bit more' I mean just that, not 'much more'.) There are two classes: - Convdata : Contains a partitioned and already FFT'ed impulse response, ready for use by - Convolver : Performs convolution on N inputs to M outputs, defined by an N * M array of Convdata *. Don't !!! And yes, sometimes you would prefer C. -- FA

On Wed, Jun 29, 2005 at 10:20:47PM +0200, Benno Senoner wrote: I don't think this requires special support from the convolution engine - it's jsut an application of it. Suppose you have the start of a reverb response: (imagine the y-scale is logarithmic) |\ | \ | | | | | | --------------------------- and you feed back the output with the correct gain and delay, then the result will be a complete exponential decay. This is too simplistic, and will probably give a 'echo' like character to the reverb, but that's the principle. It should be applied to the first part of the reverb 'tail' only, not the early reflections. But you only need a short convolution. -- FA

Florian Schmidt wrote: yes ! I was about to post it but you were faster ! yes basically the code shoud be: process(float *input,float *output, int numframes) { if(ringbuffer->read_space() == 0 && numframes == 512) { convolve(input, output, 512); return; } ringbuffer->write(input, numframes); while(numframes >0) { if(ringbuffer->read_space() >= 512) { ringbuffer->read(temp_buf, 512); convolve(temp_buf, output, 512); // does convolution and writes to the output array numframes -= 512; } else { write_silence(output,512); numframes -= 512; } } } I agree. Yes CPU spikes are always bad and that's why host and plugin should try to use the same buffer sizes preferably powers of 2 so that FFT based algorithms etc can work optimally. And in the case of VST as you said the host can always pass timestamped events to the plugin to provide sample accurate parameter changes without resorting to odd buffer sizes. My approach above is mainly useful if the host's buffer size is >= the convolution size since it does not introduce threads and offers the lowest possible latency without requiring low latency intra-thread communication. cheers, Benno