[linux-audio-user] Audio 3-D Demo --- Any Interest in Software?

[davidrclark at earthlink.net]

Thanks very much to those on the user list who listened to the demo 
and/or responded regarding 3-D Audio.  I really appreciate all of the 
feedback.  I'll try to answer some of the queries and comment on 
responses in one combined email rather than have a string of 
individual responses, so this is a little long.  However, many of the 
themes are related, so I'd prefer to answer in this all-in-one manner.

On "harshness" (Mark Constable) and "extreme" separation (Jörn)  plus 
Mark's observations of the demo clips:

Both the harshness and the extreme separation are adjustable.  These 
effects people noticed aren't a necessary result of the 3-D processing 
by any means.  The separation is exaggerated for this demo.  The 
monophonic clip was included simply to emphasize that I did not merely 
take the reverberated, stereophonic output of the synth (clips #2 and 
#4) and "improve" it a little; instead I completely started over with 
very dull, dry, monophonic recordings (#1 and #5).


Cohesiveness (Jan), preprocessing and bus-oriented reverbs (Mark 
Knecht):

The 3-D processing provides a more well-integrated or cohesive sound 
due to the physical basis of the processing.  In using typical DSP-
oriented techniques, you are essentially processing the audio in a 
non-physical manner, despite the terminology "early reflections" and 
so on.  The 3-D processing involves solution of the wave equation in 
three dimensions, providing a solid physical basis.  I have found that 
far less tweaking is necessary for this approach than with the usual 
DSP-oriented processing that is the norm. 

As Mark Knecht wrote, this approach lends itself more to 
preprocessing, or determining in advance what processing to do, then 
doing it.  The good news is that the result will be closer to 
something you can use than it would be using the normal 
mixing/processing approach.  The bad news is that the best way to use 
this new approach is to rethink the whole process of mixing and 
mastering.


ToPlug-In or not to Plug-In:

Some people would like to see this implemented as a plugin, but that's 
putting something new in an old container --- which can be done, but 
one has to ask if that's really what one wants to do.  If so, then I'd 
be happy to do it, but in the long run, it may be better to rethink 
the whole process.

The mathematical basis for all of this is the solution of the wave 
equation.  Once you've developed methods for doing that in 1-, 2-, and 
3-D, you can build a reverber/echoer/stereo-separator OR you can build 
an SF2 generator OR LOTS of other things.  If I were to make these 
programs available for someone else, how shall I package them?  I 
could build any one of a number of different programs that utilize the 
routines I've developed, each of which can do completely different 
things.  So rather than speak to developers about how to improve my 
programs or something like that which was suggested, I really need to 
speak with potential users about what they might need or want, whether 
that be a plugin or something completely different. 

For example, these same programs can also be used to create 
instruments.  (A room can be regarded as part of a three-dimensional 
instrument.)  One could solve the wave equation in two dimensions 
(drums, cymbals, etc.), in one dimension (guitars, pianos, etc.) or in 
other geometries (for example pipes --- organs, and so on).


On the approach used --- IR?:

Mark Knecht asked whether or not this work was IR-based.  I assume 
that this means "impulse response" function based.  Well, yes this is 
how the user would see the application of 3-D processing at the very 
end of the line, but there is a lot else going on.  First one needs to 
generate the impulse response functions, then generate the impulses, 
then generate the "recorded" signals.  The recorded signals can be 
decomposed into subcomponents (for example split into frequency 
bands), then the various impulse response functions can be applied.   
The programs I've written do all of this, so it's much more than 
writing a plugin.  If I were to merely do that simple part of it, then 
I'd have to supply some "canned" impulse response functions and 
transfer some information on how to utilize them properly (or 
improperly like I do!).  I could do this, but I suspect soon enough 
people would want more information or additional impulse response 
functions.  The "IR" application step is the simplest part of this 
whole process.


On documentation: 

Jörn asked about whether or not the code was documented so that one 
could see what was going on.  No, it's really not.  Some sort of 
instruction would be necessary, and I'd have to generate that.  I'm 
not aware of anywhere else I could point one to, either.  This is 
rather original work I've done, and the information is scattered about 
in mathematical physics textbooks, books on acoustics, and in books on 
signal processing.  It is done in C++, so some of the code is in a 
library --- but some of that may also be of interest.  One thing I did 
was to start completely from scratch.  You don't need anything else 
other than a C++ library to link to.  The scripts are Korn shell with 
a little Python.


What next?

To summarize a little bit: I can do a lot of different things here, 
depending upon what people are interested in.  I can try to write a 
plugin that applies impulse response functions that I have generated; 
I could perhaps make available the programs for producing them; I 
could write a program that assists in applying them; I could write an 
instrument generator; I could release a library of the utilities.  Or 
I could just do what Jörn suggested and wrap up what I've already 
done.  I suspect this would be the least useful approach for most 
people, but the best approach for me and for potential collaborators.

Thanks once again for your comments and for listening to the demo.  
I'd appreciate further discussion, either here or privately by email.   
I've got a little more on the idea of a plugin and on real-time 
concerns which I'll send a little later.