[linux-audio-user] Audio 3-D Demo --- Any Interest in Software?

[Jan Depner]

Ron is dead-set against this becoming part of JAMin (which I tend to
agree with just because of the overhead).  I was thinking of it in terms
of a polishing application.  When I was reading up on mastering (trying
to get a handle on it) one of the things that caught my eye was a
suggestion to use an overall reverb on the stereo master to try to
provide a sense of "one room", what I was calling cohesiveness, for all
of the instruments.  It seems that this may be a good way to do that, or
am I way off base here?

Jan


On Mon, 2004-01-05 at 15:49, davidrclark at earthlink.net wrote:
> Thanks very much to those on the user list who listened to the demo 
> and/or responded regarding 3-D Audio.  I really appreciate all of the 
> feedback.  I'll try to answer some of the queries and comment on 
> responses in one combined email rather than have a string of 
> individual responses, so this is a little long.  However, many of the 
> themes are related, so I'd prefer to answer in this all-in-one manner.
> 
> On "harshness" (Mark Constable) and "extreme" separation (Jörn)  plus 
> Mark's observations of the demo clips:
> 
> Both the harshness and the extreme separation are adjustable.  These 
> effects people noticed aren't a necessary result of the 3-D processing 
> by any means.  The separation is exaggerated for this demo.  The 
> monophonic clip was included simply to emphasize that I did not merely 
> take the reverberated, stereophonic output of the synth (clips #2 and 
> #4) and "improve" it a little; instead I completely started over with 
> very dull, dry, monophonic recordings (#1 and #5).
> 
> 
> Cohesiveness (Jan), preprocessing and bus-oriented reverbs (Mark 
> Knecht):
> 
> The 3-D processing provides a more well-integrated or cohesive sound 
> due to the physical basis of the processing.  In using typical DSP-
> oriented techniques, you are essentially processing the audio in a 
> non-physical manner, despite the terminology "early reflections" and 
> so on.  The 3-D processing involves solution of the wave equation in 
> three dimensions, providing a solid physical basis.  I have found that 
> far less tweaking is necessary for this approach than with the usual 
> DSP-oriented processing that is the norm. 
> 
> As Mark Knecht wrote, this approach lends itself more to 
> preprocessing, or determining in advance what processing to do, then 
> doing it.  The good news is that the result will be closer to 
> something you can use than it would be using the normal 
> mixing/processing approach.  The bad news is that the best way to use 
> this new approach is to rethink the whole process of mixing and 
> mastering.
> 
> 
> ToPlug-In or not to Plug-In:
> 
> Some people would like to see this implemented as a plugin, but that's 
> putting something new in an old container --- which can be done, but 
> one has to ask if that's really what one wants to do.  If so, then I'd 
> be happy to do it, but in the long run, it may be better to rethink 
> the whole process.
> 
> The mathematical basis for all of this is the solution of the wave 
> equation.  Once you've developed methods for doing that in 1-, 2-, and 
> 3-D, you can build a reverber/echoer/stereo-separator OR you can build 
> an SF2 generator OR LOTS of other things.  If I were to make these 
> programs available for someone else, how shall I package them?  I 
> could build any one of a number of different programs that utilize the 
> routines I've developed, each of which can do completely different 
> things.  So rather than speak to developers about how to improve my 
> programs or something like that which was suggested, I really need to 
> speak with potential users about what they might need or want, whether 
> that be a plugin or something completely different. 
> 
> For example, these same programs can also be used to create 
> instruments.  (A room can be regarded as part of a three-dimensional 
> instrument.)  One could solve the wave equation in two dimensions 
> (drums, cymbals, etc.), in one dimension (guitars, pianos, etc.) or in 
> other geometries (for example pipes --- organs, and so on).
> 
> 
> On the approach used --- IR?:
> 
> Mark Knecht asked whether or not this work was IR-based.  I assume 
> that this means "impulse response" function based.  Well, yes this is 
> how the user would see the application of 3-D processing at the very 
> end of the line, but there is a lot else going on.  First one needs to 
> generate the impulse response functions, then generate the impulses, 
> then generate the "recorded" signals.  The recorded signals can be 
> decomposed into subcomponents (for example split into frequency 
> bands), then the various impulse response functions can be applied.   
> The programs I've written do all of this, so it's much more than 
> writing a plugin.  If I were to merely do that simple part of it, then 
> I'd have to supply some "canned" impulse response functions and 
> transfer some information on how to utilize them properly (or 
> improperly like I do!).  I could do this, but I suspect soon enough 
> people would want more information or additional impulse response 
> functions.  The "IR" application step is the simplest part of this 
> whole process.
> 
> 
> On documentation: 
> 
> Jörn asked about whether or not the code was documented so that one 
> could see what was going on.  No, it's really not.  Some sort of 
> instruction would be necessary, and I'd have to generate that.  I'm 
> not aware of anywhere else I could point one to, either.  This is 
> rather original work I've done, and the information is scattered about 
> in mathematical physics textbooks, books on acoustics, and in books on 
> signal processing.  It is done in C++, so some of the code is in a 
> library --- but some of that may also be of interest.  One thing I did 
> was to start completely from scratch.  You don't need anything else 
> other than a C++ library to link to.  The scripts are Korn shell with 
> a little Python.
> 
> 
> What next?
> 
> To summarize a little bit: I can do a lot of different things here, 
> depending upon what people are interested in.  I can try to write a 
> plugin that applies impulse response functions that I have generated; 
> I could perhaps make available the programs for producing them; I 
> could write a program that assists in applying them; I could write an 
> instrument generator; I could release a library of the utilities.  Or 
> I could just do what Jörn suggested and wrap up what I've already 
> done.  I suspect this would be the least useful approach for most 
> people, but the best approach for me and for potential collaborators.
> 
> Thanks once again for your comments and for listening to the demo.  
> I'd appreciate further discussion, either here or privately by email.   
> I've got a little more on the idea of a plugin and on real-time 
> concerns which I'll send a little later.
> 
> 
>