[linux-audio-user] Audio 3-D Demo --- Any Interest in Software?
davidrclark at earthlink.net
davidrclark at earthlink.net
Mon Jan 5 16:49:41 EST 2004
Thanks very much to those on the user list who listened to the demo
and/or responded regarding 3-D Audio. I really appreciate all of the
feedback. I'll try to answer some of the queries and comment on
responses in one combined email rather than have a string of
individual responses, so this is a little long. However, many of the
themes are related, so I'd prefer to answer in this all-in-one manner.
On "harshness" (Mark Constable) and "extreme" separation (Jörn) plus
Mark's observations of the demo clips:
Both the harshness and the extreme separation are adjustable. These
effects people noticed aren't a necessary result of the 3-D processing
by any means. The separation is exaggerated for this demo. The
monophonic clip was included simply to emphasize that I did not merely
take the reverberated, stereophonic output of the synth (clips #2 and
#4) and "improve" it a little; instead I completely started over with
very dull, dry, monophonic recordings (#1 and #5).
Cohesiveness (Jan), preprocessing and bus-oriented reverbs (Mark
The 3-D processing provides a more well-integrated or cohesive sound
due to the physical basis of the processing. In using typical DSP-
oriented techniques, you are essentially processing the audio in a
non-physical manner, despite the terminology "early reflections" and
so on. The 3-D processing involves solution of the wave equation in
three dimensions, providing a solid physical basis. I have found that
far less tweaking is necessary for this approach than with the usual
DSP-oriented processing that is the norm.
As Mark Knecht wrote, this approach lends itself more to
preprocessing, or determining in advance what processing to do, then
doing it. The good news is that the result will be closer to
something you can use than it would be using the normal
mixing/processing approach. The bad news is that the best way to use
this new approach is to rethink the whole process of mixing and
ToPlug-In or not to Plug-In:
Some people would like to see this implemented as a plugin, but that's
putting something new in an old container --- which can be done, but
one has to ask if that's really what one wants to do. If so, then I'd
be happy to do it, but in the long run, it may be better to rethink
the whole process.
The mathematical basis for all of this is the solution of the wave
equation. Once you've developed methods for doing that in 1-, 2-, and
3-D, you can build a reverber/echoer/stereo-separator OR you can build
an SF2 generator OR LOTS of other things. If I were to make these
programs available for someone else, how shall I package them? I
could build any one of a number of different programs that utilize the
routines I've developed, each of which can do completely different
things. So rather than speak to developers about how to improve my
programs or something like that which was suggested, I really need to
speak with potential users about what they might need or want, whether
that be a plugin or something completely different.
For example, these same programs can also be used to create
instruments. (A room can be regarded as part of a three-dimensional
instrument.) One could solve the wave equation in two dimensions
(drums, cymbals, etc.), in one dimension (guitars, pianos, etc.) or in
other geometries (for example pipes --- organs, and so on).
On the approach used --- IR?:
Mark Knecht asked whether or not this work was IR-based. I assume
that this means "impulse response" function based. Well, yes this is
how the user would see the application of 3-D processing at the very
end of the line, but there is a lot else going on. First one needs to
generate the impulse response functions, then generate the impulses,
then generate the "recorded" signals. The recorded signals can be
decomposed into subcomponents (for example split into frequency
bands), then the various impulse response functions can be applied.
The programs I've written do all of this, so it's much more than
writing a plugin. If I were to merely do that simple part of it, then
I'd have to supply some "canned" impulse response functions and
transfer some information on how to utilize them properly (or
improperly like I do!). I could do this, but I suspect soon enough
people would want more information or additional impulse response
functions. The "IR" application step is the simplest part of this
Jörn asked about whether or not the code was documented so that one
could see what was going on. No, it's really not. Some sort of
instruction would be necessary, and I'd have to generate that. I'm
not aware of anywhere else I could point one to, either. This is
rather original work I've done, and the information is scattered about
in mathematical physics textbooks, books on acoustics, and in books on
signal processing. It is done in C++, so some of the code is in a
library --- but some of that may also be of interest. One thing I did
was to start completely from scratch. You don't need anything else
other than a C++ library to link to. The scripts are Korn shell with
a little Python.
To summarize a little bit: I can do a lot of different things here,
depending upon what people are interested in. I can try to write a
plugin that applies impulse response functions that I have generated;
I could perhaps make available the programs for producing them; I
could write a program that assists in applying them; I could write an
instrument generator; I could release a library of the utilities. Or
I could just do what Jörn suggested and wrap up what I've already
done. I suspect this would be the least useful approach for most
people, but the best approach for me and for potential collaborators.
Thanks once again for your comments and for listening to the demo.
I'd appreciate further discussion, either here or privately by email.
I've got a little more on the idea of a plugin and on real-time
concerns which I'll send a little later.
More information about the Linux-audio-user