Re: [linux-audio-user] Audio 3-D Demo --- Any Interest in Software?

Thanks very much to those on the user list who listened to the demo and/or responded regarding 3-D Audio. I really appreciate all of the feedback. I'll try to answer some of the queries and comment on responses in one combined email rather than have a string of individual responses, so this is a little long. However, many of the themes are related, so I'd prefer to answer in this all-in-one manner. On "harshness" (Mark Constable) and "extreme" separation (Jörn) plus Mark's observations of the demo clips: Both the harshness and the extreme separation are adjustable. These effects people noticed aren't a necessary result of the 3-D processing by any means. The separation is exaggerated for this demo. The monophonic clip was included simply to emphasize that I did not merely take the reverberated, stereophonic output of the synth (clips #2 and #4) and "improve" it a little; instead I completely started over with very dull, dry, monophonic recordings (#1 and #5). Cohesiveness (Jan), preprocessing and bus-oriented reverbs (Mark Knecht): The 3-D processing provides a more well-integrated or cohesive sound due to the physical basis of the processing. In using typical DSP- oriented techniques, you are essentially processing the audio in a non-physical manner, despite the terminology "early reflections" and so on. The 3-D processing involves solution of the wave equation in three dimensions, providing a solid physical basis. I have found that far less tweaking is necessary for this approach than with the usual DSP-oriented processing that is the norm. As Mark Knecht wrote, this approach lends itself more to preprocessing, or determining in advance what processing to do, then doing it. The good news is that the result will be closer to something you can use than it would be using the normal mixing/processing approach. The bad news is that the best way to use this new approach is to rethink the whole process of mixing and mastering. ToPlug-In or not to Plug-In: Some people would like to see this implemented as a plugin, but that's putting something new in an old container --- which can be done, but one has to ask if that's really what one wants to do. If so, then I'd be happy to do it, but in the long run, it may be better to rethink the whole process. The mathematical basis for all of this is the solution of the wave equation. Once you've developed methods for doing that in 1-, 2-, and 3-D, you can build a reverber/echoer/stereo-separator OR you can build an SF2 generator OR LOTS of other things. If I were to make these programs available for someone else, how shall I package them? I could build any one of a number of different programs that utilize the routines I've developed, each of which can do completely different things. So rather than speak to developers about how to improve my programs or something like that which was suggested, I really need to speak with potential users about what they might need or want, whether that be a plugin or something completely different. For example, these same programs can also be used to create instruments. (A room can be regarded as part of a three-dimensional instrument.) One could solve the wave equation in two dimensions (drums, cymbals, etc.), in one dimension (guitars, pianos, etc.) or in other geometries (for example pipes --- organs, and so on). On the approach used --- IR?: Mark Knecht asked whether or not this work was IR-based. I assume that this means "impulse response" function based. Well, yes this is how the user would see the application of 3-D processing at the very end of the line, but there is a lot else going on. First one needs to generate the impulse response functions, then generate the impulses, then generate the "recorded" signals. The recorded signals can be decomposed into subcomponents (for example split into frequency bands), then the various impulse response functions can be applied. The programs I've written do all of this, so it's much more than writing a plugin. If I were to merely do that simple part of it, then I'd have to supply some "canned" impulse response functions and transfer some information on how to utilize them properly (or improperly like I do!). I could do this, but I suspect soon enough people would want more information or additional impulse response functions. The "IR" application step is the simplest part of this whole process. On documentation: Jörn asked about whether or not the code was documented so that one could see what was going on. No, it's really not. Some sort of instruction would be necessary, and I'd have to generate that. I'm not aware of anywhere else I could point one to, either. This is rather original work I've done, and the information is scattered about in mathematical physics textbooks, books on acoustics, and in books on signal processing. It is done in C++, so some of the code is in a library --- but some of that may also be of interest. One thing I did was to start completely from scratch. You don't need anything else other than a C++ library to link to. The scripts are Korn shell with a little Python. What next? To summarize a little bit: I can do a lot of different things here, depending upon what people are interested in. I can try to write a plugin that applies impulse response functions that I have generated; I could perhaps make available the programs for producing them; I could write a program that assists in applying them; I could write an instrument generator; I could release a library of the utilities. Or I could just do what Jörn suggested and wrap up what I've already done. I suspect this would be the least useful approach for most people, but the best approach for me and for potential collaborators. Thanks once again for your comments and for listening to the demo. I'd appreciate further discussion, either here or privately by email. I've got a little more on the idea of a plugin and on real-time concerns which I'll send a little later.