[LAD] need help mixing on the fly

[David Griffith]

On Sat, 18 Feb 2017, Fons Adriaensen wrote:

> On Sat, Feb 18, 2017 at 01:44:32AM +0000, David Griffith wrote:
>
>> For a couple years I've been trying, without success, to reliably
>> mix two audio streams on the fly in C.  I'm using libao, libmodplug,
>> libsamplerate, libsndfile, and libvorbis.  The program is Frotz
>> (https://github.com/DavidGriffith/frotz), a Z-machine emulator for
>> playing old Infocom games as well as newer ones.  Could I please get
>> some help with this?
>
> Mixing audio from two audio streams just requires adding the streams
> sample by sample. I'm pretty sure that the real problem here is not
> mixing but something else. But with the limited info you provide we
> can only guess.
>
> Where are the streams coming from ? What are the formats ? Where is
> the result supposed to go ? etc. etc. etc.

The audio comes from AIFF, OGGV, and MOD files which are embedded in an 
IFF container file which also contains game data.  Using libmodplug, 
libsamplerate, libsndfile, and libvorbis; these audio chunks are turned 
into stereo streams of floats.  There are only two streams.  If two 
streams are active, mixing is done.  The resulting single stream of floats 
is converted to pcm16 and fed into libao for output.

The two streams are either "music" or "bleep".  The music stream can be 
fed with audio data from an OGGV chunk or MOD chunk, but not both.  The 
bleep stream is fed from an AIFF chunk.  If a new stream of the same type 
is started, the new one immediately takes over the old one.  This works. 
If both types are played at once, they're supposed to be mixed.  That 
process doesn't go right and I don't know where or why, though I suspect 
trouble with threads, mutexes, and/or semaphores.  The problem manifests 
in distorted sound and usually a segfault.

More detail...

A mixer thread is spawned when Frotz starts up.  It waits for one or both 
float buffers to fill whereupon it mixes their contents, converts the 
result to pcm16 and calls libao to play it.  playaiff() or playmusic() are 
spawned by the main thread as separate threads to read audio data from the 
container IFF and fill up the float buffers for the mixer to read.

The defective code is in 
https://github.com/DavidGriffith/frotz/blob/ao-curses/src/curses/ux_audio.c

I have created a test game to be loaded by Frotz for testing audio 
development.  It's at http://661.org/soundtest2.blb.  The game is a text 
adventure that simulates a single room which contains a Commodore 64, a 
Commodore Amiga, and a small box with buttons on it.  Turning on the C64 
causes an OGGV file to play.  The Amiga plays a MOD file.  The 
buttons on the small box cause AIFFs to play.  To tickle the bug, turn on 
one of the computers and then press a button on the box.

I'm not interested in using SDL or Pulseaudio for this project.


-- 
David Griffith
dave at 661.org

A: Because it fouls the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?