[linux-audio-user] Re: Boa-Conductor (was re: Raton)

[Peter Brinkmann]

David,
It's been a while, but I finally got around to thinking about how I would
properly implement something like MagicBaton in Python, and I've got a
prototype that essentially works the way you describe.

If you're interested, here's how to install and run the new code:
    - Install the latest version of MidiKinesis, available at
        http://www.math.tu-berlin.de/~brinkman/software/midikinesis/
    Once you've unpacked the tarball, say 'make' in the midikinesis
    directory.
    - Install the Python Midi package. There's a link on the MidiKinesis
    page. I guess the easiest (but not the nicest) way to make this work
    is to simply dump the contents of the Python Midi package in the
    midikinesis directory.
    - Save the attached boaconductor.py in the midikinesis directory.

Now find a MIDI file that you want to experiment with. I like Matthias
Nagorni's MIDI files, available at
http://sourceforge.net/project/showfiles.php?group_id=69130&package_id=121417

Just to make sure that your setup works, see whether you can play a MIDI
files with the midiplayer.py, by saying
    python midiplayer.py file.mid
After starting this program, you need to hook up the MIDI output ports of
midiplayer.py to some synthesizer(s). I simply plug it into my digital piano.
When that's done, hit return, and you should hear the piece.

Now, to experiment with BoaConductor, you essentially do the same thing:
    python boaconductor.py file.mid
Once again you need to hook up the MIDI output ports to some synthesizer(s),
and then you can start directing by tracing out circles in the GUI of
BoaConductor.

Since this is just a proof of concept at this point, I have spent a lot
of time on the user interface. In particular, right now any configuration
has to be done in the code itself. Here are the parameters you may want
to play with:
    - self.gain: This one lets you increase or decrease the number of
    quarters that one full revolution of the mouse corresponds to.
    - self.tempo and self.volume: These are objects that use an
    averaging process of sorts in order to remove the wobble that
    imprecise mouse motions tend to cause. They have some parameters
    that can be tweaked in order to make them more or less responsive.
    I put a comment in the code to this effect. (In case you're
    mathematically inclined, I'm using Kalman filters in order to
    remove wobble without incurring too much latency.)

I'm also generating MIDI Master Volume messages based on the average
radius of the circles you're drawing, but my piano seems to ignore
them. It would be interesting to know whether they work on your
side.

I'd be interested to hear how you like the new gadget. As I said,
it's very rough around the edges at the moment, but I think it shows
that the general idea works. I'm also having a lot of fun with it.
Any thoughts would be appreciated!
Best,
    Peter



On Thu, Jan 20, 2005 at 09:39:07AM +0200, David Baron wrote:
> 
> On Wednesday 19 January 2005 19:02, 
> linux-audio-user-request at music.columbia.edu wrote:
> > > So how would Raton-Conductor work? Again, not as simple as it sounds.
> > > Minimally, one would move the mouse in a ecliptic (or circular) motion
> > > (as suggested by MagicBaton's instructions for beginners). The size of
> > > the vertical diameter (or average diameter) would be the
> > > volume/expression and tempo or time-codes set by the change in vertical
> > > direction from down to up. Real conducting patterns are more complex but
> > > these two principals would more or less remain.
> >
> > I started tinkering after I read your message, and I created a little
> > gadget that traces mouse motions and recognizes fairly general conducting
> > patterns. One can extract timing and intensity information for the purpose
> > of generating MIDI clock events as well as MIDI controller values. I'm
> > tentatively calling it Boa Conductor.
> 
> Cool!
> >
> > I've got a few questions:
> >     1. Is anyone interested in a tool like Boa Conductor? What I've done so
> >     far was just for kicks; I now have to decide how much time and effort
> >     to put into polishing it.
> 
> I, of course, would be interested.
> 
> >     2. Are there any MIDI sequencers/players for Linux that can be driven
> >     by external clock messages? My understanding is that Rosegarden does
> >     not currently work as a slave but that may change in the future.
> >     MusE can work as a slave, but I never used it before (has anyone
> >     tried driving MusE with clock messages?). I don't think timidity
> >     expects to be driven by a MIDI clock. How about other MIDI players?
> >     3. Would it make sense to have a feature that uses JACK Transport
> >     rather than MIDI clock?
> 
> There are a few alternatives. Not much that I have on Windows or Linux support 
> clock messages. MagicBaton was a MIDI-player that added events based on the 
> mouse-conducting.
> 
> Alternatives:
> 
> 1. Run the Boa in Parallel with whatever sequencer or player is going through 
> jack or such. Boa would then put out simply omni/overall level and tempo 
> events.
> 
> 2. Run Boa as a plug-in Rosegarten, Muse or other such program. In this case, 
> it would work on one track/channel and insert expression (and tempos) or in 
> an omni mode as above. For rehearsing (MagitBaton's parlance) one track, one 
> would probably want to disable tempo changes and conduct expression. 
> Omin/overall would do level and tempo.
> 
> 3. Controlling another software via midi-clock and some volume control device. 
> This assume, naturally, that this software is available :-)
> 
> 

-- 
-------------- next part --------------
#!/usr/bin/python
# $Id: boaconductor.py,v 1.8 2005/06/19 17:57:52 brinkman Exp $
# Peter Brinkmann (brinkman at math.tu-berlin.de)

from Tkinter import *
from math import *
import time
from pyseq import *
from midiplayer import *

class SimpleKalman:
    def __init__(self, p, q, r):
        self.p=p
        self.q=q
        self.r=r
        self.x=None
    def next(self, x):
        if self.x is None:
            self.x=x
            return x
        pm=self.p+self.q
        k=pm/(pm+self.r)
        self.x+=k*(x-self.x)
        self.p=(1-k)*pm
        return self.x


class BoaConductor(Frame):
    def __init__(self, tune, parent):
        self.seq=PlaySeq(tune)
        self.ev=snd_seq_event()
        Frame.__init__(self, parent)
        self.pack(fill=X, expand=YES)
        self.master.title('BoaConductor')
        self.height=self.width=500
        self.xc, self.yc=self.width/2, self.height/2
        self.x=self.y=None
        self.gain=1
        self.phi0=4*pi
        self.callhandle=None
        self.paused=0
        self.start()
        self.cv=Canvas(self, width=self.width, height=self.height,
             relief=SUNKEN, borderwidth=2)
        self.cv.pack()
        Button(self, text='Restart', command=self.start).pack()
        Button(self, text='Quit', command=self.cleanup).pack()
        self.cv.bind('<Enter>', self.handleEntrance)
        self.cv.bind('<Motion>', self.handleMotion)
        self.cv.create_line(self.xc, 0, self.xc, self.height)
        self.cv.create_line(0, self.yc, self.width, self.yc)
    def cleanup(self):
        self.seq.handleSignal()
        self.quit()
    def handleEntrance(self, e):
        self.x, self.y=e.x, e.y
    def setTempoMTV(self, omega):
        self.seq.q.setMidiTempoValue(int(2e6*pi/omega/self.gain))
    def setVolume(self, d):
        vol=min(16384, int(d/(.7*self.width)*32768))
        for p in self.seq.oports:
            self.ev.setMasterVolume(vol)
            self.seq.q.postEvent(self.ev, p, 0)
    def start(self):
        if not self.callhandle is None:
            self.after_cancel(self.callhandle)
            self.callhandle=None
        self.seq.q.stop()
        self.seq.panic()
        self.t=None
        self.phi=0
        self.paused=0
        self.tempo=SimpleKalman(1, 1e-5, .03)  # the smaller the last value,
                        # the more responsive the behavior (beware of wobble)
        self.volume=SimpleKalman(1, 1e-5, .01)
    def startQueue(self):
        self.seq.play()
    def angle(self, x, y):
        return atan2(y-self.yc, x-self.xc)
    def distance(self, x, y):
        return sqrt((y-self.yc)**2+(x-self.xc)**2)
    def angleDiff(self, x0, y0, x1, y1):
        phi0, phi1=self.angle(x0, y0), self.angle(x1, y1)
        return min(abs(phi1-phi0), abs(phi1-phi0+2*pi), abs(phi1-phi0-2*pi))
    def handleMotion(self, e):
        if not self.callhandle is None:
            self.after_cancel(self.callhandle)
        t=time.time()
        if self.t is None:
            self.t=t
            self.x, self.y=e.x, e.y
            return
        dphi=self.angleDiff(e.x, e.y, self.x, self.y)
        if self.phi<self.phi0 and self.phi+dphi>self.phi0:
            self.startQueue()
        dt=t-self.t
        self.setTempoMTV(self.tempo.next(dphi/dt))
        self.setVolume(self.volume.next(self.distance(e.x, e.y)))
        if self.paused:
            self.paused=0
            self.seq.q.cont()
        self.x, self.y=e.x, e.y
        self.t=t
        self.phi+=dphi
        self.after(1000, self.cv.delete,
            self.cv.create_oval(e.x-2, e.y-2, e.x+2, e.y+2, fill='red'))
        self.callhandle=self.after(250, self.pause)
    def pause(self):
        self.callhandle=None
        self.seq.q.stop()
        self.paused=1
 

if __name__=='__main__':
    if len(sys.argv)<2:
        fn='trio_es_2_nagorni.mid'
    else:
        fn=sys.argv[1]
    print 'hook up midi ports, draw circles'
    tune=MidiToAlsa(fn)
    BoaConductor(tune, Tk()).mainloop()