[LAU] Music Editors for Blind Users?

Hi! A visually impaired Linux user here. I recently came across this great thread[1] and decided to write this message. My goal is a little different from the original topic author (generating MIDI via code VS. recording via a keyboard) so I decided to start a new discussion. Here's my situation (warning: very long post!): First of all, I'm just an amateur and a beginner with nearly zero knowledge about music production. I had very little contact with DAWs, even in Windows... The closest I've gotten to a "DAW" that I tried to use was Anvil Studio (many years ago), which wasn't accessible either. The little that I know about MIDI editors and DAWs is from what I read looking for such programs in Linux. I'm using Slackware Linux, with Speakup[2] as my console screen reader (with Espeak[3] through the Espeakup[4] connector, because I don't have a hardware voice synth), and Orca[5] as the screen reader for X (note: I use only ALSA, without Pulseaudio). I'm trying to create some kind of audio setup for quite some time (since around 2010). Back then, I read everywhere that it was needed to recompile the kernel with a realtime patch. Then I found the courage and did just that, but: * I never got JACK to really work. It complained that the soundcard was already being used, probably because of my screen readers which are softsynths (again, I don't have a hardware synth nor a Braille display), and I suspect they take all the soundcard for themselves. JACK only got to start if I used a second soundcard, an M-Audio Fast Track Pro (which I bought because it seemed to work on Linux), but it didn't go too well either. Only much later (last year, 2013) I came across this post http://joegiampaoli.blogspot.com/2011/06/m-audio-fast-track-pro-for-debian-… It seems the Fast Track Pro is already supported on newer (3.X) kernels, but I still didn't test that. * Most audio related apps were (and still are) made with QT GUIs. Though QT's accessibility API exists for a long time (and is well documented[6]), there were no means for Orca to communicate with it yet, so all those apps were completely inaccessible. * With the recompiled kernel, my video driver installation broke (I wasn't even using proprietary drivers or anything). X only showed a blinking cursor on the screen. Orca worked just fine, but it was a problem when a sighted person needed to use the computer. So, I went back to the stock kernel (where the video driver worked) and gave up, for the most part. I settled on doing everything from my keyboard (which requires some sighted assistance), then sending the result as a wav file to my computer, then processing it with SoX[7] (which I use all the time) and/or Ecasound[8]. But recently I began thinking of using sounds from softsynths or other programs (for example, LinuxSampler[9]) instead of my keyboard's internal sounds, and for that, I think it would be best if I could record MIDI directly into my computer. So, the search started again, and I began testing some command line applications (most of which were already mentioned on the thread in [1]): * CuSe[10]: stands for "Cursed Sequencer", and like the name suggests, is Curses based. Interestingly, it was made specifically with visually impaired users in mind. Some features look interesting, like punch in/out, a step sequencer, and (the most interesting of all) a "remote control" like thing, where it's possible to configure key combinations for controlling the software through the hardware synth's keyboard (avoiding the need to run back and forth between the synth and the computer). But it doesn't seem to have some features I want, like adding/editing lyrics to the notes, MIDI CCs and SysEx events. Unfortunately, I was never able to get any sound from it (be it through FluidSynth or my MIDI keyboard). * Midish[11]: already mentioned on the thread in [1], it supports recording and playing MIDI, adding events and SysEx messages, and it has nice filtering features, making it possible to split the keyboard or transform one event into another. It only supports raw MIDI ports and it took me a lot of time to figure that to make it work, I needed to "modprobe snd_virmidi", then (a)connect the virmidi ports to my synth's ports, then configure the inputs and outputs via the "~/.midishrc" file. It was the program I got working more successfully, and I placed my hopes on Midish for a long time, reading and rereading the manual to see if I could make it do what I wanted. I was able to record MIDI via my keyboard (though I wish there was a "remote control" feature or something similar). However, sometimes I also wanted to insert notes manually, and as already pointed out in this message from the thread in [1] http://lists.linuxaudio.org/pipermail/linux-audio-user/2014-February/096120… that isn't the most intuitive task, and Midish doesn't seem to let me add lyrics or edit notes (only cut/copy/paste measures)... Well, unless I save the song to Midish's format and open it in a text editor, but I don't want to have to memorize which note numbers are which :P. Or unless I'm missing something else. * Midiedit[12]: contrary to Midish, this only edits and doesn't record. It's contained in only one file and is a Perl script. It has two interfaces: a Curses based one and a "text editor" based one. With the Curses interface, it can playback MIDI files (I used Aconnect to hook its ports to FluidSynth and the sounds played), and as the name suggests, edit. Events (notes, CCs, program changes etc.) can be inserted, edited and deleted. They're displayed sequentially in a list, one per line, with their starting position in ticks, duration and channel. They can be navigated by using the Up and Down arrows, then edited (the bottom screen shows the available actions and which keys do what). One thing I love about it is that it plays back the notes when the cursor focuses on them. Another thing is that it treats each note as a single event, by matching the note ons with their respective note offs, which makes managing notes much easier. So far so good, but everything always has some "cons". I wish it could mute or solo tracks, and have a separate view for each track. The search function can be used to jump straight to the notes from a specific channel, and cycle through the results with "n" and "N" (similar to Vi/Vim), but the notes from the other tracks are still there and it gets confusing easily (at least for me). It doesn't seem to have an action implemented on the interface to handle lyrics and text events, but these still can be edited through the "text editor" interface, which is called "dump mode". In this mode, it opens your text editor of choice (the one in the "$EDITOR" environment variable) and represents the MIDI file's contents as readable Perl code (based on the dump output from the MIDI::Perl[13] module). All events (including text events, lyrics and SysEx) can be edited this way, (although obviously it doesn't play back the notes, and we don't have the luxury of the unification of note ons and offs). After saving and closing the editor, the Perl code is processed and the content is written back to the MIDI file. * From the same author of Midiedit, there are other interesting Curses based MIDI tools[14], like Midifade[15], which generates "sliders" for controlling MIDI events through the PC keyboard (on the command line, we specify how many sliders to generate, the ALSA port, the MIDI channel and which event each slider will control), and can, for example, change Ecasound's controller parameters, or the "knob" parameters from a softsynth. That's it for the CLI apps I tested. Midish and Midiedit are the ones I had most "luck" with, but my attempts with them weren't that productive either. My search for an accessible MIDI editor or DAW continues. Recently, I updated Slackware to 14.1. I'm using the stock kernel, no realtime patches or anything yet. First I want to know if I really need it (I read that because of changes in recent kernels, RT patches aren't strictly necessary anymore), and if there's some app that I'm able to use (if not, there's no point to even try compiling a realtime kernel :P). I finally got the QT-AT-SPI[16] bridge to compile, meaning I can access QT GUI apps to some degree now. So, I decided to test some of the most popular Linux DAWs, those QT based audio apps that taunted me since I first read about audio production on Linux (basically, the only options that were available and which I couldn't use). was MusE[17]: * The possibility of mixing MIDI with audio (samples, loops, vocals etc.). * Instrument definition files for various brands of keyboards that ease the use of those synthesizers, making it so that we don't need to memorize patch names/numbers and controller numbers. * MIDI recording. * Editing of all types of MIDI events. Besides the visual editor, there's a "event list editor" (I suppose it's something like in Midiedit), so maybe it could work for me. * A few internal instruments (some synths and a FluidSynth frontend). * DSSI support. * (apparently) native VST support too (and I assume, a few Windows VSTs through DSSI-VST). * And the "remote control" feature :D! Of course there's the multitude of functions that depend on the mouse (which I can't use), but it seems to have customizable shortcuts. It's a graphical application, obviously very far from being text-based, but I read that it also has some Python bindings, so maybe a few things can be scripted...? However, sadly, all isn't as good as it sounds. I installed MusE 2.1.2 and opened the interface. The menus work fine, but the screen reader has problems reporting the interface elements (for example, it says "spinbox" or "checkbox" but doesn't specify wish item that control corresponds to. The "MusE Settings -> Global Settings" screen is full of these examples and unreadable through the screen reader). I just took a quick "look" at the interface to have an idea of what the screen reader could read. I didn't test recording or anything yet, mainly because of the aforementioned problem with JACK, nor could sort out the setup part about "/dev/rtc"... While we're on the subject of MusE, a question for the "MusErs": if I don't use JACK (or use the "-a" switch), won't MusE ever play anything through the PC's speakers (imported audio, internal instruments, FluidSynth soundfont sounds)? Or will it just be unable to communicate with JACK aware applications? I'm thinking if I'll insist on MusE for a bit more, and if I do, maybe I'll contact the developers about the accessibility issues. Based on my limited knowledge of MIDI editors and DAWs which I acquired from what I've tested or read about so far, desired features for my dream DAW would be: * Curses based interface. * Both JACK and some form of ALSA Seq support (even if it's only with raw MIDI ports, or some workaround involving sending commands through Amidi :P). JACK would be for more complicated setups. ALSA would be for when I only want to do some quick MIDI editing and don't want to use a second soundcard just because of JACK (I need my screen reader, so I can't turn it off to free the first card). * Playback of MIDI files. * Support of a metronome click, and a "count in" feature for recording. * A "remote control" feature, or a way to start recording (or the count in) only when I press a key on my hardware instrument. * Each note shown as only one event, by matching the note on with it's respective note off. This way we don't need to struggle scrolling through the events trying to match the ons and offs. * Possibility of inserting/editing/deleting all types of events, including text events, lyrics and SysEx messages. * A way to select a track to be "active". * A separate list of MIDI events for each track (only showing the contents of the "active" track), where events can be browsed with arrow keys and be edited or deleted. Also, the possibility of inserting new events through the interface (without using the MIDI keyboard). * Playback of notes when the cursor focuses on them. * Ability to mute and solo tracks. * Support for instrument definition files, to ease the use of hardware synths (maybe some softsynths too). With that, I'd be able to select the patches and controllers specific to my hardware synth directly from the computer and with help of my screen reader, reducing the need of asking for sighted assistance to know the patch names. * DSSI support, or the possibility of interacting with some command line based DSSI host. * The ability to assign the MIDI channel and an instrument (ALSA/JACK/whatever ports and patch names) to each track (or is this a work better suited to a "patchbay" like app?). * Rendering of MIDI tracks to audio with the used synths' sounds. * Customizable keyboard shortcuts. * Some form of scripting support, like Perl or Python or Lua bindings, and the ability to assign user scripted functions to some keybinding in the editor. Some more complex features: * The ability to have both MIDI and audio in the same project. The audio doesn't need to be editable (just "importable"), only needs to have its starting position adjustable to be in sync with the MIDI tracks (triggered via some MIDI event maybe?). * Recording of MIDI while playing back the audio tracks, keeping them in sync. * Maybe a pattern editor, where the lists of events become a phrase, chains of phrases make patterns and chains of patterns make a song (but maybe it's too complex and is a discussion for another day :P). And that was my utopic idea for an accessible Linux DAW. I don't know how hard it is to implement all of that, specially the "MIDI alongside audio" part. Unfortunately I understand nothing about audio programming (honestly, I already tried :P) and very little about music production, so a lot of features I pointed out might be complete nonsense, or might be accomplished in a much better way (if anyone knows a way, preferably via command line, please let me know). As for the "patchbay" (with "jack_connect" etc.), I think even a simple Shell Script with Dialog based menus would be fine. As for post processing and mixing (applying LADSPA/LV2 effects etc.), it can be left to some other app (Ecasound for example) after all tracks are rendered to audio. And last but not least, on the thread in [1], I found Teqqer[18]. I couldn't test it yet because of the JACK problem (I seriously need to sort that out), but I took a look at the default config variables in the Python source code, and it looks like scripting support and customizable shortcuts are mostly taken care of. From what I understand from the shortcuts, the interface for inserting/editing MIDI events is a bit similar to Midiedit (Up/Down to cycle through events, although note ons and offs are separate) and it seems to have one view for each track (Left/Right to cycle through tracks), and it even supports patterns! :D So, minus the recording part (from what I understood, it wasn't made for that), the lack of ALSA (without JACK) support, I think it has the potential to become something like what I'm looking for (plus, I always wanted to try a tracker, but never found a "screen reader friendly" one). I'll definitely be watching Teqqer to see where it goes! While my dream DAW doesn't come true, does anyone know a command line way to record MIDI while simultaneously playing an audio file and keeping both in sync (even if it involves JACK)? I think Ecasound can be setup to do the opposite, playing back a MIDI file while recording audio (but it doesn't support MIDI recording, just playback... Or does it? :P). Well, that's it. Sorry for the overly long post. Thanks for your time! [1]: http://lists.linuxaudio.org/pipermail/linux-audio-user/2014-January/095910.… [2]: http://www.linux-speakup.org/ [3]: http://espeak.sourceforge.net/ [4]: https://github.com/williamh/espeakup [5]: https://wiki.gnome.org/Projects/Orca [6]: http://qt-project.org/doc/qt-4.8/accessible.html [7]: http://sox.sourceforge.net/ [8]: http://nosignal.fi/ecasound/ [9]: http://www.linuxsampler.org/ [10]: http://pi4.informatik.uni-mannheim.de/~haensel/cuse/index_en.html [11]: http://www.midish.org/ [12]: http://www.pjb.com.au/midi/midiedit.html [13]: http://search.cpan.org/perldoc?MIDI [14]: http://www.pjb.com.au/midi/index.html [15]: http://www.pjb.com.au/midi/midifade.html [16]: http://projects.kde.org/qtatspi [17]: http://www.muse-sequencer.org/ [18]: https://github.com/fps/teqqer -- ____________________ Blog: http://aiyumi.warpstar.net/