[LAD] NSM - handling large files

[Joel Roth]

On Thu, Mar 29, 2012 at 03:44:17PM +0200, Emanuel Rumpf wrote:
 
> Back to life - back to reality
> 
> 1. We start qtractor as part of a session, create some midi-tracks,
> include some external wav-files.
> 
> Should the SM "know" about these external files, as I suggested ?
> (Allowing the user to find out basic info about it. )
> Or leave it completely to qtractor ?

As a developer of limited abilities, I would like the bar
of integrating Nama with other programs to be as low
as possible. :-)

As I understand it, the primary motivation for a session
manager is to be able to save and restore the state of an
audio project[1] that consists of multiple interconnected
programs running on a multiprocessing Linux system.

In other words, the minimum capability sufficient to be
useful would be some sort of checkpointing mechanism.

A secondary goal, discussed extensively here, is that it
would be nice to be able to copy a session or export a
session. This is where all the contentious issues with
handling filesystem objects comes up. 

The latter goal should be optional, IMO, rather than required.

Managing files, seems relatively simple in the case of 
of Nama, which is a purely non-destructive editor,

Once recorded, Nama *never* modifies or deletes audio files,
so these files don't need to be involved in checkpointing.
(If a user wants to delete WAV files associated with a bad
take, that is her responsibility.)

All the mutable state information for a project is
serialized and stored as JSON. State files for a project
can be named. In future they will be placed under
VCS to add branching (and possibly
merging) abilities.

If Nama (or Qtractor, Ardour, etc.) is to clone a project,
that concern is internal to the app: I wouldn't expect a
session manager to know it should link the files in the .wav
directory and copy the other files. (Straight copying of
files might not even be appropriate if those files are
under a VCS.)

> 2. We record a new file in qtractor. Where would it be stored ?
> 
> Should the SM "know" about this file ?

There could be advantages to knowing, however if the aim is
to integrate multiple apps, it will be simpler if the SM and
app only need to communicate the project name and checkpoint
ID. (It gets more complicated than that since the SM 
will need to know that Nama should be responsible for
configuring Ecasound and for either connecting Ecasound to
JACK clients or reporting when Ecasound is ready to be
connected.)

> 3. A session is duplicated.
> How does the SM handle the recorded and external files?

Branching a project (assuming VCS, which I believe is the
future :-) would be cheaper than duplicating a project.

If the SM wants an app to duplicate a project, I would
expect the SM to ask the app to create a new project as a
copy of an existing project. Whether the app chooses to use
symlinking, hardlinking, copying files or branching under a
VCS seems to be the app's internal concern.

> 4. A session is exported.
> Jonathan said, current practice is a simple directory copy.
> Well ! Simple and not bad at all.
> But what about the external files ?
> Just make some symlinks for them ?

The state files and audio files for a Nama project
are in a single directory, easy to copy.

However the .namarc config file specifying the ALSA devices
to be used, the audio recording format, the mixdown format
and many other program behaviors currently exists only
in $HOME.

Even if this file were linked or copied into the project
directory, it may not be seamlessly portable to the
audio hardware and configuration of another system. 

If the process of exporting cannot be defined unambiguously,
or involves vast complexity, we might be more successful by
initially restricting our *first* steps to checkpointing
functionality.

Minimizing complexity seems important, as 
the Linux audio world has already gone through
several session management APIs, with none yet
widely adopted.

Regards,

Joel

1. I use the terms "project" and "session" interchangeably.
 
> 
> -- 
> E.R.
-- 
Joel Roth