[LAD] NSM - handling large files

[David Robillard]

On Thu, 2012-03-29 at 18:29 -0700, J. Liles wrote:
[...]
> Currently, when you drag n' drop an external audio file into a Non-DAW
> timeline (as opposed to recording it from within Non-DAW), the file
> remains external with its path recorded
> in the project's journal. Using a symlink for this would be better in
> *at least* the following two ways:
> 
> 1) Allows archiving scripts etc. to discover and import the external
> source *without having to understand the Non-DAW journal format*
> 2) It would allow Non-DAW to import external sources *without having
> to update (or break) any existing references*
> 
> I cannot imagine any argument that could propose that these are bad things.

Me neither.  Well, not a good one, anyway :)

> If all Linux Audio software dealt with external references in this
> way, archiving/export would be much less problematic.

Yep.  I arrived at this via the experience of actually doing it - a
special magic solution seems feasible when you've got blinders on and
are just thinking about an SM program, but when you realize this can
cross-cut all the way from inside a generic library for loading standard
file(s) formats, through a plugin, through the host, through the session
manager, it's clear links are the only way.  It can be hard enough to
get saving right *without* making the load all weird.

It's not a "YOU MUST DO THIS FOR NON SESSION MANAGER", it's a "you
should do this because it makes external file references manageable"

> However, I would also like to offer an interesting little statistic...
> I, personally, have hundreds of projects representing terabytes of
> data, and in all that I don't have a single project which refers to
> anything external to its project directory. This is something that
> only effects certain users who make extensive use of sound-clip or
> sample libraries. Not people who just do plain old
> recording/synthesis/mixing. So let's try not to make a mountain out of
> a mole hill. What is the actual percentage of users who have
> references to external files *and* a strong need to export their
> sessions? I suspect that it is in fact a very low number.

Certainly true for recorded audio files, sharing those between programs
is pretty esoteric (and if you're doing it, as Fons says, you "know what
you're doing" anyway).  In the greater scheme of things, though, using
sample banks and such is certainly not nichey.
 
When you do work with such things, it really sucks to not have a
reliable way to roll up a finished piece so it will actually work in the
future.

Plus, being able to easily share full sessions and collaborate and stuff
is cool and open sourcey :)

> Furthermore, in addition to the plain old symlinks, a truly robust
> solution might also store e.g. SHA1 hashes of external files, so that
> any mismatch is detectable.

Good idea.  Certainly can't hurt to know, though computing a hash of
some files might be extremely expensive...

-dr