samples/symlinks/sessions - Linux-audio-dev

James Morris

13 Jul 13 Jul

5:13 p.m.

On 13/07/12 James Morris <james(a)jwm-art.net> wrote:

Hi, My sampler app has Non Session Mangement implemented but is currently still referring to external files by their original path. I want to use the symlink method as discussed fairly extensively here but I'd like to know if there is any recommended strategy for naming the symlink of a sample. It could so happen that as far as the filesystem is concerned the only discerning uniqueness between two samples is in the path (ie kit1/snare1.wav and kit2/snare1.wav). I've come up with three possible solutions to this (in my current order of preference): 1) symlink-to-sample created in a subdir named using a hash* of the full path to external file 2) painstakingly re-create the full path within the session dir and add the symlink into that. 3) some horrible text manipulation of the full path (ie replace / with _) that is bound to fail. * J. Liles mentioned SHA1 here: http://linuxaudio.org/mailarchive/lad/2012/3/30/189343 Are there other/better options or disagreements about (1) being a good choice over the other options I've presented?

Have just realized that a problem with (1) is the loss of information contained in the naming of the path. Is that a problem or not? James.

Reply

James Morris

7:24 p.m.

On 13/07/12 renato <rennabh(a)gmail.com> wrote:

...

On Fri, 13 Jul 2012 16:11:23 +0100 James Morris <james(a)jwm-art.net> wrote:

On 13/07/12 James Morris <james(a)jwm-art.net> wrote:

Hi, My sampler app has Non Session Mangement implemented but is currently still referring to external files by their original path. I want to use the symlink method as discussed fairly extensively here but I'd like to know if there is any recommended strategy for naming the symlink of a sample. It could so happen that as far as the filesystem is concerned the only discerning uniqueness between two samples is in the path (ie kit1/snare1.wav and kit2/snare1.wav). I've come up with three possible solutions to this (in my current order of preference): 1) symlink-to-sample created in a subdir named using a hash* of the full path to external file 2) painstakingly re-create the full path within the session dir and add the symlink into that. 3) some horrible text manipulation of the full path (ie replace / with _) that is bound to fail. * J. Liles mentioned SHA1 here: http://linuxaudio.org/mailarchive/lad/2012/3/30/189343 Are there other/better options or disagreements about (1) being a good choice over the other options I've presented?

Have just realized that a problem with (1) is the loss of information contained in the naming of the path. Is that a problem or not?

if it's a symlink, one could just see the path to original file with ls -l ? renato

Yes while it is a symlink but if archived with tar -h then only the basename + extension will remain. A future user might find information encoded in the path (ie name of sample library) helpful (corrupt sample?). Perhaps a small text file showing the path could be added. I like the idea of hashing the path, as descending through multiple levels of nested-but-otherwise-empty sub directories is irksome. james.

Reply

J. Liles

10:10 a.m.

On Sun, Jul 15, 2012 at 7:55 PM, David Robillard <d(a)drobilla.net> wrote:

...

On Fri, 2012-07-13 at 15:57 +0100, James Morris wrote:

Hi, My sampler app has Non Session Mangement implemented but is currently still referring to external files by their original path. I want to use the symlink method as discussed fairly extensively here but I'd like to know if there is any recommended strategy for naming the symlink of a sample. It could so happen that as far as the filesystem is concerned the only discerning uniqueness between two samples is in the path (ie kit1/snare1.wav and kit2/snare1.wav). I've come up with three possible solutions to this (in my current order of preference): 1) symlink-to-sample created in a subdir named using a hash* of the full path to external file 2) painstakingly re-create the full path within the session dir and add the symlink into that. 3) some horrible text manipulation of the full path (ie replace / with _) that is bound to fail. * J. Liles mentioned SHA1 here: http://linuxaudio.org/mailarchive/lad/2012/3/30/189343 Are there other/better options or disagreements about (1) being a good choice over the other options I've presented?

I just used the original name of the file, with a number added for uniqueness if necessary (which is very seldom the case). It works and is much more human-friendly and straightforward than the above options. A "make this path unique by sticking a number on the end of it" function turns out to be pretty useful anyway. Of course, you need to actually check for existence of files to create it, which might be a problem in some cases (though not any I've encountered), but anything that assumes a mapping based on the current path is a unique identifier for a particular file's contents is bound to fail anyway.

This is exactly what I do in Non-DAW as well. Use the original name and add an ordinal number until it's unique.

Reply

David Robillard

8 p.m.

On Mon, 2012-07-16 at 09:45 +0100, James Morris wrote:

...

On 15/07/12 David Robillard <d(a)drobilla.net> wrote:

On Fri, 2012-07-13 at 15:57 +0100, James Morris wrote:

Hi, My sampler app has Non Session Mangement implemented but is currently still referring to external files by their original path. I want to use the symlink method as discussed fairly extensively here but I'd like to know if there is any recommended strategy for naming the symlink of a sample. It could so happen that as far as the filesystem is concerned the only discerning uniqueness between two samples is in the path (ie kit1/snare1.wav and kit2/snare1.wav). I've come up with three possible solutions to this (in my current order of preference): 1) symlink-to-sample created in a subdir named using a hash* of the full path to external file 2) painstakingly re-create the full path within the session dir and add the symlink into that. 3) some horrible text manipulation of the full path (ie replace / with _) that is bound to fail. * J. Liles mentioned SHA1 here: http://linuxaudio.org/mailarchive/lad/2012/3/30/189343 Are there other/better options or disagreements about (1) being a good choice over the other options I've presented?

I just used the original name of the file, with a number added for uniqueness if necessary (which is very seldom the case). It works and is much more human-friendly and straightforward than the above options. A "make this path unique by sticking a number on the end of it" function turns out to be pretty useful anyway. Of course, you need to actually check for existence of files to create it, which might be a problem in some cases (though not any I've encountered), but anything that assumes a mapping based on the current path is a unique identifier for a particular file's contents is bound to fail anyway.

Well I spent what time I had free over the weekend implementing the first option. I did take a look beforehand to see what NON-Daw did (it does what you recommend), but disliked how it didn't check for an existing symlink pointing to the file it was creating a new symlink for. I decided checking existence of one directory to be easier than examing each symlink in a directory to see where it points. However I'd not be surprised if these benefits were outweighed by other costs.

FWIW, since I originally did quite a bit of advocating of this scheme because of my experience doing it for plugins, I should mention an important part of the system there: The *host* gets to make all these decisions. e.g. the host could make symlinks, or always copy, or use hashing and copy when necessary, or just let things use absolute path references, and so on. This is achieved with a callback to map paths to/from state. It might be nice for a session manager to be able to do the same thing, though this means a round-trip for each path, and is likely much less feasible for a host of other reasons (a one-file C "library" to Do The Right Thing is probably more useful), but I thought I'd mention it. Cheers, -dr

Reply

J. Liles

22 Jul 22 Jul

12:17 a.m.

On Fri, Jul 20, 2012 at 5:24 AM, James Morris <james(a)jwm-art.net> wrote:

...

On 16/07/12 David Robillard <d(a)drobilla.net> wrote:

On Mon, 2012-07-16 at 09:45 +0100, James Morris wrote:

On 15/07/12 David Robillard <d(a)drobilla.net> wrote:

On Fri, 2012-07-13 at 15:57 +0100, James Morris wrote: > Hi, > > My sampler app has Non Session Mangement implemented but is > currently still referring to external files by their original > path. > > I want to use the symlink method as discussed fairly extensively > here but I'd like to know if there is any recommended strategy > for naming the symlink of a sample. > > It could so happen that as far as the filesystem is concerned the > only discerning uniqueness between two samples is in the path (ie > kit1/snare1.wav and kit2/snare1.wav). > > > I've come up with three possible solutions to this (in my current > order of preference): > > > 1) symlink-to-sample created in a subdir named using a hash* of > the full path to external file > > 2) painstakingly re-create the full path within the session dir > and add the symlink into that. > > 3) some horrible text manipulation of the full path (ie replace / > with _) that is bound to fail. > > > * J. Liles mentioned SHA1 here: > http://linuxaudio.org/mailarchive/lad/2012/3/30/189343 > > Are there other/better options or disagreements about (1) being a > good choice over the other options I've presented? I just used the original name of the file, with a number added for uniqueness if necessary (which is very seldom the case). It works and is much more human-friendly and straightforward than the above options. A "make this path unique by sticking a number on the end of it" function turns out to be pretty useful anyway. Of course, you need to actually check for existence of files to create it, which might be a problem in some cases (though not any I've encountered), but anything that assumes a mapping based on the current path is a unique identifier for a particular file's contents is bound to fail anyway.

Well I spent what time I had free over the weekend implementing the first option. I did take a look beforehand to see what NON-Daw did (it does what you recommend), but disliked how it didn't check for an existing symlink pointing to the file it was creating a new symlink for. I decided checking existence of one directory to be easier than examing each symlink in a directory to see where it points. However I'd not be surprised if these benefits were outweighed by other costs.

FWIW, since I originally did quite a bit of advocating of this scheme because of my experience doing it for plugins, I should mention an important part of the system there: The *host* gets to make all these decisions. e.g. the host could make symlinks, or always copy, or use hashing and copy when necessary, or just let things use absolute path references, and so on. This is achieved with a callback to map paths to/from state. It might be nice for a session manager to be able to do the same thing, though this means a round-trip for each path, and is likely much less feasible for a host of other reasons (a one-file C "library" to Do The Right Thing is probably more useful), but I thought I'd mention it. Cheers, -dr

Any thoughts about symlinking to symlinked samples? I decided to follow symlinks when storing sample paths in memory and then recreate the symlink path when saving. This avoids symlinking to symlinks. I believe it provided some other minor benefits but can't exactly recall what they were now. james.

When does this situation come up? The whole point of the symlinks is simply to permit some kind of session portability without the session manager or user having to understand the project format. So, for that purpose, nothing really matters except that a tar or a copy should include all (within reason) of the external resources the session requires.

Reply

J. Liles

1:34 a.m.

On Sat, Jul 21, 2012 at 3:50 PM, James Morris <james(a)jwm-art.net> wrote:

...

On 21/07/12 "J. Liles" <malnourite(a)gmail.com> wrote:

On Fri, Jul 20, 2012 at 5:24 AM, James Morris <james(a)jwm-art.net> wrote:

[..]

Any thoughts about symlinking to symlinked samples? I decided to follow symlinks when storing sample paths in memory and then recreate the symlink path when saving. This avoids symlinking to symlinks. I believe it provided some other minor benefits but can't exactly recall what they were now. james.

When does this situation come up? The whole point of the symlinks is simply to permit some kind of session portability without the session manager or user having to understand the project format. So, for that purpose, nothing really matters except that a tar or a copy should include all (within reason) of the external resources the session requires.

User creates session. User then creates 2nd session and wants to use same samples as 1st session but, for whatever reason, rather than loading samples from original paths, goes instead to 1st session dir and loads the samples from that. User then decides to delete 1st session thereby breaking 2nd session.

Nothing is fool-proof...

Reply

Paul Davis

3:54 a.m.

On 7/21/12, James Morris <james(a)jwm-art.net> wrote:

...

User creates session. User then creates 2nd session and wants to use same samples as 1st session but, for whatever reason, rather than loading samples from original paths, goes instead to 1st session dir and loads the samples from that. User then decides to delete 1st session thereby breaking 2nd session.

peventing this scenario is the origin of ardour's use of hard-links, at least it is if the files are on the same filesystem as the 2nd session. by popular demand, its being removed ...

Reply