13 Jun
2011
13 Jun
'11
1:43 p.m.
Hi,
I'm trying to cancel an ongoing sf_* I/O operation (from another
thread of course). I'm finding this extremely unpleasant to implement.
* libsndfile sf_read_* and sf_write_* loop until the requested number
of frames has been transferred -- so signals are useless, even if I
were to disable SA_RESTART behavior
* a pthread_cancel() in the middle of I/O seems to mess up the
SNDFILE's data structure to the point that it is unusable later on; so
besides having to create a new thread after cancelling, I have to
re-open the SNDFILE (or manage a pool of back-up SNDFILE's, complete
with duplicate fd's)
Also, libsndfile doesn't seem to employ a user-space cache like normal
C streams (FILE *) -- instead it goes directly to read(2) and
write(2). This of course makes it exceedingly inefficient to read 1
frame at a time (due to repetead syscall overhead).
So... it's hard to process low-latency dynamic I/O requests with
libsndfile. It would be nice if libsndfile could allow short reads and
writes via some sf_command parameter.
BTW, I've had trouble sending messages to the sndfile-users mailing
list (I subscribed, then it still rejected my mails). In any case,
since there seems to be no mailing list archive, it doesn't seem like
a very trustworthy mailing list.
-- Dan
13 Jun
13 Jun
1:56 p.m.
Dan Muresan wrote:
I'm trying to cancel an ongoing sf_* I/O operation
(from another
thread of course). I'm finding this extremely unpleasant to implement.
Maybe its a bad idea. :-)
* libsndfile sf_read_* and sf_write_* loop until the
requested number
of frames has been transferred -- so signals are useless, even if I
were to disable SA_RESTART behavior
* a pthread_cancel() in the middle of I/O seems to mess up the
SNDFILE's data structure to the point that it is unusable later on; so
besides having to create a new thread after cancelling, I have to
re-open the SNDFILE (or manage a pool of back-up SNDFILE's, complete
with duplicate fd's)
Yep, sounds like a bad idea.
Also, libsndfile doesn't seem to employ a
user-space cache like normal
C streams (FILE *) -- instead it goes directly to read(2) and
write(2). This of course makes it exceedingly inefficient to read 1
frame at a time (due to repetead syscall overhead).
Reading one frame at a time sounds like a bad idea too.
So... it's hard to process low-latency dynamic I/O
requests with
libsndfile. It would be nice if libsndfile could allow short reads and
writes via some sf_command parameter.
It does. You can read any number of frames from 1 through to as many
frames as the file contains.
Basically you could define how fine grained you want you interruption
to be and then only read enough frames so that you can interrupt between
reads.
Say you want to be able to interrupt down to a 1/100th of a second and
your sample rate is 48000kHz. If you read 480 frames in each read and
get your interruption without jumping through difficult hoops.
BTW, I've had trouble sending messages to the
sndfile-users mailing
list (I subscribed, then it still rejected my mails).
I just checked, and the address you used to post this email to the LAU
list are not subscribed to the libsndfile-users list. Thats why the list
is rejecting your email.
In any case,
since there seems to be no mailing list archive, it doesn't seem like
a very trustworthy mailing list.
It works if you are suscribed, but I'm just as happy answering
libsndfile questions here (or better yet, the LAD list since this
is really a development question).
Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/
11:57 p.m.
Hi Erik -- please CC me so I can reply (I don't receive messages from
LAU directly). I'm quoting manually here:
I don't think so... Issuing large requests, then cancelling as needed
gives a process the lowest possible latency for unpredictable seeks
(caused e.g. by user commands), while keeping CPU usage low (by
avoiding syscall and context switching overhead)
I meant "short reads" in kernel-speak sense: read(2) can return a
number of bytes less than the number requested when interrupted by a
signal (if SA_RESTART is disabled). My proposal was to add a
sf_command() parameter that disables the looping behavior of sf_read()
on EINTR, and makes it return exactly as many frames as the first
read() call manages to get.
On second thought, though, this proposal could not possibly work
without a userspace (libsndfile) cache, because read() might return
incomplete frames, which would need to be processed in a later call. I
discovered the virtual I/O in sndfile, and that seems more
appropriate. So I withdraw my proposal.
I'm trying
to cancel an ongoing sf_* I/O operation (from another
Maybe its a bad idea. :-)
Reading one frame at a time sounds like a bad idea
too.
1 frame at a time was an extreme example. The point was that
libsndfile doesn't employ a user-space cache, but direct system calls.
Reading 10, 100 or 480 frames at a time will still incur syscall
overhead (== CPU usage), and progressively larger cancel latencies.
libsndfile. It
would be nice if libsndfile could allow short reads and
writes via some sf_command parameter.
It does. You can read any number of frames
from 1 through to as many
frames as the file contains. Basically you could define how fine grained you want
you interruption
to be and then only read enough frames so that you can interrupt between
That was exactly the strategy I originally described as inadequate,
due to syscall overhead (in the absence of userspace caching)...
I just checked, and the address you used to post this
email to the LAU
list are not subscribed to the libsndfile-users list. Thats why the list
is rejecting your email.
That's exactly the problem: I subscribed about two weeks ago, received
a confirmation, and sent a message at that time (which received no
bounces, but no replies either). Now, the mailserver somehow forgot
about me and is rejecting my messages. Or something...
libsndfile questions here (or better yet, the LAD list
since this
is really a development question).
You're right -- I've realized right after hitting "Send". Please feel
free to respond on LAD (but please CC me). I am replying on LAU this
one time to clear up the issues that were already brought up, since my
original message seemed confusing.
-- Dan
14 Jun
14 Jun
2:12 a.m.
Dan Muresan wrote:
I meant "short reads" in kernel-speak sense: read(2) can return a
number of bytes less than the number requested when interrupted by a
signal (if SA_RESTART is disabled). My proposal was to add a
sf_command() parameter that disables the looping behavior of sf_read()
on EINTR, and makes it return exactly as many frames as the first
read() call manages to get.
I accept good quality, clean patches with tests for the functionality
you are adding. Wherever possible, they should be cross platform.
Was that a confirmation or a request for confirmation? Joining a
mailing list usually involves:
a) Send a request to join.
b) Receiving a "confirm that you want to join" message.
c) Sending "confirm that you want to join".
d) Receiving a "yes, you are mow a member" message.
Hi Erik -- please CC me so I can reply (I don't
receive messages from
LAU directly). I'm quoting manually here:
I don't think so... Issuing large requests, then cancelling as needed
gives a process the lowest possible latency for unpredictable seeks
(caused e.g. by user commands), while keeping CPU usage low (by
avoiding syscall and context switching overhead)
Let me put it this way:
a) When I designed libsndfile over 10 years ago, I never dreamed
anybody would try this.
b) In the 10 years libsndfile has been around and the probably
hundreds of applications it has been used in, noone has suggested
that it would be a good idea if libsndfile could do this.
To me that suggests that either you have a completely unique
problem to solve or that they are other solutions to your
problem that other people use to get around the same problem.
My guess is that your problem is not completely unique.
I'm
trying to cancel an ongoing sf_* I/O operation (from another
Maybe its a bad idea.
:-) Reading one
frame at a time sounds like a bad idea too.
1 frame at a time was an extreme example. The point was that
libsndfile doesn't employ a user-space cache, but direct system calls.
Reading 10, 100 or 480 frames at a time will still incur syscall
overhead (== CPU usage), and progressively larger cancel latencies.
libsndfile. It would be nice if libsndfile could allow short reads and
writes via some sf_command parameter.
It does. You can read any number of frames
from 1 through to as many
frames as the file contains. On second thought, though, this proposal could not
possibly work
without a userspace (libsndfile) cache, because read() might return
incomplete frames, which would need to be processed in a later call.
Modifying libsndfile to do fread/fwrite style buffering would be
relatively easy. Again, patches accepted.
I just
checked, and the address you used to post this email to the LAU
list are not subscribed to the libsndfile-users list. Thats why the list
is rejecting your email.
That's exactly the problem: I subscribed about two weeks ago, received
a confirmation, and sent a message at that time (which received no
bounces, but no replies either). Now, the mailserver somehow forgot
about me and is rejecting my messages. Or something...
All other complaints of this sort that I have received have been
from people who couldn't figure out the subscribe procedure or
joined from one email address and sent mail to the list from
another.
Cheers,
Erik
--
----------------------------------------------------------------------
Erik de Castro Lopo
http://www.mega-nerd.com/
2:17 a.m.
On Mon, Jun 13, 2011 at 8:12 PM, Erik de Castro Lopo
<mle+la(a)mega-nerd.com> wrote:
Modifying libsndfile to do fread/fwrite style
buffering would be
relatively easy. Again, patches accepted.
is it even necessary to modify it? can't you just use the virtual I/O
API you added?
8:29 a.m.
is it even necessary to modify it? can't you just
use the virtual I/O
API you added?
Paul, that's exactly what I wrote in my last message:
incomplete frames, which would need to be processed in
a later call. I
discovered the virtual I/O in sndfile, and that seems more
appropriate. So I withdraw my proposal.
The virtual I/O API is a very nice addition.
-- Dan
8:46 a.m.
To me that suggests that either you have a completely
unique
problem to solve or that they are other solutions to your
problem that other people use to get around the same problem.
My guess is that your problem is not completely unique.
Certainly not unique. My guess is that nobody worried about a
(possibly small) performance penalty, and instead used simpler
strategies. Or they gave up libsndfile, or they never used libsndfile
in the first place (e.g. due to the scaling factors issue).
Modifying libsndfile to do fread/fwrite style
buffering would be
relatively easy. Again, patches accepted.
Given the virtual I/O layer in libsndfile, this is superfluous (and,
furthermore, a userspace cache adds an extra layer of copying, which
in *other* use cases could reduce performance).
Was that a confirmation or a request for confirmation?
Joining a
mailing list usually involves:
I know. Maybe I messed something up -- I don't have copies anymore.
However if that were the case the message I sent two weeks ago should
have bounced back too.
The main problem that I see though is the lack of a mailing list
archive. So let's continue on LAD.
-- Dan
4901
days inactive
4902
days old
linux-audio-user@lists.linuxaudio.org
6 comments
3 participants
participants (3)
-
Dan Muresan -
Erik de Castro Lopo -
Paul Davis