[linux-audio-dev] NPTL jack+ardour: large memlock required (was: Jack and NPTL (again?))
24 Nov
2005
24 Nov
'05
8:39 p.m.
Hi
On Tues Nov 22 I wrote:
I've recently moved a system over to Slackware
10.2 which utilises NPTL and
I'm aware of the issues NPTL has raised in the past. Based on a comment on
the Jack website though I sort of assumed that things were now in hand and
that Jack had a workaround in place for the issue.
Despite this I have found that jackd itself (when run using set_rtlimits)
gives an error (-11, EAGAIN) when "creating realtime thread".
I have since discovered (thanks to strace) that the problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be part of
the NPTL pthread_create() function. The error returned by mmap2() (EAGAIN)
indicates either a locked file or "too much locked memory" (according to the
manpage). Because this is an anonymous map the problem must have been the
latter. Increasing the "locked memory" limit from 20MB to 40MB made jackd
start without having to resort to the LD_ASSUME_KERNEL hack.
Another problem then surfaced: with jackd running under NPTL, jack
applications using NPTL (eg: ardour) suddenly started to require all of
their memory to be lockable. For ardour with a large session this can get
quite large (I saw 130MB at one point last night).
So, with jackd running with LD_ASSUME_KERNEL = 2.4.19 and a memlock limit of
20MB everything worked and ardour (with NPTL - ie: without LD_ASSUME_KERNEL)
was obviously not trying to lock excessive amounts of memory. With jackd
running with NPTL and a large enough memlock limit things work but jack
clients appear to require much more memory be locked.
One thing I will try next is recompiling ardour; perhaps there's something
funny there. In any case though, does any of this ring a bell with anyone?
Regards
jonathan
24 Nov
24 Nov
8:58 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required (was: Jack and NPTL (again?))
On Fri, 25 Nov 2005 11:10:05 +1030 (CST)
Jonathan Woithe <jwoithe(a)physics.adelaide.edu.au> wrote:
One thing I will try next is recompiling ardour;
perhaps there's something
funny there. In any case though, does any of this ring a bell with anyone?
Did you try the --unlock jack switch?
Flo
--
Palimm Palimm!
http://tapas.affenbande.org
9:31 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Fri, 25 Nov 2005 11:10:05 +1030 (CST)
Jonathan Woithe <jwoithe(a)physics.adelaide.edu.au> wrote:
Um, no. To be honest I didn't know about it until you mentioned it. The
description of what it does looks like it might assist - I'll give it a
go and let you know.
OOI, why wasn't this required when jack was not running under NPTL?
Thanks for the suggestion.
Regards
jonathan
--
* Jonathan Woithe jwoithe(a)physics.adelaide.edu.au *
* http://www.physics.adelaide.edu.au/~jwoithe *
***-----------------------------------------------------------------------***
** "Time is an illusion; lunchtime doubly so" **
* "...you wouldn't recognize a subtle plan if it painted itself purple and *
* danced naked on a harpsichord singing 'subtle plans are here again'"
*
One thing I will try next is recompiling ardour;
perhaps there's something
funny there. In any case though, does any of this ring a bell with anyone?
Did you try the --unlock jack switch?
27 Nov
27 Nov
7:35 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Fri, 25 Nov 2005 11:10:05 +1030 (CST)
Jonathan Woithe <jwoithe(a)physics.adelaide.edu.au> wrote:
I tried this over the weekend. It didn't help; with the mlock rlimits reset
to 20MB jack couldn't start. With a 40MB mlock limit jack started but then
ardour couldn't load projects due to the thread call failing. In other words,
this flag has made no practical difference to the situation.
Regards
jonathan
--
* Jonathan Woithe jwoithe(a)physics.adelaide.edu.au *
* http://www.physics.adelaide.edu.au/~jwoithe *
***-----------------------------------------------------------------------***
** "Time is an illusion; lunchtime doubly so" **
* "...you wouldn't recognize a subtle plan if it painted itself purple and *
* danced naked on a harpsichord singing 'subtle plans are here again'"
*
One thing I will try next is recompiling ardour;
perhaps there's something
funny there. In any case though, does any of this ring a bell with anyone?
Did you try the --unlock jack switch? 
25 Nov
25 Nov
2:57 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required (was: Jack and NPTL (again?))
On Fri, 2005-11-25 at 11:10 +1030, Jonathan Woithe wrote:
I have since discovered (thanks to strace) that the
problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be
part of the NPTL pthread_create() function. The error returned by
mmap2() (EAGAIN) indicates either a locked file or "too much locked
memory" (according to the manpage). Because this is an anonymous map
the problem must have been the latter. Increasing the "locked memory"
limit from 20MB to 40MB made jackd start without having to resort to
the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that.
The real fix is for JACK to set a reasonable thread stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
As a bonus this will make the footprint of your other threaded apps like
Evolution smaller, though those apps don't care as much as their thread
stacks are pageable.
Lee
28 Nov
28 Nov
7:11 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
Lee
On Fri, 2005-11-25 at 11:10 +1030, Jonathan Woithe
wrote:
Um, yes. :(
I have since discovered (thanks to strace) that
the problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be
part of the NPTL pthread_create() function. The error returned by
mmap2() (EAGAIN) indicates either a locked file or "too much locked
memory" (according to the manpage). Because this is an anonymous map
the problem must have been the latter. Increasing the "locked memory"
limit from 20MB to 40MB made jackd start without having to resort to
the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that. The real fix is for JACK to set a reasonable thread
stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
I will try this and see what happens. Do you know if anyone is preparing a
fix for JACK?
Regards
jonathan
8 Dec
8 Dec
2:24 a.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Fri, 2005-11-25 at 11:10 +1030, Jonathan Woithe
wrote:
I tried this. Although this reduced the size of the mlock calls I found I
still needed an mlock limit of at least 40MB for jack and ardour to start.
However, even with this loading a large session into ardour failed to load
with no error given (beyond the fact that the loading failed).
You mentioned in another post that you're working on a "proper" fix for
jack. I'll wait for this to be implemented before digging into this
issue further since the correct fix might change things. Drop me a line
when a fix has made it into CVS and I'll resume my testing.
Regards
jonathan
I have since discovered (thanks to strace) that
the problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be
part of the NPTL pthread_create() function. The error returned by
mmap2() (EAGAIN) indicates either a locked file or "too much locked
memory" (according to the manpage). Because this is an anonymous map
the problem must have been the latter. Increasing the "locked memory"
limit from 20MB to 40MB made jackd start without having to resort to
the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that.
The real fix is for JACK to set a reasonable thread stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
10 Dec
10 Dec
12:47 a.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Thu, 2005-12-08 at 16:50 +1030, Jonathan Woithe wrote:
Um, all my fix will do is reduce the thread stack size to something more
reasonable, say 256KB per thread. You're still going to need an mlock
limit of more than 40MB to do anything useful.
Lee
On Fri,
2005-11-25 at 11:10 +1030, Jonathan Woithe wrote:
I tried this. Although this reduced the size of the mlock calls I found I
still needed an mlock limit of at least 40MB for jack and ardour to start.
However, even with this loading a large session into ardour failed to load
with no error given (beyond the fact that the loading failed).
You mentioned in another post that you're working on a "proper" fix for
jack. I'll wait for this to be implemented before digging into this
issue further since the correct fix might change things. Drop me a line
when a fix has made it into CVS and I'll resume my testing. I have since discovered (thanks to strace) that
the problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be
part of the NPTL pthread_create() function. The error returned by
mmap2() (EAGAIN) indicates either a locked file or "too much locked
memory" (according to the manpage). Because this is an anonymous map
the problem must have been the latter. Increasing the "locked memory"
limit from 20MB to 40MB made jackd start without having to resort to
the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that.
The real fix is for JACK to set a reasonable thread stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
11 Dec
11 Dec
8:37 p.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Thu, 2005-12-08 at 16:50 +1030, Jonathan Woithe
wrote:
Um, all my fix will do is reduce the thread stack size to something more
reasonable, say 256KB per thread. You're still going to need an mlock
limit of more than 40MB to do anything useful.
How large is reasonable? I'm finding that for large (2 hours, 12 tracks)
ardour sessions I'm needing an mlock limit of in excess of 128 MB when
running under NPTL (I use 256 MB to be safe); otherwise ardour can't even
open the session, let alone allow any work to be done. When NPTL *isn't* in
use an mlock limit of the order of 40 MB (or even 20 MB) is perfectly fine
for the same ardour session and doesn't even result in any error/warning
messages regarding mlock failures.
Regards
jonathan
On Fri,
2005-11-25 at 11:10 +1030, Jonathan Woithe wrote:
I tried this. Although this reduced the size of the mlock calls I found I
still needed an mlock limit of at least 40MB for jack and ardour to start.
However, even with this loading a large session into ardour failed to load
with no error given (beyond the fact that the loading failed).
You mentioned in another post that you're working on a "proper" fix for
jack. I'll wait for this to be implemented before digging into this
issue further since the correct fix might change things. Drop me a line
when a fix has made it into CVS and I'll resume my testing. I have since discovered (thanks to strace) that
the problem lies in an
mmap2() call which requests a size of around 8MB. It appears to be
part of the NPTL pthread_create() function. The error returned by
mmap2() (EAGAIN) indicates either a locked file or "too much locked
memory" (according to the manpage). Because this is an anonymous map
the problem must have been the latter. Increasing the "locked memory"
limit from 20MB to 40MB made jackd start without having to resort to
the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that.
The real fix is for JACK to set a reasonable thread stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
12 Dec
12 Dec
2:37 a.m.
New subject: [linux-audio-dev] NPTL jack+ardour: large memlock required
On Mon, 2005-12-12 at 11:09 +1030, Jonathan Woithe wrote:
OK. I'm not sure what causes this. But I have two ideas.
I have looked at the NPTL (2.3.5) source, and unless you use
pthread_attr_setstack() to manage the stacks yourself, it maintains an
internal cache of thread stacks that could be related to this problem -
I didn't look hard enough to figure out at which point it starts to
free() things - possibly the cache is allowed to grow very big. And
since NPTL does not care as it assumes that thread stacks are pageable I
can see where with mlock()ed stacks this could get quite big.
Also, I've found that if a thread is created joinable (the default) and
exits but is never joined, the parent thread will leak ~200 bytes of
memory for each thread until you go OOM. Creating the threads detached
prevents the memory leak.
Lee
On Thu,
2005-12-08 at 16:50 +1030, Jonathan Woithe wrote:
Um, all my fix will do is reduce the thread stack size to something more
reasonable, say 256KB per thread. You're still going to need an mlock
limit of more than 40MB to do anything useful.
How large is reasonable? I'm finding that for large (2 hours, 12 tracks)
ardour sessions I'm needing an mlock limit of in excess of 128 MB when
running under NPTL (I use 256 MB to be safe); otherwise ardour can't even
open the session, let alone allow any work to be done. When NPTL *isn't* in
use an mlock limit of the order of 40 MB (or even 20 MB) is perfectly fine
for the same ardour session and doesn't even result in any error/warning
messages regarding mlock failures. On Fri,
2005-11-25 at 11:10 +1030, Jonathan Woithe wrote:
> I have since discovered (thanks to strace) that the problem lies in an
> mmap2() call which requests a size of around 8MB. It appears to be
> part of the NPTL pthread_create() function. The error returned by
> mmap2() (EAGAIN) indicates either a locked file or "too much locked
> memory" (according to the manpage). Because this is an anonymous map
> the problem must have been the latter. Increasing the "locked memory"
> limit from 20MB to 40MB made jackd start without having to resort to
> the LD_ASSUME_KERNEL hack.
I stumbled across the same problem a few weeks ago working on another
project. This is glibc allocating an 8MB (!) stack for each thread. It
gets the default thread stack size from getrlimit(). With mlockall()
you will go OOM really fast like that.
The real fix is for JACK to set a reasonable thread stack size in
jack_create_thread. You can work around the problem by appending this
line to /etc/security/limits.conf:
* hard stack 512
for a 512KB stack.
I tried this. Although this reduced the size of the mlock calls I found I
still needed an mlock limit of at least 40MB for jack and ardour to start.
However, even with this loading a large session into ardour failed to load
with no error given (beyond the fact that the loading failed).
You mentioned in another post that you're working on a "proper" fix for
jack. I'll wait for this to be implemented before digging into this
issue further since the correct fix might change things. Drop me a line
when a fix has made it into CVS and I'll resume my testing.
6704
days inactive
6722
days old
linux-audio-dev@lists.linuxaudio.org
10 comments
3 participants
participants (3)
-
Florian Schmidt -
Jonathan Woithe -
Lee Revell