[LAD] memory allocation for the real time thread, opinions wanted

[Tim Goetze]

[Adrian Knoth]

>Iain Duncan <iainduncanlists at gmail.com> wrote:
>
>Hi
>
>>- I could pre-allocate a giant list of messages and pluck the data off
>>that
>>list when I need to make a new one
>>- I could pre-allocate a block of memory and allocate off that
>
>I'm writing from my phone, so for the sake of brevity, I will only 
>talk about one option you might have missed:
>
>Google for "A Study in Malloc: A Case of Excessive Minor Faults - USENIX" by Ezolt.
>
>You can instruct your malloc implementation to operate on a 
>previously allocated (and prefaulted) memory pool. free() will then 
>never return the memory to the OS, but back to this memory pool.
>
>This way, if you have a rough estimation for your overall memory 
>requirements, you simply allocate it once in a non-real-time context 
>and then use malloc/free at will.

This, I'm afraid, is an idea with a fair share of issues.  It does not 
seem a trivial problem to estimate the total memory needs of a 
reasonably complex application.  And even if you can solve this, you 
will find that common malloc implementations (such as the one in 
glibc) protect the heap with mutexes.  Regardless of memory 
availability, your real-time thread can always stall in malloc() or 
free() when a heap mutex is found locked by another thread.

As for the original question, I'd suggest handling the maintenance and 
memory management of large lists of events in low-priority, 
low-frequency threads.  These threads periodically feed the realtime 
thread bursts of a small number of events, ahead of time and using 
preallocated FIFO queues which can be of reasonably modest size.  (In 
fact, I seem to remember that this approach has been discussed on this 
list recently, if perhaps briefly.)

Cheers, Tim