On Thu, 2004-07-15 at 04:53, Damien Cirotteau wrote:
> On Thu, Jul 15, 2004 at 10:35:47AM +0200, Free Ekanayaka wrote :
> > |--==> "DC" == Damien Cirotteau <damien.cirotteau(a)agnula.org> writes:
> >
> > DC> Hi all,
> > DC> some very interesting links submited by simmo on IRC
> >
> > DC> File system:
> > DC> http://kerneltrap.org/node/view/3466
> >
> > DC> Voluntary Kernel Preemption:
> > DC> http://kerneltrap.org/node/view/3440
> >
> > Yes these are definitively interesting!
> >
> > I'll read them carefully and see if we can adopt some new trick :)
>
> Two first thing that i can retain:
> - Don't use reseirfs for now (especially with 2.6)
> - Enable CONFIG_SND_DEBUG in the kernel so it will be easier to trace
> the xruns. It looks that it is a very useful feature to understand where
> the xruns comes from and is very good feedback to send to kernel folks
The intrepid user in need of low latency should try 2.6.8-rc1-mm1. I
sent a bunch of XRUN traces to Andrew Morton, and this kernel contains
numerous latency fixes as a result. Some of these fixes were to
extremely important areas (get_user_pages, a fix for
framebuffer-scrolling issue, and more). The improvement should be
major. He is going to be unavailable until July 26th, so the next few
weeks are a great opportunity to test and collect data.
You are correct in that reiserfs as shipped with the kernel should
probably not be used for now if you need low latency. However, I
strongly suspect that SuSe (which users reiser by default) contains
fixes for this which are not in the mainstream kernel. By enabling
CONFIG_SND_DEBUG *and* turning on XRUN debugging at runtime (echo 2 >
proc/asound/cardX/pcmX/xrun_debug) you will be able to see whether this
is affecting you.
Here is the announcement:
http://www.ussg.iu.edu/hypermail/linux/kernel/0407.1/1453.html
Lee
Paul Davis <paul(a)linuxaudiosystems.com> wrote:
>
> >It's too bad that the multimedia community didn't participate
> >much during the 2.5.xx development leading up to 2.6.0. If they
> >had done so, the situation might be different today. Fortunately,
> >fixing up the multimedia problems isn't too risky to do during
> >the stable 2.6.xx series.
>
> I regret that this description is persisting here. "We" (the audio
> developer community) did not participate because it was made clear
> that our needs were not going to be considered. We were told that the
> preemption patch was sufficient to provide "low latency", and that
> rescheduling points dotted all over the place was bad engineering
> (probably true). With this as the pre-rendered verdict, there's not a
> lot of point in dedicating time to tracking a situation that clearly
> is not going to work.
No, this is wrong. 2.6+preempt can satisfy your latency requirements
without any scheduling points. All it requires is that the long-held locks
be addressed. I've already done a metric ton of work in that area (notably
removal of the buffer_head LRUs and rewriting the truncate code) but more
apparently remains to be done. We know that reiserfs has problems.
But what can I do? I set up a preempt-on-ext3 test box, thrash the crap
out of it and see 300 usecs worst-case latency. So I am left empty-handed,
wondering what on earth is happening out there.
I am deeply skeptical about claims that autoregulated swappiness can make
any difference.
I am deeply skeptical about claims that CPU scheduler changes make any
difference. A scheduler change shouldn't improve responsiveness of
!SCHED_OTHER tasks at all, so perhaps there are application priority
inversion problems, or applications aren't setting SCHED_FIFO/RR correctly.
I do not know.
I am also fairly skeptical about claims that voluntary-preempt helps,
because it only pops a couple of locks, and I doubt that testers are
hitting the code paths which those changes address anyway.
So Something Is Up, and I don't know what it is.
Please double-check that there are no priority inversion problems and that
the application is correctly setting the scheduling policy and that it is
mlocking everything appropriately.
And please ensure that people are setting xrun_debug, and are sending
reports.
> The kernel is not going to provide adequate latency for multimedia
> needs without either (1) latency issues being front and center in
> every kernel developer's mind, which seems unlikely and/or (2)
> conditional rescheduling points added to the kernel, which appears to
> require non-mainstreamed patches.
>
Nope, the conditional rescheduling points provide zero benefit on a
preemptible kernel.
Something weird is happening, I don't know what it is, I cannot reproduce
it and I need help understanding what it is, OK? The sooner we can do
that, the sooner it gets fixed up.
Greetings:
As promised, here's a set of test criteria used by Alan Belkin in his
1994 review of notation programs for the Macintosh. I hope that the
authors of Linux music notation software will consider this list against
the features of their own efforts.
I'm not interested in comparing "ours against theirs". The Mac
programs tested were all WYSIWYG notation editors, including Finale,
Composer's Mosaic, Encore, Lime, and Nightingale, while some of the best
Linux music notation software is devoid of any GUI. Nevertheless, the
criteria seem adequate as base requirements for any music notation
software, and I'm very interested in the opinions and evaluations of the
Linux developers of such software. I know that the authors of NoteEdit,
LilyPond, MusE/Musescore, Denemo, Rosegarden, Common Music Notation, and
perhaps other significant notation editors are represented on the
LAD/LAU lists, and I hope they will respond on-list to the criteria
presented here. I also welcome comments from users regarding the
presence or absence of the listed features in their favorite Linux
notation program.
I have only slightly altered Mr Belkin's original criteria where it
was Mac-specific. The evaluations in his original article were either
qualitative (good, bad, ugly, etc), quantitative, (1, 4, 12, etc), or
affirmative/negative (yes/no). So, here we go:
Note entry:
mouse & keyboard
MIDI step-time
MIDI realtime w. flexible quantization
audition other saves while recording
retain performance data for playback
number of independent rhythmic layers per staff
maximum number of staves per system
Entry of slurs, articulations, dynamics, etc.:
intelligent default placement
apply to multiple staves at once
Selection in regional edits:
vertical, horizontal slices within and across measures, staves,
system, pages, etc.
non-contiguous
conditional selection
Editing:
click & drag positioning of symbols
transposition (note, staff, selection, etc)
enharmonic change by region
rhythm: change note values (ease of use)
rhythm: auto-rebar
cut/copy/paste: music
cut/copy/paste: non-musical items, formats, etc.
mirroring (intelligent copies)
Special/custom notation:
unusual staves
simultaneous key signatures
unconventional time signatures
additive time signatures
simultaneous different time signatures
drawing tool
user-created symbols
user-selectable fonts for all elements
chord notation: graphic, playback, learn via MIDI
fretboard notation
figured-bass notation
unusual note heads (slashes, harmonics, etc)
easily adjustable cross-staff beaming
Lyrics:
mass create
create on page
import from text editor
auto layout
multiple fonts
flexible placement
MIDI playback:
ALSA or OSS support
channel support
playback includes modifiers (crescendi, dynamics, etc)
direct editing of MIDI data
import patch lists (GM, GS, etc)
scrolling playback
edit during playback
Entry layout:
flexible engraver spacing within measure
account for dynamics, slurs, annotative text, etc.
Page layout:
auto layout with engraver spacing
reduce or enlarge symbols, staves, text, systems, by any percent,
locally or globally
full control of measures per system
full control of systems per page
remove empty staves within systems
flexible spacing of staves within systems
Part extraction:
automatic with new layout
dynamic links to master score
File operations:
follow Linux standards (?)
simultaneous multiple files open
printed output: PS, PDF, DVI, etc.
Interface/overall ease of use:
undo/redo any operation
user-defined key bindings
user control over notational defaults
views: scroll, page, template, any percent, multiple simultaeous views
priorities clear
logical organization
simple language and icons
overall speed
on-line help
documentation
ease of learning
general solidity and stability
In his article Mr Belkin also addressed the problem of tuplets, noting
that at that time only Finale realized anything other than triplets when
converting from MIDI input (file or realtime). I should also note that
this list is hardly meant to be a complete set of expected features:
after all, it's from an article published ten years ago. I'm sure we've
advanced well beyond the state of the art in 1994... right ? :)
Best regards,
Dave Phillips
Hello,
aaron(a)nquit.com wrote:
> And did you find RoseGarden and Ardour we're sync'd up tightly?
> I have yet to find them REALLY syncing really tightly....
>
> what distro are you on? what versions of the software? what hardware?
Sorry for not replying earlier. We wanted to do some tests first, and that
got delayed a little.
About syncing: first of all, we have been using JACK for the sync
mechanism. We also tried using MTC, but we could not get that to work.
Syncing seems to work okay, in the sense that things keep in sync over
time, but we had to adjust offsets to correct the initial sync.
We made some tests to get a more clear picture, and in fact we got
rather confused instead. I will just give you the results, without
trying to explain what happens. I am sure some of you will be able to
shed more light on these results. And maybe they can help
ardour/jack/rosegarden developers in some way or another
Test setup:
- jack configured at 44100 Hz, and we tried with 2 buffers of 2048 and
2 buffers of 4096
- A MIDI track in rosegarden4, with a beat at 120 BPM.
- MIDI output (USB MIDI-Sport 2x2) connected to an external MIDI module
- Audio output from the external MIDI module connected to an Emagic EMI
2|6 USB audio device.
- In Ardour, we create two tracks, one to record the audio from the MIDI
module, one to record the ardour click (jack loopback).
I know the buffersize of jack is very big, but for doing this test, the
larger audio latency makes things more obvious. Also, we have not been
able to use the EMI 2|6 with ALSA with a period smaller then 2048
without strange ticks. I should contact the ALSA list about this, but on
the other hand, this device is a horrible piece of overpriced plastic
crap with lousy connectors, that I would not recommend to anyone even with
smaller buffer-sizes...
Results:
About the recorded (jack loopback) ardour-click: Strangly enough, the
first click occurs before time=0. Looking at the second click, which
should be at 0.5 sec = 22050 frames, we can see how much the clicks
are ahead of time:
- jack@2x2048: click at 20002 frames; offset = -2048 frames
- jack@2x4096: click at 17954 frames; offset = -4096 frames
Indeed: exactly 1 buffer. Strange, isn't it?
More complicated is the recorded MIDI click. These occur even BEFORE the
recorded jack clicks. Now, I am not 100% certain if maybe we set the
delay of the MIDI track in Rosegarden (I don't have access to the
machine right now, but I will tell you as soon as I do), but I don't
think so, and even so, the results are strange.
- jack@2x2048: click at 18450 frames; offset = -3600 frames
- jack@2x4096: click at 14100 frames; offset = -7950 frames
I don't see the relation between those numbers...
Versions used:
Hardware
* Emagic EMI 2|6
* USB Midi-Sport 2x2 (in a Steinberg box)
* Acer Travelmate 800
Sofware
* Kernel 2.4.26
- patches: Preemptive 2.4.26-pre5-1, RTC 2.4.25, 2.4.25-low-latency
* Alsa 1.0.5a
* Jack 0.98.1
* Ardour 0.9 Beta 17.1 (Ardour/gtk 0.520.9, libardour 0.820.1)
* Rosegarden4 0.98
Maarten
Lee revel wrote:
>> Note that this info because available because someone set
>> /proc/asound/*/*/xrun_debug. We need more people doing that.
>> -
>This goes back to the need for ALSA documentation. Someone needs to
>write some. This will probably require paying that person. Hopefully
>SuSe is working on this, though I suspect I would have heard something.
Nope. It requires someone letting people like me know about it.
I will add the info to somewhere accesble and obvious RSN.
--
Patrick Shirkey - Boost Hardware Ltd.
Http://www.boosthardware.comHttp://www.djcj.org/LAU/guide/ - The Linux Audio Users guide
Http://www.djcj.org/gigs/ - Gigs guide Korea
Http://www.nana7.net - Bar Nana - Itaewon, Seoul, Sth Corea
========================================
Apparently upon the beginning of the barrage, the donkey broke
discipline and panicked, toppling the cart. At that point, the rockets
disconnected from the timer, leaving them strewn around the street.
Tethered to the now toppled cart, the donkey was unable to escape before
the arrival of U.S. troops.
United Press International
Rockets on donkeys hit major Baghdad sites
By P. MITCHELL PROTHERO
Published 11/21/2003 11:13 AM
Yes! i was just about to write the list this very same message!
-------
NQuit
www.nquit.com
-------
----- Original Message -----
From: Dave Phillips
Sent: 7/13/2004 11:43:00 AM
To: linux-audio-dev@music.columbia.edu;linux-audio-user@music.columbia.edu
Subject: [linux-audio-dev] Ardour named Project Of The Year by LJ
> Greetings Earthlings:
>
> Just a quick note to say that the August 2004 issue of the Linux
> Journal has selected Ardour as Project Of The Year for its 2004 Editors
> Choice awards. Congratulations to Paul and everyone on the team !
>
> Best regards,
>
> dp
>
>
Hi,
I'm writing a pattern-based audio sequencer, and wonder about the file
format to choose when saving the whole song. My application handles the
usual stuff : metadata (bpm, settings), patterns, and samples.
Do you believe I should use (yet another) binary file format, embedding
patterns, samples, and possibly roasted chicken into a big standalone
(that's the good point) package ?
Or may I produce an XML file (doesn't <pattern> looks nicer than
[pattern] ? ;), with an option to mimic the save "Web-page, Complete"
mozilla behaviour ?
That is, when saving song.xml, sticking a song.xml.files/ folder in the
same directory, to store samples.
--
og
Maged Michael from IBM, who is publishing lots of practical lock-free
algorithms these days, has just published a paper describing a
lock-free memory allocation algorithm:
http://www.research.ibm.com/people/m/michael/pldi-2004.pdf
It seems plausible that you could use this to safely allocate memory
from RT threads. The only questions I have about its practicality are:
1. how easy/possible is it to use two malloc() implementations in the
same program? My brief research suggests that mixing system malloc(3)
and sbrk(2) (the latter is the underlying mechanism for obtaining more
memory from the OS) is not guaranteed to be safe. A possible solution
I have encountered is to obtain memory from the OS by mmap(/dev/zero)
instead of using sbrk(2).
2. When your lock-free malloc needs more memory from the OS it will
still take a system call to do it. I believe I have heard it said in
the past that system calls of any sort are unacceptable in RT code, but
isn't this a bit a of a hard-line position?
Josh
From: "Bill Huey (hui)" <bhuey(a)lnxw.com>
> On Tue, Jul 13, 2004 at 11:44:59PM +0100, Martijn Sipkema wrote:
> [...]
> > The worst case latency is the one that counts and that is the contended case. If
> > you could guarantee no contention then the worst case latency would be the
> > very fast uncontended case, but I doubt there are many (any?) examples of this in
> > practice. There are valid uses of mutexes with priority inheritance/ceiling protocol;
> > the poeple making the POSIX standard aren't stupid...
>
> There are cases where you have to use priority inheritance, but the schemes that are
> typically used either have a kind of exhaustive analysis backing it or uses a simple
> late detection scheme. In a general purpose OS, the latter is useful for various kind
> of overload cases. But if your system is constantly using that specific case, then it's
> a sign the contention in the kernel must *also* be a problem under SMP conditions. The
> constant use of priority inheritance overloads the scheduler, puts pressure on the
> cache and other negative things that hurt CPU local performance of the system.
>
> The reason why I mention this is because of Linux's hand-crafted nature of dealing
> with this. These are basically contention problems expressed in a different manner.
> The traditional Linux method is the correct method of deal with this in a general
> purpose OS. This also applies to application structure as well. The use of these
> mechanisms need to be thought out before application.
To be honest, I don't understand a word of what you are saying here. Could you
give an example of a ``contention problem'' and how it should be solved?
> > > > It is often heard in the Linux audio community that mutexes are not realtime
> > > > safe and a lock-free ringbuffer should be used instead. Using such a lock-free
> > > > ringbuffer requires non-standard atomic integer operations and does not
> > > > guarantee memory synchronization (and should probably not perform
> > > > significantly better than a decent mutex implementation) and is thus not
> > > > portable.
> > >
> > > It's to decouple the system from various time related problems with jitter.
> > > It's critical to use this since the nature of Linus is so temporally coarse
> > > that these techniques must be used to "smooth" over latency problems in the
> > > Linux kernel.
>
> > Either use mutexes or POSIX message queues... the latter also are not
> > intended for realtime use under Linux (though they are meant for it in
> > POSIX), since they don't allocate memory on creation.
>
> The nature these kind of applications push into a very demanding space where
> typical methodologies surrounding the use of threads goes out the window. Pushing
> both the IO and CPU resources of a kernel is the common case and often you have to
> roll your own APIs, synchronization mechanisms to deal with these problem. Simple
> Posix API and traditional mutexes are a bit too narrow in scope to solve these
> cross system concurrency problems. It's not trivial stuff at all and can span
> from loosely to tightly coupled systems, yes, all for pro-audio/video.
>
> Posix and friends in these cases simply aren't good enough to cut it.
I find this a little abstract. Sure, there might be areas where POSIX doesn't supply
all the needed tools, e.g. one might want some scheduling policy especially for
audio, but to say that POSIX isn't good enough without providing much
explanation...
--ms