<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style>
</head>
<body class='hmmessage'><div dir='ltr'>
> From: clemens@ladisch.de<br><div>> To: nickycopeland@hotmail.com<br>> CC: d@drobilla.net; linux-audio-dev@lists.linuxaudio.org<br>> Subject: Re: [LAD] Pipes vs. Message Queues<br>> <br>> Nick Copeland wrote:<br>> > > I got curious, so I bashed out a quick program to benchmark pipes vs<br>> > > POSIX message queues. It just pumps a bunch of messages through the<br>> > > pipe/queue in a tight loop.<br>> <br>> This benchmark measures data transfer bandwidth. If increasing that<br>> were your goal, you should use some zero-copy mechanism such as shared<br>> memory or (vm)splice.<br>> <br>> > You might be running into some basic scheduler weirdness here though<br>> > and not something inherently wrong with the POSIX queues.<br>> <br>> The difference between pipes and message queues is that the latter are<br>> typically used for synchronization, so it's possible that the kernel<br>> tries to optimize for this by doing some scheduling for the receiving<br>> process.<br><br>Not sure about that. The CPU(95%) was all in the kernel, not in the process <br>itself so any improvements to what it scheduled for the process would only<br>translate into a small percentage difference. Isn't it more likely that the pipe<br>code is using an inefficient kernel lock on the pipe to ensure it is thread safe?<br>Please don't misunderstand my 'not sure about', I am relieved to say I am not <br>a kernel programmer but understanding these kinds of limitations is interesting<br>as it bears directly on application implementation (see below).<br><br>> > The results with 1M messages had wild variance with SCHED_FIFO,<br>> <br>> SCHED_FIFO is designed for latency, not for throughput. It's no<br>> surprise that it doesn't work well when you have two threads that both<br>> want to grab 100 % of the CPU.<br><br>Dave can comment on what he wanted to actually achieve, I was interested in <br>whether the results could be shown to be general. I take your points on the use<br>of SCHED_FIFO but there are still some weirdness<br><br>> It's no surprise that it doesn't work well<br><br>It does work very well, just not with piped messages.<br><br>> when you have two threads that both want to grab 100 % of the CPU<br><br>My system does have 200% available though, it's was dual core and the question<br>I raised was why there is a scheduling problem between the two separate threads<br>with pipes whilst it could be demonstrated that there was no real need to have <br>such contention.<br><br>Perhaps I should revisit another project I was working on which was syslog event<br>correlation: it used multiple threads to be scalable to >1M syslog per second<br>(big installation). I was testing it with socketpair()s and other stuff. I would be<br>interested to know if scheduler changes affect it too.<br><br>I actually quite like your idea of shared memory - dump a ringbuffer over that and<br>it could give interesting IPC. Am not going to test that as it would be a significant<br>change to Dave's code but on the Intel platform it could give some very high<br>performance without the need for any recursion to the kernel. The event correlator<br>would not benefit from the use of shmem since it was threaded, not multiprocessed.<br><br>Kind regards, nick<br></div> </div></body>
</html>