13 Dec
2011
13 Dec
'11
4:24 a.m.
Hi everyone, I have a fairly simple, known size message format, it's just
five numbers, either ints or floats, wrapped in a structure. I guess I need
some kind of serialization to send this over a jack ringbuffer, but I've
zero experience with serialization in C++. Can anyone tell me what the
easiest or best way to do this is? Should I use the boost serialization
library? FWIW, I would like to eventually add osc messaging in too if that
affects the best choice.
Thanks!
Iain
13 Dec
13 Dec
4:31 a.m.
On Tue, Dec 13, 2011 at 3:24 AM, Iain Duncan <iainduncanlists(a)gmail.com>wrote;wrote:
I guess I need some kind of serialization to send this
over a jack
ringbuffer, but I've zero experience with serialization in C++.
I don't really understand what your asking here, you want to be able to set
the order of the messages in the ringbuffer?
Its a FIFO queue, as in First In - First Out. So the order of the messages
is the same as you write them...
Perhaps I'm misunderstanding you :S
W.r.t OSC, there's 2 ways to run an OSC server, polling or having
callbacks. If you have callbacks it involves a having a seperate OscServer
thread. This means you'll need to use a seperate RingBuffer too, as the
JACK ringbuffer is a SRSW queue. (single read, single write)
If you're using the GUI thread to poll the OscServer, then you can use the
same RingBuffer and you'll have more control over the order of GUI / OSC
events...
Then again, if you can do the "ordering" on the recieving side of the
RingBuffers, then it really makes no difference :D
All the best, -H
5:44 a.m.
On Mon, Dec 12, 2011 at 7:31 PM, Harry van Haaren <harryhaaren(a)gmail.com>wrote;wrote:
On Tue, Dec 13, 2011 at 3:24 AM, Iain Duncan
<iainduncanlists(a)gmail.com>wrote;wrote:
I guess I need some kind of serialization to send
this over a jack
ringbuffer, but I've zero experience with serialization in C++.
I don't really understand what your asking here, you want to be able to
set the order of the messages in the ringbuffer?
Its a FIFO queue, as in First In - First Out. So the
order of the messages
is the same as you write them...
Perhaps I'm misunderstanding you :S
Yup, what I'm talking about is being able to put a data structure on to the
ring buffer. It needs be castable to a const *char, so the structure needs
a way to be converted to a string. I can't just put my DataMessage
structure on there because there is no automatic conversion from my own
struct to a string. I know how to do this in Python, but not sure what the
best way to do it in C or C++ is.
thanks
Iain
2:53 p.m.
On Mon, Dec 12, 2011 at 11:44 PM, Iain Duncan <iainduncanlists(a)gmail.com> wrote:
Yup, what I'm talking about is being able to put a
data structure on to the
ring buffer. It needs be castable to a const *char, so the structure needs
a way to be converted to a string.
these two statements are not related. in an awful lot of C code,
"pointer to char" means "pointer". in newer better C code, one uses
void*. in newer, better code than that, one doesn't use raw pointers
much at all.
are you working in C or C++ ?
14 Dec
14 Dec
4:02 a.m.
On Tue, Dec 13, 2011 at 5:53 AM, Paul Davis <paul(a)linuxaudiosystems.com>wrote;wrote:
On Mon, Dec 12, 2011 at 11:44 PM, Iain Duncan
<iainduncanlists(a)gmail.com>
wrote:
Thanks Paul. I'm working in C++, but I'm using the jack C api, which from
the docs I see has a signature for
size_t jack_ringbuffer_write ( jack_ringbuffer_t * rb, const char * src,
size_t cnt )
My DataMessage structure is just a simple C structure for now. Is there a
recommended way of writing it to the ringbuffer given that I want to do
something like this:
void MessageQueue::push( DataMessage msg ){
// write to the ring buffer, converting DataMessage to a string
unsigned int written = jack_ringbuffer_write( mRingBuffer, (char *)
&msg , sizeof(DataMessage) );
// etc
}
Thanks
Iain
Yup, what I'm talking about is being able to
put a data structure on to
the
ring buffer. It needs be castable to a const
*char, so the structure
needs
a way to be converted to a string.
these two statements are not related. in an awful lot of C code,
"pointer to char" means "pointer". in newer better C code, one uses
void*. in newer, better code than that, one doesn't use raw pointers
much at all.
are you working in C or C++ ?
4:33 a.m.
On Tue, Dec 13, 2011 at 10:02 PM, Iain Duncan <iainduncanlists(a)gmail.com> wrote:
Thanks Paul. I'm working in C++, but I'm using
the jack C api, which from
the docs I see has a signature for
size_t jack_ringbuffer_write ( jack_ringbuffer_t * rb, const char * src,
size_t cnt )
My DataMessage structure is just a simple C structure for now. Is there a
recommended way of writing it to the ringbuffer given that I want to do
something like this:
void MessageQueue::push( DataMessage msg ){
// write to the ring buffer, converting DataMessage to a string
unsigned int written = jack_ringbuffer_write( mRingBuffer, (char *) &msg
, sizeof(DataMessage) );
// etc
}
as long as the struct is POD (Plain Old Data - no embedded pointers,
etc), this will work fine.
however, you need to keep in mind that under some easily encounterable
circumstances, the write may not return sizeof(DataMessage). this is a
very easy mistake to make with ringbuffers (ditto for read).
it can be avoided via careful sizing of the ringbuffer and always
read/writing "whole objects" OR by carefully checking the results of
read/write.
the jack ringbuffer design is particular bad in this respect because
it can only hold size-1 bytes (where size is its actual size).
ardour has some slightly better behaved, templated ringbuffers for this purpose.
4:59 a.m.
My DataMessage structure is just a simple C structure
for now. Is there a
recommended way of writing it to the ringbuffer given that I want to do
something like this:
void MessageQueue::push( DataMessage msg ){
// write to the ring buffer, converting DataMessage to a string
unsigned int written = jack_ringbuffer_write( mRingBuffer, (char *)
&msg
, sizeof(DataMessage) );
// etc
}
as long as the struct is POD (Plain Old Data - no embedded pointers,
etc), this will work fine.
however, you need to keep in mind that under some easily encounterable
circumstances, the write may not return sizeof(DataMessage). this is a
very easy mistake to make with ringbuffers (ditto for read).
it can be avoided via careful sizing of the ringbuffer and always
read/writing "whole objects" OR by carefully checking the results of
read/write.
Thanks for the tips. What should one do if one detects a partial write? Is
it best to have integrity checks on both ends of the ringbuffer?
the jack ringbuffer design is particular bad in this respect because
it can only hold size-1 bytes (where size is its
actual size).
Does this mean a good way to initialize it is to make the ringbuffer some
multiple of the sizeof(myMessageStruct) plus one byte?
Thanks again for all the help.
iain
8:27 a.m.
New subject: easiest way to serialize messages for sending over a ringbuffer?
Thanks for the tips. What should one do if one detects
a partial write? Is it best to have integrity checks on both ends of the ringbuffer?
Avoid them. There should be some calls for checkRingBufferWriteSpace(), make sure it is
large enough to cater for your object size. You still have a few hoops and jumps to go
through if the buffer is filling up but it is definitely going to be easier that catering
for partial writes. Consider also logging messages when it is full and if it happens (too
much or in my opinion at all) then look into why the reader is going too slow. If it is a
general issue then you need to rearchitect what you are doing, if it is just a transient
issue then bang more capacity on the ringbuffer.
Regards, nick.
8:37 p.m.
Thanks for the
tips. What should one do if one detects a partial
write? Is it best to have
integrity checks on both ends of the ringbuffer?
Avoid them. There should be some calls for checkRingBufferWriteSpace(),
make sure it is large enough to cater for your object size. You still have
a few hoops and jumps to go through if the buffer is filling up but it is
definitely going to be easier that catering for partial writes. Consider
also logging messages when it is full and if it happens (too much or in my
opinion at all) then look into why the reader is going too slow. If it is a
general issue then you need to rearchitect what you are doing, if it is
just a transient issue then bang more capacity on the ringbuffer.
15 Dec
15 Dec
12:05 a.m.
New subject: easiest way to serialize messages for sending over a ringbuffer?
if I understand what you're saying, the producer
should check for sizeof(MyStruct) space before writing,
Yes. Partial writes are possible but are going have to take care of when you write the
remainder, and also cater for partial reads as well. These would introduce some ugly
timing issues between producer and consumer.
and just delay writing that message if there isn't
enough space. I guess that's where I would log that there a write failed then?
Well, saying 'just' might be an understatement - there are issues here too
but they should be less that writing partial structures.
There was mention of writing pointers to buffers, I would avoid this as well for the
possible case that you want to mount your ringbuffers in shared memory between processes
rather than between threads. It might not be your goal now but it would be painful if that
changed in the future.
Regards, nick.
14 Dec
14 Dec
10:52 p.m.
On Mon, Dec 12, 2011 at 10:24 PM, Iain Duncan <iainduncanlists(a)gmail.com> wrote:
Hi everyone, I have a fairly simple, known size
message format, it's just
five numbers, either ints or floats, wrapped in a structure. I guess I need
some kind of serialization to send this over a jack ringbuffer, but I've
zero experience with serialization in C++. Can anyone tell me what the
easiest or best way to do this is? Should I use the boost serialization
library? FWIW, I would like to eventually add osc messaging in too if that
affects the best choice.
Can you send a pointer to your struct over the ringbuffer instead?
11:27 p.m.
On Wed, Dec 14, 2011 at 4:54 PM, Paul Davis <paul(a)linuxaudiosystems.com> wrote:
On Wed, Dec 14, 2011 at 4:52 PM, Paul Coccoli
<pcoccoli(a)gmail.com> wrote:
Agreed about the ownership rules, but I don't think it's that tricky.
Use one ringbuffer to pass pointer into the RT thread and another to
pass them back to the "main" (non-RT) thread. The non-RT thread is
then responsible for all memory management. You wouldn't call
free/delete in an RT thread anyway, would you?
Can you send a pointer to your struct over the
ringbuffer instead?
only if you establish very clear rules about buffer ownership. that
can be tricky to do.
4757
days inactive
4758
days old
linux-audio-dev@lists.linuxaudio.org
12 comments
5 participants
participants (5)
-
Harry van Haaren -
Iain Duncan -
Nick Copeland -
Paul Coccoli -
Paul Davis