15 Feb
2007
15 Feb
'07
11:12 a.m.
Hi Pau!
I'll excerpt a bit out of order here:
On Wed, 2007-02-14 at 15:56 +0100, Pau Arumi wrote:
I wanted to start looking on how gstreamer can be run
(and
synchronized to other streams) with a callback-based system. So could
point-me to relevant code of the work already done, design docs, mail
discussions, whatever? Also, I'm not very sure about which list this
discussion should belong.
GStreamer has two basic modes, push and pull. Push is optimized for the
case when you are dealing with a container format, and don't know
exactly what will come out of the format. The source starts a thread,
pushing buffers, and the rest of the pipeline deals with buffers as they
come.
An audio sink in push mode spawns an additional thread to write to the
audio device. It communicates with the rest of the pipeline via a ring
buffer.
Multiple sinks within the same pipeline are typically driven from
separate threads. Synchronization is easy with multiple threads: a clock
is selected somehow and the sinks perform blocking waits on the clock
until it is time to render their data. In the audio sink case it is the
thread feeding the ring buffer that performs the blocking wait.
Typically, the audio sink also provides the pipeline clock based on how
many samples have been played.
Pull mode, more appropriate to this list ;), has to be explicitly
supported by elements. It's a pain to implement a decoder such that you
can ask for 64 frames of audio data, which is why they normally run in
push mode. Pull mode in GStreamer is more appropriate to when dealing
with a format with a constant number of bytes per unit time. In pull
mode, the sink is the one that spawns the thread, driving the pipeline
by pulling N bytes as it can. The process of thread spawning is generic
enough so that the jack process() callback can run the gstreamer
pipeline directly, pulling out only the number of samples that it wants.
In the past there have been other scheduling mechanisms in GStreamer
(cothreads, notably), but hopefully we have seen the last of those.
gstreamer-devel(a)lists.sourceforge.net would be the list to continue a
discussion, as it's probably more specific than this list is interested
in. The best written documents we have, which are only slightly out of
date, are these:
http://webcvs.freedesktop.org/gstreamer/gstreamer/docs/design/part-overview…
in the section "Dataflow and buffers", as an overview;
http://webcvs.freedesktop.org/gstreamer/gstreamer/docs/design/part-activati…
as how the threads get spawned or not, and how the scheduling method
is chosen; and
http://webcvs.freedesktop.org/gstreamer/gstreamer/docs/design/part-negotiat…
for some trickiness regarding format ("caps") negotiation.
On Wed, 2007-02-14 at 15:56 +0100, Pau Arumi wrote:
I see in your blog that pull-mode in gstreamer needs
code-review. I'm
interested to help since I'm planning to do some research on
integrating
data-flows of different nature during the next few months for my PhD.
Later I can give more details on what I want to achieve. It's likely
that integrating other systems with CLAM will be one of the outcomes.
Interesting! Most of the review that it needs is GStreamer-specific: how
to ensure optimal behavior appropriate for audio synthesis in a very
general framework. It's not an area that has been well-explored from the
GStreamer side, although I've tried to make sure that the current stable
series can support efficient pull-mode in a compatible manner. So
getting use cases, ideas, analysis, and example apps would be useful
indeed.
PS: in your blog you talk a lot about Catalan things:
are you also
based in Barcelona? If so, why don't we meet and talk about that?
I am indeed, although my Catalan is not yet up to standard ;-) In fact I
think we're only a few blocks away. I'll follow with a private mail :)
Cheers,
Andy.
--
http://wingolog.org/