On Tue, 2008-05-06 at 15:57 +0200, Lars Luthman wrote: 2 slightly different things .... * MIDI sync - distribution of MIDI data with sample accurate sync WRT audio data * VIDEO sync - information allowing a JACK client to know the precise audio sample at which a video frame begins (not implemented by any freely distributed JACK backend) video-jack is about providing something like MIDI sync but for video *data*.

On Tue, 2008-05-06 at 16:35 +0200, Malte Steiner wrote: You missed my precise point. VideoJACK is like the *first* of my two examples (MIDI "sync") and not like the second (Video frame onset information). There has never been any disagreement with the idea that videojack is a cool idea. What there has been is a widespread feeling in the JACK developer community that the video JACK stuff has been done suboptimally, in ways that don't reflect a good understanding of JACK's design, and without bothering to actually ask us "how should we do this?" I'd love to see videojack be a part of mainline JACK, but its not going to happen with the current set of patches that have been proposed. To be honest, I was suprised that everyone else at the JACK "summit" at LAC2008 seemed to agree with me about that. --p

Paul Davis wrote: VideoJACK does sound nice, and it would be a shame to witness forkage when things could be settled in a more productive manner. I'm in no way qualified to draft any API plans for JACK, which arguably is the perfect excuse for doing just that. ;) I've understood that the current VideoJACK approach is to run two JACK servers, one for video and one for audio. If it already works with two completely different servers, wouldn't it also work if audio and video ports just had different callbacks inside the same server? The audio callback API wouldn't need to be changed in any way. Video streaming capabilities would require adding a smallish additional API. Here's a crude suggestion: /* * Set the video frame rate to num1/num2 */ int jack_set_video_rate (jack_client_t *client, jack_nframes_t num1, jack_nframes_t num2); /* * Set the video processing callback function */ int jack_set_video_callback (jack_client_t *client, JackProcessCallback video_callback, void *arg); /* * Get the number of frames in a video port buffer */ jack_nframes_t jack_video_get_frame_count (void *port_buffer); /* * Get a frame from a video port buffer */ int jack_video_get_frame (jack_video_frame_t *frame, void *port_buffer, jack_nframes_t event_index); /* * A video frame type */ typedef struct _jack_video_frame_t jack_video_frame_t; struct _jack_video_frame_t { /* The video frame's timestamp (corresponds to an audio frame index within the same period) */ jack_nframes_t time; /* The size (in bytes) of the raw frame data */ size_t size; /* Pointer to raw video frame data. Can be of any format; it is up to the application to manage that stuff */ void *raw_data_ptr; }; What is stopping us from adding an API layer such as the one outlined above? It would seem that separate callbacks for audio and video would serve the VideoJACK crew's needs. Is there something I'm not seeing here? Juuso

Stephen Sinclair wrote: Yes, I suppose the suggestion I posted could be about streaming arbitrary blobs of data. But I'd like to hear from someone who can actually tell if the idea is viable, I was mainly following my intuition. Juuso

Juuso Alasuutari &lt;juuso.alasuutari(a)gmail.com&gt; writes: I think we should have this as part of control API instead. Unless we allow changing of sample rate on-the-fly too. I think we could reuse audio process callback. I think instead of port_buffer (void *) we should use jack_port_t. Also I think we could use jack_port_get_buffer() instead of inventing new function (jack_video_get_frame). I think we should either have frame size as part of global setup or we should leave that to reside in data pointed by raw_data_ptr (varsize frames). With jack_port_get_buffer() instead of jack_video_get_frame(), I think it would be better to have: /* Return timestamp of video frame (corresponds to an audio frame index within the same period) */ jack_nframes_t jack_video_get_frame_offset( jack_port_t * video_port, jack_nframes_t frame); -- Nedko Arnaudov <GnuPG KeyID: DE1716B0>

torbenh(a)gmx.de writes: I think with PAL (25Hz), you have one frame each 40ms. With interlaved video you get 20ms. With not low latency setup you will get more than one frame almost always. I dont get why there should be separate thread for video. I think it should be processed in same libjack thread. If you cannot process the video you will get xrun. Just as with audio. So you probably need to get faster CPU or decrease CPU usage in some way. The only reason I can imagine is to have separate audio and video xruns semantics. So you can drop frame without droping audio. -- Nedko Arnaudov <GnuPG KeyID: DE1716B0>

Juuso Alasuutari: Correct me please, if I'm wrong: jack's distinct feature seems to be the n to n plumbing plus internal summing if needed, not the streaming. To make video plumbing any interesting I reckon you'd want something like Fusion or Blender's node system. Nothing of which, just a hunch, will ever end up in jack graphs. Why not keep things simple and use jack's (maybe to be enhanced) timing information as a studio clock to sync video frames? Meaning what "videojack" really wants to be is a jack audio client with good video-specific plumbing on its own. Wolfgang

Paul Davis: Couldn't a video app use jack's audio time information to trigger video frames? Dummy backend and voila? What am I missing? Wolfgang

Hi, I have a project which requires a python programmer to do paid work on a temporary contract to provide a desktop interface with gtk running on a Linux box. If you are interested in applying please send me a message off list and I will get back to you privately. Cheers. -- Patrick Shirkey Boost Hardware Ltd.