2 Mar
2011
2 Mar
'11
6:34 p.m.
On Wed, 2011-03-02 at 14:20 +0100, Olivier Guilyardi wrote:
The UI would draw into a rectangular area which could:
- either be embedded in a larger GL scene, if the host UI is GL based,
- or be displayed by QGLWidget, rendered into a GdkGLWindow, etc..
Also, what's rather funny is that there would be some kind of symmetry between
the run() and draw() functions exposed by a plugin. One for audio, the other for
graphics/video... And both called in a cyclic manner by the host.
Plugin and UI are not the same thing, nor should they be.
OK, well, do you have any active plans to do OpenGL plugin UIs any time
soon?
In one sense. The problem it's solving also shows some of its
weaknesses.
Hmm, own web server, I don't know really.. This sounds a bit bloated to me.
... Yeah, that would be the obvious reason. ;)
And how does one draws a frequency curve, even one which doesn't update in
realtime? With SVG? Do you see how heavy this is all going to be?
Yep. I see how heavy it is, and I also see that it is the only
technology stack you have, if you want to do this kind of thing. I also
see it's getting a whole lot less heavy extremely rapidly.
Anyway, go look at some modern canvas experiments. Drawing a curve
quickly is well within what works fine right now. Drawing and animating
a whole ton of them smoothly is current reality.
Why you are trying to pick apart web UIs in the same email as you're
arguing where one size does not fit all I don't know... I want a remote
control that works on any device out of the box. It's about as blatantly
obvious as anything can be that web is literally the only way to go for
that, because the browser is actually on those devices. QED.
Do I think /all/ desktop PC hosts and plugin should use it? No. It's for
"remote control" things (even if not remote). Is GL good? Yes. I sure
wish I used it for the Ingen canvas. Has anyone actually showed up yet
who wanted to write a GL UI? No. Unless you're one, talking about it is
a waste of time.
Not sure what you mean here, but I'm a big No Silver Bullet fan, and generally I
don't believe in no-need-to-code-anymore claims.
You just mentioned controller apps where users build their own UIs
(touchOSC etc), sooo... I guess you don't believe in reality? ;)
On 03/02/2011 12:22 AM, David Robillard wrote:
No, not really.
Apart from mobile web apps, a way to build cross-platform apps is using a
lightweight toolkit, with OpenGL for all graphics, and a couple of other entry
points such as an audio I/O callback, etc..
With this method, notably used by game devs, there's one code base, with thin
platform drivers.
Thin platform drivers, i.e... platform specific native code.
Can we PLEASE stop beating this dead horse? My point is that NO SOFTWARE
NEEDS TO BE INSTALLED AT ALL WHATSOEVER AT ALL PERIOD AT ALL. Get it?
Good. Let's move on.
On Tue, 2011-03-01 at 19:36 +0100, Olivier
Guilyardi wrote:
Hmm, what
platform specific native code? In my idea, the host would handle
setting up the GL viewport and such.
... "the host would <platform specific native code>" ;) What you're talking about makes sense if the host
itself runs in the browser...
But switching from a native UI to a browser repeatedly is going to look rather
odd IMO. And if your idea is mainly intended for browser-based hosts then it
will not integrate properly with a large range of existing software.
The idea is to put the entire performance/control/et UI in the browser,
so it can be remote. You've missed the remote idea. Remotes good, even
on a desktop. People go out and spend money on special purpose devices
for this all the time. Tablets can give you a completely generic
reconfigurable... that.
The "host", as in the plugin host that does audio things, is obviously
not in the browser, that's on the machine doing audio things, C, Jack,
etc.
That said, for instance on Android, you can embed a
small WebView into the UI of
a host. So it could make sense.
Yep.
But (http) network operations are going to bring
some more overhead. All of this really isn't going to be smooth.
It's not a network operation if it's local. Local socket communication
is very fast... and something that's arriving in the browser.
http://en.wikipedia.org/wiki/WebSockets
As far as I'm concerned a non-crap app design should be capable of doing
engine <=> UI communication over a <whatever, e.g. socket> anyway, even
if you only intend to use it on a local machine. You WANT a huge
meaningful wall between those two things. For one thing, if your UI
stuff crashes, your audio doesn't, and your set doesn't trainwreck.
The other win of UI<=>audio separation is that not all of our DSP code
runs on a computer with a monitor at all. Little embedded boxes that do
DSP stuff don't have big screens... but they usually have USB ports (or
network, or MIDI, or something).
That UI <=> Plugin separation is necessary is, IMO, a given, and not up
for debate. You could throw it all out the window and gain virtually
nothing in return, but... well, doesn't sound wise, does it?
The plugin would expose a draw() callback
in which it would only perform standard OpenGL calls.
I'm not an expert about GL portability, but AFAIK this might allow the UI
provided by a plugin to run unchanged on Linux, iOS, Android, Mac, Windows, etc..
Probably, since the host deals with all the non-portable things. What
you describe here sounds like how a GL UI type should indeed work. It
would be better if there was some equivalent of a "widget" the UI could
return, but I guess that is not the case for GL. So far, nobody
has showed up with a desire to write a GL plugin UI.
Extensions don't get invented until there's an actual use for them. If
someone does, drop by lv2-devel, and we'll get this done (or don't, and
just do it).
Yes, I've developed a long time for the desktop, and I never used OpenGL before
I got myself into mobile development. Now that I'm familiar with it, I quite
enjoy it, and am amazed by the perspectives in terms of portability. The next SLV2
will abstract away UI types, so if/when this gets
invented, it will magically just work in existing hosts, since embedding
GL in most toolkits is possible.
Sounds good. This is where LV2 extensibility shows some of its strengths I guess. I'm confused now. You are talking about UIs
automatically built by the host from
the plugin parameters, ports, metadata, etc..
Yes, partially...
To me, this is not a plugin UI,
... except a plugin could provide such a (plugin specific) UI as well.
My "dirg" plugin provides a web-based grid UI in the browser, for
example. Currently it just does this by running its own web server, but
this needs to be pushed up to the host for obvious reasons. this is the absence of plugin specific UI, and the
kind of thing which you can
already do with LADSPA plugins, etc.. So, automatically generated web UIs from
plugin metadata is of course possible, but I wasn't talking about that.
I'm talking about those cases where the plugin needs to provide a specific UI,
as it's currently being done with LV2 using GTK, for example in Calf plugins,
etc.. But here you mention a mix of automatically generated UIs and "special web
UI fragments" provided by the plugin.
"special web UI fragments", or, in other words, a plugin specific UI.
Yep. I think that I understand you idea of doing this with
a browser. No need to
setup OSC, ports, map URIs, etc.. But this sounds like a web-based controller
project. It's application level IMO, not sure it belongs to LV2, apart from the
UI fragments idea, but this all sounds like fundamental research to me.
My LV2 host /is/ an LV2 plugin ;) Think about the implications of that
for a minute and a big lightbulb should go off (Hint: Want to make
plugins with universal UIs? Don't want to write code?) Okay, well, this Web UI idea would certainly be good
in certain cases. And the
truth is that I'm not really convinced about the OpenGL idea, because it may
result in inconsistent appearance, with widgets style which do not match each other.
This is audio. Give people the ability to pixmap at all and they're
going to make some big retarded quasi hardware photo thing, each
entirely different, and usually horrific screen real-estate wasters.
Consistency between UIs ain't going to happen. GL would probably be even
more consistent than the toolkits, ironically, because it tends to push
you towards vectorey things.
But this later problem could be solved with a simple
audio-oriented UI toolkit,
which would render using OpenGL. A such toolkit could actually be very
lightweight. For instance you do not necessarily need to embed entire charsets.
A couple of characters (digits, "dB", etc..) could be sufficient in many cases.
Aaaand now we're entered the realm of many years of work that nobody is
going to do. Unless JUCE counts.
Speaking of doing tons of work, GL gives you an open-ended way to draw
things... and that's it. One you actually need to make those things
interactive, you need something else. Something else that isn't just GL,
and probably isn't portable, and is a ton of more work regardless. The
DOM inherently makes it trivial to deal with events, and that's all
built-in. A mess of little toolkits, all alike, is not a fun situation,
and plopping it on a GL backend doesn't make it go away.
-dr