On 03/02/2011 12:22 AM, David Robillard wrote:
On Tue, 2011-03-01 at 19:36 +0100, Olivier Guilyardi
wrote:
Hmm, what
platform specific native code? In my idea, the host would handle
setting up the GL viewport and such.
... "the host would <platform specific native code>" ;)
No, not really.
Apart from mobile web apps, a way to build cross-platform apps is using a
lightweight toolkit, with OpenGL for all graphics, and a couple of other entry
points such as an audio I/O callback, etc..
With this method, notably used by game devs, there's one code base, with thin
platform drivers.
So, with such portable hosts, the platform specific native code goes into the
toolkit. And both the host and plugin would be rather platform-agnostic.
Going to something in a web browser doesn't
involve any of that. It
doesn't involve anything on the "thing with a UI on it" whatsoever
except "go to this URL in your web browser". i.e. no platform specific
native code (including GL) is required, at all, to operate the UI. It'd
work on a hypothetical(*) tablet you can't even install software on
whatsoever.
Okay, I see. But we are talking about plugins, are we? To me a plugin is a small
module which embeds in a host UI or appears as a small window on top of it.
What you're talking about makes sense if the host itself runs in the browser...
But switching from a native UI to a browser repeatedly is going to look rather
odd IMO. And if your idea is mainly intended for browser-based hosts then it
will not integrate properly with a large range of existing software.
That said, for instance on Android, you can embed a small WebView into the UI of
a host. So it could make sense. But (http) network operations are going to bring
some more overhead. All of this really isn't going to be smooth.
In order to avoid network operations, on Android, you could expose some of your
Java classes as Javascript objects into the WebView, this is entirely possible.
But now that's going to be a lot of native code for binding say a slider to
javascript and then to java and then down to the native plugin...
The plugin
would expose a draw() callback
in which it would only perform standard OpenGL calls.
I'm not an expert about GL portability, but AFAIK this might allow the UI
provided by a plugin to run unchanged on Linux, iOS, Android, Mac, Windows, etc..
Probably, since the host deals with all the non-portable things. What
you describe here sounds like how a GL UI type should indeed work. It
would be better if there was some equivalent of a "widget" the UI could
return, but I guess that is not the case for GL.
The UI would draw into a rectangular area which could:
- either be embedded in a larger GL scene, if the host UI is GL based,
- or be displayed by QGLWidget, rendered into a GdkGLWindow, etc..
Also, what's rather funny is that there would be some kind of symmetry between
the run() and draw() functions exposed by a plugin. One for audio, the other for
graphics/video... And both called in a cyclic manner by the host.
So far, nobody has showed up with a desire to write a
GL plugin UI.
Extensions don't get invented until there's an actual use for them. If
someone does, drop by lv2-devel, and we'll get this done (or don't, and
just do it).
Yes, I've developed a long time for the desktop, and I never used OpenGL before
I got myself into mobile development. Now that I'm familiar with it, I quite
enjoy it, and am amazed by the perspectives in terms of portability.
The next SLV2 will abstract away UI types, so if/when
this gets
invented, it will magically just work in existing hosts, since embedding
GL in most toolkits is possible.
Sounds good. This is where LV2 extensibility shows some of its strengths I guess.
I'm
confused now. You are talking about UIs automatically built by the host from
the plugin parameters, ports, metadata, etc..
Yes, partially...
To me, this is not a plugin UI,
... except a plugin could provide such a (plugin specific) UI as well.
My "dirg" plugin provides a web-based grid UI in the browser, for
example. Currently it just does this by running its own web server, but
this needs to be pushed up to the host for obvious reasons.
Hmm, own web server, I don't know really.. This sounds a bit bloated to me.
this is the
absence of plugin specific UI, and the kind of thing which you can
already do with LADSPA plugins, etc.. So, automatically generated web UIs from
plugin metadata is of course possible, but I wasn't talking about that.
I'm talking about those cases where the plugin needs to provide a specific UI,
as it's currently being done with LV2 using GTK, for example in Calf plugins,
etc.. But here you mention a mix of automatically generated UIs and "special web
UI fragments" provided by the plugin.
"special web UI fragments", or, in other words, a plugin specific UI.
Yep.
And how does one draws a frequency curve, even one which doesn't update in
realtime? With SVG? Do you see how heavy this is all going to be?
Regarding
remote controllers.. Well this is already in the wild, tablets
included, with OSC and MIDI. For example, there is TouchOSC (sorry, not free,
but a great project) which even has a UI editor.
Same kind of realm as this stuff, yes, except the UI design is
integrated with the host and plugins, and is roughly infinity times
slicker as a result.
Slicker for who? The whole point of controllers is that the user can choose the
one he/she likes. Hardware controller, tablet-based controller with knobs, or
sliders, or pad, or even one which uses the device sensors such as the
accelerator, the compass, etc..
I think that I
understand you idea of doing this with a browser. No need to
setup OSC, ports, map URIs, etc.. But this sounds like a web-based controller
project. It's application level IMO, not sure it belongs to LV2, apart from the
UI fragments idea, but this all sounds like fundamental research to me.
My LV2 host /is/ an LV2 plugin ;) Think about the implications of that
for a minute and a big lightbulb should go off (Hint: Want to make
plugins with universal UIs? Don't want to write code?)
Not sure what you mean here, but I'm a big No Silver Bullet fan, and generally I
don't believe in no-need-to-code-anymore claims.
Is it about web-based controllers on a tablet or
whatever? Yes. But it
is not /only/ about that. If you have such a UI, well, it works just
fine on the PC too. One UI written, both problems solved, objectively
better solution - unless you need the performance of OpenGL. I don't
deny that sometimes you do, but personally I'm more into making music
and less into fancy screenshots and idiotic pixmaps of physical boxes or
whatever.
Now, first, I'll repeat what I said earlier: "I tend to believe that, generally,
what is supposed to work in all cases doesn't work well in specific cases. It
may do the job somehow, but it's not really adapted. And if this results in poor
acceptance, then your investment is lost."
Also, I'm into music too. The greatest reward (apart from money ;) I currently
get when working on my product is enthusiastic feedback from musicians.
Now, think of a music instrument. Do you mean that a grey guitar with
unpersonalized design is what musicians need? I strongly disagree. And that's
why I think that powerful UIs with their own style can be a good thing. It can
stimulate creativity. And each user can choose the one which best fits his mood.
Hopefully its fast enough. Seems fine. Good enough for
controllers and
basic/loop/pattern sequencing, maybe not for actually playing keys and
pads and whatnot.
Okay, well, this Web UI idea would certainly be good in certain cases. And the
truth is that I'm not really convinced about the OpenGL idea, because it may
result in inconsistent appearance, with widgets style which do not match each other.
But this later problem could be solved with a simple audio-oriented UI toolkit,
which would render using OpenGL. A such toolkit could actually be very
lightweight. For instance you do not necessarily need to embed entire charsets.
A couple of characters (digits, "dB", etc..) could be sufficient in many cases.
--
Olivier