6 Dec
2010
6 Dec
'10
11:10 a.m.
I have been looking for a while to LLVM as a possible technology to
build a DSP execution engine, providing the runtime flexibility needed
by real-time interactive DSP applications (like patcher languages), or
by plug-in based processors, and in the same moment
the powerful link time optimizations that such a system can provide.
Such a task is daunting for a single developer project like mine, but it
may become feasible if such engine could be useful for multiple
projects, so to become a community initiative (i am sending this
mail to the Linux Audio Developers mailing list and to the LLVM mailing
list, feel free to send it elsewhere if you find it useful).
So, I'll explain in a few lines exactly what i thinking of, and i
propose that if anybody is interested in such initiative she/he can
contact me directly, and we will see if there is enough interest
for actually start something. Of course, i may also miss an existing
project on the same field, in this case, let me know :->.
The idea is the following: build a tool kit, based on LLVM, to
dynamically build the execution chain of DSP applications; almost all
DSP applications using plug-ins have something of this kind.
This toolkit will be based on code generation and JIT, and should have a
flexible architecture so to accomodate multiple plugin API/ABI, to be
able to call, from the same execution chain, DSP operation written
for different contexts (for example, LADSPA plug-ins, PD objects, and so
on). The code generation would allow this integration for a very low run
time cost.
But the most interesting part is actually providing a definition of
native, LLVM compiled, plug-ins, and to use the LLVM link time
optimizations for doing things like inlining, loop fusions etc, between
different DSP operation; this can provide quite a speed improvement for
those chains composed of simple operators.
The execution chain should be flexible enough to provide support for
multi-rate synchronous data flows, and should include support for
multiple threads execution (probably different chains for different
threads).
To complete the project, let's add a library of DSP operators, a
reasonable high level API to use it, and documentation to help the DSP
application designer (i find the LLVM learning curve quite steep :).
Contact maurizio(a)dececco.name if you are interest in setting up such a
project; if enough developers with the right skills and enough interest
are found I'll set up a project mailing list to further discuss the subject.
Cordially,
Maurizio
__________
Maurizio De Cecco -
Music: http://www.myspace.com/mauriziodececco
Blog: http://maurizio.dececco.name/
Software: http://www.jmax-phoenix.org/
6 Dec
6 Dec
12:11 p.m.
On Mon, Dec 6, 2010 at 6:10 AM, Maurizio De Cecco <jmax(a)dececco.name> wrote:
I have been looking for a while to LLVM as a possible
technology to build a
DSP execution engine, providing the runtime flexibility needed by real-time
interactive DSP applications (like patcher languages), or by plug-in based
processors, and in the same moment
the powerful link time optimizations that such a system can provide.
My own gut feeling on this is that you'd be better off figuring out
how to do this in the context of Faust, which already does a important
subset of what you are describing, though notably without (I think)
the LLVM part.
Note also that hosts which run plugins at the level of LADSPA/LV2,
VST, AU, DSSI etc are unlikely to be easy candidates for any
cross-plugin optimization. I can say for sure that, for example,
Ardour3 has an object called a Processor whose ::run() method
encapsulates all DSP done within Ardour, but nevertheless it would be
more or less impossible to do any kind of optimization that looked
"across" all the Processors in a signal chain (eg. gain, pan, plugins,
etc, etc).
--p
1:11 p.m.
Hi, and thanks for the feedback.
My own gut feeling on this is that you'd be better
off figuring out
how to do this in the context of Faust, which already does a important
subset of what you are describing, though notably without (I think)
the LLVM part.
Surely a contribution to the discussion by the Faust people would
be welcome; and even more than to the discussion :->. By the way, they
do something with LLVM, but i do not know exactly what (Faust is included
in the list of projects using LLVM).
I think the interest, if any, would be in having a *low* level
library generic enough to be used by multiple applications, so
to factor the work. It may be that an existing project have internally
a large part of the needed code, though.
Note also that hosts which run plugins at the level of
LADSPA/LV2,
VST, AU, DSSI etc are unlikely to be easy candidates for any
cross-plugin optimization.
I was actually talking about two different context, reusing existing
binaries, and having cross plugin optimisations for native plugins.
But, if the plugins are compiled with a different compilation chain
(using the LLVM compiler producing LLVM byte code, so introducing a different
binary format for them), than cross plugin optimisation is possible also for
existing plugins like a LADSPA plugin.
Would that be worthwhile ? Difficult to know for sure, but it is an interesting
subject to explore.
Maurizio
9:26 p.m.
Maurizio De Cecco wrote:
Surely a contribution to the discussion by the Faust
people would
be welcome; and even more than to the discussion :->. By the way, they
do something with LLVM, but i do not know exactly what (Faust is included
in the list of projects using LLVM).
Thanks to Stephane Letz' work, Faust has had an LLVM backend for some
time. (You need to get the faust2 branch from the Faust git repo for
that, however.) Faust2 supports the creation of both LLVM assembler and
bitcode. The latter can be loaded and executed directly by other LLVM
clients (such as my own Pure, which also sports inlining of Faust
programs in Pure scripts now).
Albert
--
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email: Dr.Graef(a)t-online.de, ag(a)muwiinfa.geschichte.uni-mainz.de
WWW: http://www.musikinformatik.uni-mainz.de/ag
7 Dec
7 Dec
10:40 a.m.
I realised that my yesterday answer miss the point a little,
so here it is a few linse of complement.
Going back to the Paul comment:
I can say for sure that, for example,
Ardour3 has an object called a Processor whose ::run() method
encapsulates all DSP done within Ardour, but nevertheless it would be
more or less impossible to do any kind of optimization that looked
"across" all the Processors in a signal chain (eg. gain, pan, plugins,
etc, etc).
This is exactly the point: what about having a way to build the 'run' methods
for a class of applications (PD have such a method, jMax have such a method,
many others application have such a methods) that *would* allow cross plugin
optimisations for a certain class of plugins (those written or compiled according
to the engine 'native' rules) and being able to use the other plugins keeping the
glue code run time cost very low thanks to dynamic code generation.
Essentially a generic basic infrastructure using LLVM code generation and optimisation
capabilities.
And by the way, i am not sure to understand the reference to Faust, that is one of the
language/system used to define and assemble DSP algorithms, and to generate plugins,
but it is not a generic container for DSP executions, AFAIK.
Maurizio
6 Dec
6 Dec
12:48 p.m.
Ciao Maurizio,
2010/12/6 Maurizio De Cecco <jmax(a)dececco.name>me>:
I have been looking for a while to LLVM as a possible
technology to build a
DSP execution engine, providing the runtime flexibility needed by real-time
interactive DSP applications (like patcher languages), or by plug-in based
processors, and in the same moment
the powerful link time optimizations that such a system can provide.
Such a task is daunting for a single developer project like mine, but it may
become feasible if such engine could be useful for multiple projects, so to
become a community initiative (i am sending this
mail to the Linux Audio Developers mailing list and to the LLVM mailing
list, feel free to send it elsewhere if you find it useful).
It's some time that I actually have a somewhat related idea: when you
use physics-based modeling techniques a crucial point is proper
scheduling of operations, which has NOT to be done on a "processing
object" (i.e., plugin or whatever) basis, but by considering
dependencies among single input/output expressions even inside
"processing objects" (this may sound unclear, I know).
This means that in practice, if you deal with compiled code (a.k.a.
plugins), you just can't reasonably do it, as opposed to specialized
tools, and in particular special audio programming languages that can
either be interpreted or compiled to some sort of "understandable"
bytecode that preserves input/output time relationships (i.e., delays)
and then run in a virtual machine that solves the scheduling issue.
An example of such specialized tools is the audio DSP language I wrote
called Permafrost. Here, the scheduling issue is solved when compiling
the Permafrost source code to LV2 plugin source (C code and Turtle/RDF
metadata), but this is a one way operation - you can't use the output
plugin code to build a new plugin taking into account these issues,
whether at runtime or not.
My idea was to define an intermediate DSP bytecode, hopefully also
capable of including metadata, and a virtual machine that schedules,
optimizes and runs the whole chain/graph.
A stupid example of what can be achieved with such a thing is the
following: suppose you have a physics-based simulator of a tube
amplifier that allows you to "plug" a physics-based loudspeaker model
into it (if you are into this kind of stuff, suppose it is WDF-based)
- you would be able to change such loudspeaker model while the system
is running and have it all correctly scheduled, optimized and
perfectly working.
On the other hand, I believe you are aiming for something different,
hence I do suggest you (as Paul did) to contact FAUST developers,
since they're pretty much into this kind of stuff, especially when it
comes to LLVM.
BR,
Stefano
7 Dec
7 Dec
5:01 p.m.
Hiho,
Nick Collins has been working on some stuff like this for SuperCollider.
Haven't tried it on Linux yet, though, but that's more because of lack of
time.
sincerely,
Marije
On Monday 06 December 2010 12:10:02 Maurizio De Cecco wrote:
I have been looking for a while to LLVM as a possible
technology to
build a DSP execution engine, providing the runtime flexibility needed
by real-time interactive DSP applications (like patcher languages), or
by plug-in based processors, and in the same moment
the powerful link time optimizations that such a system can provide.
Such a task is daunting for a single developer project like mine, but it
may become feasible if such engine could be useful for multiple
projects, so to become a community initiative (i am sending this
mail to the Linux Audio Developers mailing list and to the LLVM mailing
list, feel free to send it elsewhere if you find it useful).
So, I'll explain in a few lines exactly what i thinking of, and i
propose that if anybody is interested in such initiative she/he can
contact me directly, and we will see if there is enough interest
for actually start something. Of course, i may also miss an existing
project on the same field, in this case, let me know :->.
The idea is the following: build a tool kit, based on LLVM, to
dynamically build the execution chain of DSP applications; almost all
DSP applications using plug-ins have something of this kind.
This toolkit will be based on code generation and JIT, and should have a
flexible architecture so to accomodate multiple plugin API/ABI, to be
able to call, from the same execution chain, DSP operation written
for different contexts (for example, LADSPA plug-ins, PD objects, and so
on). The code generation would allow this integration for a very low run
time cost.
But the most interesting part is actually providing a definition of
native, LLVM compiled, plug-ins, and to use the LLVM link time
optimizations for doing things like inlining, loop fusions etc, between
different DSP operation; this can provide quite a speed improvement for
those chains composed of simple operators.
The execution chain should be flexible enough to provide support for
multi-rate synchronous data flows, and should include support for
multiple threads execution (probably different chains for different
threads).
To complete the project, let's add a library of DSP operators, a
reasonable high level API to use it, and documentation to help the DSP
application designer (i find the LLVM learning curve quite steep :).
Contact maurizio(a)dececco.name if you are interest in setting up such a
project; if enough developers with the right skills and enough interest
are found I'll set up a project mailing list to further discuss the
subject.
Cordially,
Maurizio
__________
Maurizio De Cecco -
Music: http://www.myspace.com/mauriziodececco
Blog: http://maurizio.dececco.name/
Software: http://www.jmax-phoenix.org/
_______________________________________________
Linux-audio-dev mailing list
Linux-audio-dev(a)lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-dev
8 Dec
8 Dec
11:20 a.m.
On Tuesday12/7/10 6:00 PM, nescivi wrote:
Hiho,
Nick Collins has been working on some stuff like this for SuperCollider.
Haven't tried it on Linux yet, though, but that's more because of lack of
time.
Wouldn't be nice to put together people working on the subject and see
if for once we can produce a common reference framework and
interoperable things, instead of having a different kind of wheel for
each project ?
Maurizio
4894
days inactive
4896
days old
linux-audio-dev@lists.linuxaudio.org
7 comments
5 participants
participants (5)
-
Albert Graef -
Maurizio De Cecco -
nescivi -
Paul Davis -
Stefano D'Angelo