Hi everyone,
I'm Wim Taymans and I'm working on a new project called PipeWire you might
have heard about [1]. I have given some general presentations about it during
its various stages of development, some of which are online [2].
PipeWire started as a way to share arbirary multimedia, wich requires vastly
different requirements regarding format support, device and memory management
than JACK. It wasn't until I started experimenting with audio processing that
the design started to gravitate to JACK. And then some of JACKs features became
a requirement for PipeWire.
The end goal of PipeWire is to interconnect applications and devices through
a shared graph in a secure and efficient way. Some of the first applications
will be wayland screen sharing and camera sharing with access control for
sandboxed applications. It would be great if we could also use this to connect
audio apps and devices, possibly unifying the pulseaudio/JACK audio stack.
Because the general design is, what I think, now very similar to JACK, many
people have been asking me if I'm collaborating with the linux pro-audio
community on this in any way at all. I have not but I really want to change
that. In this mail I hope to start a conversation about what I'm doing and I
hope to get some help and experience from the broader professional audio
developers community on how we can make this into something useful for
everybody.
I've been looking hard at all the things that are out there, including
Wayland, JACK, LV2, CRAS, GStreamer, MFT, OMX,.. and have been trying to
combine the best ideas of these projects into PipeWire. A new plugin API was
designed for hard realtime processing of any media type. PipeWire is LGPL
licensed and depends only on a standard c library. It's currently targeting
Linux.
At the core of the PipeWire design is a graph of processing nodes with arbirary
input/output ports. Before processing begins, ports need to be configured with a
format and a set of buffers for the data. Buffer data and metadata generally
lives in memfd shared memory but can also be dmabuf or anything that can be
passed as an fd between processes. There is a lot of flexibility in doing this
setup, reusing much of the GStreamer experience there is. This all happens on
the main thread, infrequently, not very important for the actual execution of
the graph.
In the realtime thread (PipeWire currently has 1 main thread and 1 realtime data
thread), events from various sources can start push/pull operations in the
graph. For the purpose of this mail, the audio sink uses a timerfd to wake up
when the alsa buffer fill level is below a threshold. This causes the sink to
fetch a buffer from its input port queue and copy it to the alsa ringbuffer. It
then issues a pull to fetch more data from all linked peer nodes for which there
is nothing queued. These peers will then eventually push another buffer in the
sink queue to be picked up in the next pull cycle of the sink. This is somewhat
similar to the JACK async scheduling model. In the generic case, PipeWire has to
walk upstream in the graph until it finds a node that can produce something (see
below how this can be optimized).
Scheduling of nodes is, contrary to JACKs (and LADSPA and LV2) single 'process'
method, done with 2 methods: process_input and process_ouput. This is done to
support more complex plugins that need to decouple input from output and to also
support a pull model for plugins. For internal clients, we directly call the
methods, for external clients we use an eventfd and a shared ringbuffer to send
the right process command to the client.
When the external client has finished processing or need to pull, it signals
PipeWire, which then wakes up the next clients if needed. This is different from
JACK, where a client directly wakes up the peers to avoid a server context
switch. JACK can do this because the graph and all client semaphores are shared.
PipeWire can't in general for a couple of reaons: 1) you need to bring mixing of
arbitrary formats to the clients 2) sandboxed clients should not be trusted with
this information and responsability. In some cases it would probably be possible
to improve that in the future (see below).
This kind of scheduling works well for generic desktop style audio and video.
Apps can send buffers of the size of their liking. Bigger buffers means higher
latency but less frequent wakeups. The sink wakeup frequency is determined by
the smallest buffer size that needs to be mixed. There is an upper limit for the
largest amount of data that is mixed in one go to avoid having to do rewinds in
alsa and still have reasonable latency when doing volume changes or adding new
streams etc.
The idea is to make a separate part of the graph dedicated to pro-audio. This
part of the graph runs with mono 32bit float sample buffers of a fixed size and
samplerate. The nodes running in this part of the graph also need to have a
fixed input-output pattern. In this part of the graph, negotiating the format
becomes trivial. We can preallocate a fixed size buffer for each port that is
used to send/mix data between nodes. Exactly like how JACK works. In this
scenario it would be possible to bring some of the graph state to trusted
clients so that they can wake up their peers directly.
As it turns out, the generic scheduling mechanism simplifies to the JACK way of
scheduling and the option to do some optimisations (can directly start push from
the sources, bundle process_input/output calls, mixing on ports is simplified by
equal buffer sizes, ...)
There is a lot more stuff that I can talk about and a lot of things that need
to be fleshed out like latency calculations, an equivalent of JACK transport,
session management, ... But this mail is already getting long :)
I would very much like to hear your ideas, comments, flames, thoughts on this
idea. I think I'm at a stage where I can present this to a bigger audience and
have enough experience with the matter to have meaningful discussions.
PipeWire is currently still in heavy development, many things can and do
still change. I'm currently writing a replacement libjack.so[3] that runs jack
clients directly on PipeWire (mixing and complicated scheduling doesn't
work yet).
Hope to hear your comments,
Wim Taymans
[1] pipewire.org
[2] https://www.youtube.com/watch?v=6Xgx7cRoS0M
[3] https://github.com/PipeWire/pipewire-jack
Announcing a new release of project "droning", tune 281 "The Bay of
Atlantis".
*Stream it here:* https://louigi.bandcamp.com/album/281-the-bay-of-atlantis
*Word from the author:*
Extensive work went into this creation.
I wanted the tune to create a feeling that this is one solid composition,
not a soundtrack with distinct segments, but something rather like an ocean
which is in one instance is calm and in the other - furious. But still just
one single ocean.
To all of you travelers out there, and to those of us who find visiting
nonexistent places important.
*Technical specs:* Qtractor, Rakarrack, Carla, Kluppe, seq24 and a number
of LV2 plugins. Zyn is used, although a number of sounds came from other
sources.
--
Louigi Verona
https://www.patreon.com/droninghttps://louigiverona.com/
People,
I am not a professional LA user but I have regard for what serious LA
users have to say. A post turned up on the Fedora XFCE list about
removing xfce4-mixer for F29 - I responded with:
"Every time I upgrade I immediately UNinstall PA and use ALSA only - so
I still depend on xfce4-mixer . ."
Someone replied that PA has greatly improved since the early days
especially and "controlling streams separately is an added feature" -
but I can do that with the .asoundrc I have now - are there any good
reasons for me to reconsider the situation the next time I do a fresh
install? (I realise I am likely to get biased comments here but I am
not going to post on a PA list . .).
Thanks,
Phil.
--
Philip Rhoades
PO Box 896
Cowra NSW 2794
Australia
E-mail: phil(a)pricom.com.au
Guitarix release 0.37.0
Guitarix is a tube amplifier simulation for
jack (Linux), with an additional mono and a stereo effect rack.
Guitarix includes a large list of LV2 and LADSPA plugins, and support
LADSPA / LV2 plugs as well in it's racks.
The guitarix engine is designed for LIVE usage, and feature ultra fast,
glitch and click free preset switching and is full Midi and remote
controllable (the Web UI is not included in the distributed tar ball).
From the changelog:
* new plug Ampimpulse-stereo
* new build parameters in wscript by Simon van der Veldt
please check ./waf --help for mor info.
* fix Live Looper freezes the remote UI
* fix Tuner stick is not saved in remote UI
* fix DrumSequencer issue in remote UI
* fix rc files for older clearlooks engine
* fix disable online preset download in remote UI
* fix GxDigitalDelay sync BPM when UI is not shown
* add new commandline switch (_d, --disable-multi-client)
to allow running guitarix as single client application.
This is useful when guitarix is exported as LV2 plugin.
* fix portmap when running as single client
* again some fixes and additions on the DK_simulator
contributed by Yoann Le BORGNE
Refer to our project page for more information:
http://guitarix.org
Download Site:
http://sourceforge.net/projects/guitarix/
enjoy
regards
hermann
spectmorph-0.4.0 has been released.
Overview of Changes in spectmorph-0.4.0:
----------------------------------------
* Windows is now supported: provide 64-bit Windows VST plugin
* Plugin UI redesign
- use pugl library for portability (uses GL + cairo) instead of Qt5
- use categories for instruments
- directly support instrument names in linear morphing
- get rid of Qt5 dependency for libspectmorph, smjack, VST and LV2 plugins
- UI now has "Zoom" feature to support higher DPI displays
* Use non-linear configurable new velocity -> volume mapping for midi
* New instrument: French Horn
* Improved tools for building custom instruments
- tools are now installed by default
- sminstbuilder files support new syntax for relative paths
- encoder cache moved to ~/.cache/smenccache, which is created if necessary
- use number of processors as default for jobs
* LPC/LSF support removed
* Some portability fixes for macOS (which however isn't supported yet)
What is SpectMorph?
-------------------
SpectMorph is a free software project which allows to analyze samples of
musical instruments, and to combine them (morphing). It can be used to
construct hybrid sounds, for instance a sound between a trumpet and a flute; or
smooth transitions, for instance a sound that starts as a trumpet and then
gradually changes to a flute.
SpectMorph ships with many ready-to-use instruments which can be combined using
morphing.
SpectMorph is implemented in C++ and licensed under the GNU LGPL version 3
Integrating SpectMorph into your Work
-------------------------------------
SpectMorph is currently available for Linux and Windows users. Here is a quick
overview of how you can make music using SpectMorph.
- VST Plugin, especially for proprietary solutions that don't support LV2.
(Available on Linux and 64-bit Windows)
- LV2 Plugin, for any sequencer that supports it.
- JACK Client.
- BEAST Module, integrating into BEASTs modular environment.
Note that at this point, we may still change the way sound synthesis works, so
newer versions of SpectMorph may sound (slightly) different than the current
version.
Links:
------
Website: http://www.spectmorph.org
Download: http://www.spectmorph.org/downloads/spectmorph-0.4.0.tar.bz2
There are many audio demos on the website, which demonstrate morphing between
instruments.
--
Stefan Westerfeld, Hamburg/Germany, http://space.twc.de/~stefan
