[LAD] Fwd: DAFxTRa 2008: announcement and call for participation

[stefan kersten]

i'm sure the linux audio community has something to contribute to  
this initiative!

Begin forwarded message:
> From: "Graham Coleman" <gcoleman at iua.upf.edu>
> Date: 25. April 2008 18:01:28 GMT+02:00
> Subject: DAFxTRa 2008: announcement and call for participation
>
> Apologies for multiple postings. We will attempt a public evaluation
> of digital audio effects. We encourage your feedback and
> participation.
>
> --
> Salutations!
>
> We announce and call for your participation in a cross-community
> evaluation of audio effects.
>
> DAFxTRa 2008 (DAFx Transformation RAting) is a new initiative  
> promoted by
> MTG-UPF and DAFx-08 aimed at evaluating and comparing algorithms for
> audio effects.
> Our goal is to have the main evaluation in September 2008 during  
> DAFx-08
> (http://www.acoustics.hut.fi/dafx08/) but to make it happen we need  
> the
> involvement of the audio effects research community.
>
> We do not know of any major initiative to compare audio effects
> algorithms, which might be the related to the difficulty of the
> evaluation task. It is hard because it requires standardized  
> procedures for
> carefully controlled subjective experiments. But we believe
> that now is the time to try it, so we can all learn from the  
> process and in
> turn improve our audio effects algorithms.
>
> Inspired by the success of MIREX in the evaluation of Music  
> Information
> Retrieval algorithms, and having acquired some experience by  
> organizing the
> audio description contest at ISMIR 2004-Barcelona, we want to  
> promote a
> similar initiative for the digital audio effects community.
>
> However most audio effects tasks do not afford an objective
> measure of ground truth, like in MIR, and thus we have to define
> specific evaluation strategies. We want to do it by involving the
> developers of algorithms interested in participating and by specifying
> with them the evaluation process. The participants should learn  
> from the
> process and the whole DAFx community should benefit from it.
>
> Our initial aims are the following:
> 1. Propose several audio effects categories. Initial proposal: time- 
> scaling,
> pitch-shifting, source separation, morphing, distortion effects and
> deconstruction.
> 2. Select the categories for which there is a sufficient number of
> participants.
> 3. Select sounds from Freesound (http://freesound.iua.upf.edu) to be
> used as test sounds for each category.
> 4. Define the evaluation procedure for each category.
> 5. Ask the participants to submit the transformed sounds (not the
> algorithms).
> 6. Perform the evaluation both live at DAFx-08 and also online in  
> Freesound.
> 7. Publish the results of the evaluation.
>
> Our proposed time-line is:
> - 1st August 2008: Finalize categories and evaluation procedures
> - 30th August 2008: Submit transformed sounds
> - 1st-4th September 2008: Run life evaluations at DAFx-08 Conference
> - 10th-30th September 2008: Run on-line evaluations on the  
> Freesound site
> - 15th October 2008: Publish the results
>
> If you are interested in participating or in getting involved in the
> process join the DAFxTRa mailing list
> (http://iua-mail.upf.es/mailman/listinfo/dafx-eval) for an open  
> discussion.
> Results of the discussion and organizational details of the evaluation
> will be posted in a wiki (http://smcnetwork.org/wiki/DafxTRa2008)
>
> Your input and ideas will be most welcome.
>
> Graham Coleman (MTG-UPF) (contact person)
> Jordi Bonada (MTG-UPF)
> Perfecto Herrera (MTG-UPF)
> Xavier Serra (MTG-UPF)