Hi Ede,

 20:  <CPU List> IR-IO-APIC   20-fasteoi   ohci_hcd:usb9, snd_rme9652, vfio-intx(0000:07:00.0)
 
Have you tried plugging the device into a different USB port? Sometimes this makes a difference for the USB interrupt.

So some USB, my passed-through parallel card and my soundcard are all sharing interrupt #20.
 
Do you *use* the parrrallel port? If not, you can remove the kernel module with rmmod, and then the interrupt won't fire..

And finally you could set the CPU affinity for each interrupt, so at least they're going to be handled by a different CPU than the one that's going to handle the audio interrupt.

Some details about latency tuning, CPU affinities and tuning here:
http://openavproductions.com/real-time-latency-tuning/

Cheers, -Harry

--