Nope. You aren't thinking right. You don't have to change your entire platform. Digital signal processing is as sanely possible on Ubuntu as it is on Gentoo, Slackware, Arch, Debian..err and what have you. The underlying merits have little to do with anything professional audio, except for (1) number of relevant packages and (2) quality of those packages. Both criteria are met by Ubuntu Studio, as well as JackLab and x-popular-audio-distro. Now if by (2) we mean "stable", 64Studio (as well as x-stable-audio-distro) is probably the way to go. Stable also means losing out on the latest (not necessarily the greatest).

Debian? Why waste your time? You've already spent a good amount with its derivatives. The way I see it - you're not quite sure what you want. You're assuming benefits. So if you're looking forward to advantages with regards to music production and audio engineering in general, forget about it. There are plenty of intelligent power-users and hackers in the more "user-friendly" software communities; many of them have little time to worry about "optimising", hence settling on such "easy" platforms for production. An operating system does not dictate its user, but the other way around.

Alright, I've preached my share - although only on multimedia-related advantages. Now if you're a sucker for control, it's a different story. You no longer base your choice against "digital signal processing", but other technical merits related to managing and maintaing an OS - which in this case is a Linux distribution. Here I can recommend Debian, yes, along with Gentoo and Arch. You do want to go with one that has an "audio community" at minimum.

On Debian's side, there's nothing to ponder about. You have hundreds of packages. I would say the same for Gentoo, except that those packages are source-based. I personally dislike compiling; there is no gain. Arch, on the other hand, has a small (audio) community (more than enough to get digital signal processing to work). It has third-party pro audio projects a la Planet CCRMA or Pro Audio Overlay, both binary- and source-based, and it has the AUR (a large database of user-contributed source-based packages that are candidates for offical binary repositories).

Arch sits nicely in between Debian and Gentoo on the technical merits' scale, though personally I'd say it's unlike either. You install and configure what you want and get to work immediately. No worries about compiling, no thinking about -devel packages even if you need to compile, quick access to a unified build system when you do need to compile (PKGBUILDs are three times simpler than an ebuild; creating a binary package is 5 times simpler than producing a DEB or RPM), intelligent community, and..are the points getting a little excessive? Well, not to worry. Arch has downsides for your satisfaction too - like no help if you don't help yourself (I think this is good but many think otherwise). It's an OS for those who know they want it (I do think you may fit that criteria if you're honest about "streamlined"). It is early morning here after 5 hours of sleep and 2 hours before let's-go-down-to-the-office; I just felt like writing (or typing). Hope this is helpful. Please, enjoy reading ;) 

P.S: Bottom line - Thou Shalt Not Switch Distro for Pro Audio Merits Alone

PS2='\[\e[1;31m\]>>\[\e[m\] \[\e[1;37m\]' ..hey no that's not what I meant to type :( 

Update before sending: Florian above beat me to it, but I'm sending this anyway for your reading pleasure :) (many smileys for your pleasure too, in case it's morning)