Not enough information. I recommend starting jackd with strace
This code tries to call sched_setscheduler() with SCHED_FIFO. My bet is
that your "almost vanilla kernel" fails to fulfil this request, but
strace will tell you for sure.
The question is why said call should fail. The only thing that comes to
my mind are CGROUPS. Maybe your old kernel comes without, the new kernel
supports them and the configuration is set in a way that disables
SCHED_FIFO by default.