I made a Prototype that uses jack directly now.
I decided on working with Jack directly because I want to implement some features that would require rewriting parts of rtmidi or portaudio for my purposes.
I want to enumerate ports and get raw ports, so I can have control over properties.

I took a look into the rtmidi source, looking for the part how they get all the ports that jack knows about.
I did nothing that would help me, so how could I do something about this?