not sure if this helps but I use an Oxygen 49 USB with my Focusrite and I don't have any latency at all.
I would make sure your Focusrite drivers are installed properly and that the Oxygen is selected under midi devices.
also make sure to install the Oxygen's drivers (from Avid)
personally I would plug your interface into one port, the oxygen in to the other. use the third port with your hub (for mouse, exc).
but I believe enough good advise has been given I don't want to add any confusion.
PS- unless the Scarlett is a 3rd gen, I would try at all cost to plug it in to a USB 2 port.
that's what Focusrite told me when I asked about possibly better results using a USB 3 port.