What sort of latency issues are you having?
There are two sorts of latency, that which you specify with the audio buffer settings and that which is imposed on you by plugins.
512 buffers seems a tad high for tracking, most folk aim for 128 or 64 or even 32 in a well optimised system.
When tracking you shouldn't have any high latency plugins loaded at all. Basically anything that uses (or can be set to use) lookahead buffers is a complete no no.
To test, start with a totally clean, new project. No plugins at all, just one track to do test recording, no Master bus, just route the track directly to the soundcard output and see how low you can get the audio buffer before you start getting crackling and/or drop outs.