The noise may very well be coming from the synthesizer itself. Of all the synths I've owned over the years, the vast majority of them have had noise levels that are only acceptable for live use but pretty awful for recording. One exception was my old Jupiter, which had balanced XLR outputs and was very clean. The
worst one I ever had was a Casio product.
Many synths feature S/PDIF connections, which is definitely the way to go for extremely low noise levels. But if you don't have that and must record from the audio outputs, you're just going to have to mitigate the noise problems after recording. Use slip-edits to trim clips so they start right at the start of signal, use short fades at the start and end of the clips. If it's really bad you might have to use volume automation to kill the noise during gaps in the audio. Recording hot won't help, as the noise is coming from the final amplifiers in the synth and turning up the volume turns up the noise, too.
If you're getting hum (or buzzing) from the synth it's probably a shielding or grounding problem. It might be worth taking it to a knowledgeable technician to see what can be done about it. It could even be a loose ground connection within the case. More likely, it's just a bad design in the synth. I've seen power transformers located right next to I/O jacks; not much you can do about that, but a tech might offer a useful opinion.
This unit uses an external wall-wart power supply, which are often hum sources, too. They're typically not well-filtered and poorly grounded. I'd be inclined to try a higher-quality aftermarket 12VDC power supply. Or, if you do take it to a well-equipped technician, chances are he'll have a well-filtered variable-voltage power supply you can try out.