The Pre-X1 default was 500ms for years. X1 lowered it to 250 which caused problems for a lot of systems. What works best is very system-dependent but does not necessarily correlate to system horsepower. You might go a long time at any given setting with no problem, and then find that one particular part of one particular track in one particular project at one particular tempo results in dropped notes, and you have to bump it up a little.
I've found the old default of 500 works great 99.9% of the time. Every once in a while, I'll notice a dropped note, and have to bump it up to 550 or 600 for that one project. Sometimes just moving the note(s) a single tick one way or the other will fix it. I have a sense that it might happen more often with hard-quantized MIDI where there can be a lot going on right on the beat with multiple synths and drum parts hitting at the same time. I think it helps to record stuff in real time and not over-quantize it.
MIDI to Hardware outputs isn't buffered so it isn't affected. Really what's being buffered is not MIDI per se, but pre-rendered audio from soft synths. For this reason, I'd stay away from extremely high settings such as Gustabo
suggested uses. This can unecessarily delay the start of playback, and can have other undesirable side-effects such as glitches when looping or moving the Now time during playback. For the record, I was unable to reproduce Gustabo's issues with MFX at lower buffer settings.