If I accidentally drop an instrument onto an existing track in Track View, it is impossible to delete it from the instrument rack.
 
After some fiddling, I found if I added more instruments (correctly this time, onto a blank are of the Track View), then deleted them, the instrument in question would at some point disappear from the rack of its own accord.

Plug-ins in a FX Rack may not be deleted from the Synth Rack. They must be deleted from the FX Rack.

These are all instrument VSTs, not FX. I'm dropping them on the track view, audio or instrument track, doesn't matter. Try it, it's easy. Drag from instrument dropdown onto a preexisting track in the Track View. I'd be interested to know if it's only me.
 
I notice the non-deletable instruments have no Freeze button in the Synth Rack.

It does not matter whether an instrument or effect w/ MIDI enabled, dropping the plug-in in the FX rack causes a slightly different behavior that adding the instrument along with new tracks to support the plug-in.
 
Adding instruments to the FX Rack is a very old method and while still supported has been reported to cause problems.
 
If the plug-in is in the FX Rack, it must be deleted from the FX Rack.

Aaaah. OK, now I got it.
I had my Track View on "Custom" and the Inspector on ProChannel.
Right, now I see the instruments in the FX bins.
And yes, I can delete them without a problem from there.
 
Many thanks, scook - as always
 
...but still, a bit strange.
If something's in the Instrument rack, you'd be within your rights to expect to be able to delete it there.
Oh, well.
But now I know.