Does the VST3 migration, i.e. substitution work for your custom plug-in layout menus or just the Sonar default menus?
 
Regarding this the Help documentation says:
 Hide Related VST2 Plug-ins. 
If you have both VST2 and VST3 versions of a plug-in, you can instruct SONAR to automatically hide any redundant VST2 plug-ins from SONAR’s plug-in menus

The answer appears to be - just the Sonar default menus.
You custom plug-in layout menus are not modified/altered/updated by the VST3 Migration setting.
 

Right. Do you think it should also hide them from custom plugin layouts?

Yes, otherwise you would uncheck/disable the option.
 

Sonar X3 Help
If you have both VST2 and VST3 versions of a plug-in, you can instruct SONAR to automatically hide any redundant VST2 plug-ins from SONAR’s plug-in menus.

 
It doesn't say "Sonar's default plug-in menus"
BTW there currently is a bug in accessing  the help on VST Settings, CWBRN-19924, you currently have to select another preference topic, hit the Help button and then scroll up or down to see it.