Using Firefox 56.0.1 here and the pages are almost instant.
It may not actually be youtube that is the issue. Links sometimes bounce through some google metric sites or pull in ad sites for cookies and the like. It could be your system is having trouble getting to one of those and that is slowing down the overall loading.
Have you tried changing the DNS servers in your WAN config? When I used the default Verizon DNS server, I found they would bounce through or block some URL's (perhaps those sites were down). Try using 8.8.8.8 and 8.8.4.4 (google primary and secondary) for your DNS servers and see if it makes any difference.