The good stuff stands up over time, regardless of initial popularity. "Legs", that's what they call it in the movie business to describe a film that opens weak but gradually builds an audience. They say it "has legs".
Ashlee Simpson's "Autobiography" went platinum in 2004, but who's listening to it today? Nobody, I suspect.
"Pet Sounds", OTOH, barely made a blip on the charts on its release and was widely dismissed by critics. But it changed pop music forever, influenced every record producer since, and makes every Top N list of the greatest albums of all time.