If you want the most silent power supply, go with a fanless one. There are a number out there you can find and so long as you aren't doing high end gaming, they'll all be more than fine for your computer. As ever, I'm a Seasonic fanboy so the Seasonic Platinum Series Fanless is what I'd look at. No fan noise, only some very minor electronics noise (all electronics make a bit of noise just in operation).
For fans the key is large and slow spinning. If you keep the system with just the CPU in it (no discrete graphics card) one case fan should be fine. I'd look at Noctua and Phanteks. The Noctua NF-P14 FLX is my top pick, so long as it is mounted on the side of the case, not a top (they have trouble spinning in that alignment). Their noise is low and the profile is unoffensive, and the static pressure they have is quite good. For CPU cooling, you want to get something with a nice large fan on it again so it can spin slow. Arctic Cooling is my go-to company for CPU coolers. A Freezer 13 is probably what I'd look at. Very overkill, but because of that the fan should rarely, if ever, spin above minimum. A Freezer i11 would work fine too.
You can get cases that do purely passive cooling, but I think it is more effort and expense than it is worth. Good fans are quiet enough.
Next issue is harddrives. Those suckers make more noise than you might think. If you have the money and can go all SSD, problem solved. If not, then you want to try and keep them dampened, if possible. I'd stay away from WD drives and go with Seagate or HGST, they make less noise in my experience (even though HGST and WD are the same parent company).
A case can of course help keep all the noise in. My top quiet case is an Antec P183B. They do a really good job with isolation in that case, building it out of multiple layers in a metal-plastic-metal configuration to attenuate noise, and they have dampening on everything including some nice silicone grommets for the HDDs.
If you want to attenuate it further, you can get acoustic foam for the inside of your case. Putting it on the walls will help stop the higher frequency fan noise from going anywhere and make it a little quieter.
Also make sure to consider your monitor. If you get a really quiet room, monitors can become an audible noise source in some cases. I love my PA301W, but it buzzes audibly. It isn't loud, but you can hear it when the room is silent since the ambient noise level is down around 20dB. TFTCentral does a good job reporting on monitors making noise so check on that before you get one. In general, LED monitors that do not use PWM dimming tend to make less noise than CCFL monitors or monitors that use PWM.