OK, thanks for the info, and hope you didn't feel like I was thinking you were a simpleton or something... Just always have to make sure the basics are done right.
I don't know for sure why the amp would still be coming on when you have removed the fuse, but here's a few things that could be the case: shorted fuse holder; a short to another power wire somewhere downstream of the fuse; Maybe the amp isn't actually turning on and just appears as though it is (Weird design of power indicator circuitry on amp that lights the power-on indicator any time there is 12v present at the remote turn-on lead); circuitry in amp is fried and allowing power to back-feed from remote turn-on lead. I placed those ideas in the order of which would be most likely to least likely.
As for the noise in the output, make sure your signal cables(RCA connectors) are routed as far as feasible away from power wires of any kind, especially the amp's power cable, as they will induce "alternator whine" in the signal from the magnetic field created by power flowing through the wire. Also, make sure each speaker has its own dedicated positive and negative speaker wires running contiguously between the amp and speaker (no grounding of negative speaker wires or using a common negative terminal on amp, each one must have its own positive AND negative.)
Hope this information is helpful to you, but if you still have issues after checking these things, feel free to post back another reply and try to be as specific about your problem as possible, i.e. what any noise sounds like, whether all of your speakers are powered by an amp, or you have some of them running off the stereo head-unit and some on amp.