I would suggest:
1. Checking wires for corrosion, including but not limited to: Ground wires, alternator wiring loom, battery cables.
2. Check battery connections, make sure everything is tight and corrosion is cleaned away.
3. Check fuse for alternator in the underhood fuse box and also check the fusible link connected to the alternator harness.
4. If all else fails, I'd remove the alternator and have it bench tested, might also want to test the battery while you're at it.
Good Luck.
Edit:
I will say, when I press the window up switch, when the window is all the way up and I press the switch it will drop the gauge, but only because there's a serious load and the motor is working hard. But it's always done that, but if you're just idling or driving and the headlights make it dim, something else is going on.