It appears that the proper ground is not getting connected for the window operation, so you'll have to follow the ground circuit through the window wiring system and find out where it is not getting connected. Here's how I'd work through the problem:
1) Since your other windows are working, we can assume that the driver's master window switch is getting proper ground into it and since connecting ground makes the window work, we know that the driver's switch is connecting positive properly.
2) Since you can get positive voltage at each pole of the window motor connector, you know that there is no break in the wires between the switches and the motor.
3) Check for ground coming from the driver's door window switch at the passenger door window switch, if there is ground from the driver's switch, then the passenger switch is probably faulty-- replace it. If there's no ground passing through from the driver's door switch then move to the driver's door switch, unplug it and check to see if it is connecting the ground properly-- if not, it is faulty and needs to be replaced. Shortcut-- If you get positive voltage alternating at the window motor connector when you operate the driver's switch, but no ground, then the problem is at the driver's switch since the passenger switch is passing the voltage through properly.
My bet would be that the driver's switch is messed up since that gets used the most. I had a strange problem where the passenger window would only go down, not up. I disconnected the window motor connector at the door and my multi-meter showed positive voltage alternating between wires as it should when I operated the switch, then I checked for ground and it turned out that the driver's door switch was connecting the ground only in the downward position.