Are you sure it's a fuse box? On my 1999 Explorer, the fuse box under the hood is on the driver's side, up near the fire-wall. On mine there is a small tab on the back (as you face it standing on the driver's side of the vehicle) that I depress, and then the lid flips towards you.
PS. My windshield washer reservoir, and I think pump, are on the passenger side.