Check the linkage. I believe the process is to block the tires, disconnect the shift cable at the transmission, have someone hold the gear shift dead center in "D" on the column. Then move the shift bracket on the transmission forward until it is in park, then back exactly 3 detents (R, N, D) which is "D" in the transmission world. Adjust the cable until there is no slack in either direction so that the column and transmission are in perfect sync.
When I did my valve body rebuild there was a modification to the manual valve that does the shifting. The stock system could allow the valve to move further than expected and energize (slowly burn) the forward clutch. There is a kit to limit its movement. The Z-link could also be bent or the valve's link hole could be bored out and have too much play.
Luckily there are a few things you can do to check the system WITHOUT having to drop the transmission! If after adjusting the linkage there is still a problem with reverse I'd drop the pan and check the condition of the fluid and look for debris in the bottom.
(Some people would recommend to just check the torque of the valve body bolts and look for gasket leaks, but since you've already dropped the pan and lost the fluid there is no sense to waste it and start all over again if you have to go back in IMHO.)
Remove the reverse servo piston cover CAREFULLY because it is loaded with a BF spring. Check the condition of the gasket, piston and the seal around it. If they look good, then it is probably worthwhile to rebuild the valve body (not as scary as it sounds, I thought it was like fun brain surgery) to make sure all the valves move freely and replace the gaskets to ensure there are no leaks. Just make sure to use all new gaskets. Call around to a couple of shops to see if they have the valve body gasket kit and new filter in stock. I think I got mine at Napa, in stock. Then make sure you torque it all down to the proper spec. After it is all back in the car you might want to also adjust the bands.
There is a wonderful thread by Glacier991 below on doing a valve body rebuild in the A4LD that starts halfway down the page in this thread:
http://www.explorerforum.com/forums/showthread.php?t=101571&page=3
You might want to also see if you can get the ATSG A4LD manual. Send me a PM for more info.
The reverse band is cast iron and solid when compared to the other bands. It could have also gone bad, but to get to it you have to remove EVERYTHING in the case in a full rebuild. Let's see if we can avoid that first...
GOOD LUCK!