Nah, I don't think you need a new engine, just a tune-up. Change your fuel filter, have your injectors cleaned, plugs and wires, you know, normal maintenance. The rattle under load is actually a fairly common problem for the first gen 4.0, it is generally not valves, but lifters or rockers. I have heard of some guys getting that loud tick from their coil pack even. Sometimes though, it is just a timing/mixture problem though, which can be caused by exhaust leaks before the O2 sensor, bad O2 sensor, plugged injectors, or EGR problems. You solved your EGR problem, now you need to find your exhaust leak, which can sometimes be difficult. Honestly, if you spent a while with EGR leaks, I wouldn't even be surprised to hear your O2 sensor went bad. Besides, O2 sensors are only designed to last for 60k-100k miles. They usually last a lot longer, but 60k-100k miles is the service interval.
The places to look for exhaust leaks are where one pipe meets another. On the passenger side, the manifold to y-pipe joint is pretty easy to inspect, on the drivers side, it is a little harder. They are sealed with exhaust doughnuts, and those things do deteriorate over time. Under the car, you have a flange where it joins the catalytic converter, and after the cats, you have another joint that is sealed using an exhaust doughnut. Your last place to look for an exhaust leak is where the manifold meets the head.
The noise and belt bobbing problem, that sounds like a bad idler pulley, but the best thing to do is to pull the serpentine belt off, then see that all the pulleys spin freely. Before doing that, you can check that your fan clutch spins freely, and look for large cracks in your fan. When your vehicle is off and cool, your fan should be able to spin freely with the belts on.
The oil leak.... Go to the store and buy a couple cans of engine degreaser. Get your engine nice and warm, apply engine degreaser, wait, then rinse. Repeat until the engine is not greasy. From there, you should be able to find leaks. It is not unheard of for an old engine to seep a little bit of oils out seals, and a little seepage isn't going to hurt anything, but it is good to nip those things in the bud when they are discovered. Another good thing about cleaning your engine, it will make those exhaust leaks easier to find. If you still can't find or feel an exhaust leak, you can visually inspect the engine for dark colored streaks of carbon. You find one, you found your leak.
You aren't mentioning any problems that sound head gasket related, so I am pretty sure you can rule that one out.
Actually, most of your problems could possibly be caused by your exhaust leak. You may also have a bad O2 sensor. When you have gotten your engine nice and clean so you can determine where your leaks are coming from, the next step will be to do a tune-up as soon as is financially convenient. Plugs, wires, fuel filter, injectors cleaned and probably replace your O2 sensor, since chances are, it is bad anyways given the number of miles on your truck.
Your transmission issues..... I usually don't actually recommend DIY tranny swaps to novice mechanics, because transmissions are heavy, and they are awkward parts to lift without the proper tools and experience. I can lift an A4LD into place by hand, but I still use a tranny jack or a jack with a tranny adapter on it, just because if you slip, you are underneath the transmission, and it has the potential to cause you major booboos. I would recommend just having a shop swap transmissions for you or rebuild yours, but that would cost around $1500. The only better advice I can give you, make friends with a gearhead, and buy lots of beer.
One last thing.... Cribb, I so wish I could find 5 speed sploders or 5 speed 4.0 rangers around here. I'm up to 6 1st gens I have bought now for people or parts, still have yet to run across a 5 speed 4wd. I found one 5 speed sport, but it was 2wd. Not a lot of sports in this area though, I suppose if I could find more sports, I would probably be able to find a 5 speed 4wd.
Mine will eventually get a 5.0 and a C4, but again I don't rely on it for transportation; I have a Japanese clown car for that 
