It doesn't sound like you have much experience with off-road vehicles so I'll give you a few tidbits that'll help.
4wd really just means the vehicle has the capability of putting power to each of the 4 wheels. It doesn't mean it is going to put power there.
Most have the same issue 2wd vehicles have. You know, when your in mud in a 2wd vehicle and only 1 tire spins. Well, a 4 wheel drive vehicle (without traction devices) has the same issue except it has it on each axle. That means normally you would only have 2 tires spinning and its normally the front left and the right rear. Basically its the tires with the least amount of traction which is getting the power. That is just how a differential works.
If you add a traction device like limited slip or a locking differential into the axle then you can get power to both wheels at the same time. The type and style of traction device will determine when and how much power can get to both tires. Since you saw 3 tires spinning you may have a limited slip differential which means there are clutches in there and pre-load on them determines how much power can get to the tire that isn't spinning.
So, since you were spinning and you had a few tires spinning (front and rear) you just need more traction for the amount of surface pressure your tires are putting down. Most likely the issue was your tires tread pattern. Street tires aren't very aggressive and once they get some mud in there they might as well be slicks.
A trick you can do to increase the contact patch and get more traction is to lower the tire pressure of the tires until they are about 25% shorter (sidewall height) than normal height. IIRC this can increase the contact patch by 200% (I can't remember the # but it was quite a bit). This will spread the weight out so your tires don't try to dig
ok.. enough rambling
As for the jerking, most 4wd vehicles to bounce/jerk etc when in 4x4 mode BUT it really depends on the 4x4 system. Systems that don't lock the front/rear drive shafts don't have this issue but then they have an even worse time once a tires starts to spin. Any 1 tire spinning causes the vehicle to stop moving unless something fancy is done. Some newer vehicles use the brakes to stop a tire from spinning so the power can be put to another tire.
Hopefully that info helps..
~Mark
