Welcome to the forum!
This member introduction forum is more about just introducing yourself while you will get more topic views by posting in the appropriate 5th gen forum. Perhaps a moderator can move the topic there.
The size change, that will make the ride smoother, at the expense of (mostly) cornering (and rapid lane changing, etc, turning) stability, and yet, cornering stability isn't great with high center of gravity vehicles anyway, plus the taller sidewalls on the 18" give you more protection against potholes. To clarify, these do not handle badly due to a switch from 20" to 18". We're talking that last 5% difference if you drive it like you stole it.
The key to keep the speedo correct is switch to the same (close enough) diameter tire. When the wheel size goes down, the aspect ratio goes up. The right size in 18" is probably 245/60R18, but if your vehicle is programmed for something else, a dealer or DIY Forscan app with an OBD2 dongle, can program in a new size. You can compare tire dimensions between different sizes here:
The best tire size calculator. Use our tire size calculator to find tire size specs and suggested tire sizes. Then check prices for all available tire sizes.
tiresize.com
Plus the 18" tires are less expensive.
There is no clearance issue or computer/ABS/traction-control/etc issue if you stick to same width and diameter tires like the size I listed above, or it is within the programmable sizes the vehicle expects, and get the right rims, most easily just getting some 18" 5th gen rims, with the appropriate offset for an Explore of this generation. Even with the heavy duty brakes that have the larger diameter rotor and caliper spacing, that fits within 18" rims fine.
Some people even go with wider tires and a little larger diameter. That is a separate topic you can research if you want the pros and cons of doing that, but you mentioned no reason to do so, so there is no expected benefit - except, a higher sidewall ratio, will soften the ride a bit more, just as it does when it happens from moving from a 20" to an 18" wheel, and since it holds more air, you can run a little bit lower PSI too. I'm throwing the (possibly unpopular idea out there) that you could even go with a 245/65R18, maybe even an 245/70R18. You would need to research this and it might be more bother than it's worth. It would definitely be best to recalibrated/program the speedometer for these size increases. PeterK9 will come along and tell us whether that is within the 3% recommendation.
As far as whether it is worth the hassle, I would wait until your current tires are due for replacement, unless you have a spine problem where every little bump in the road is painful. I can't answer the question of whether it's worth the money to buy new rims. I mean I'd look for good condition 18" alloys at a junkyard first, then you can sell your 20" rims to make up the difference. Would I buy brand new OEM 18" aluminum alloy rims from Ford (dealers) for an 11 year old Explorer? No, but that's just me. YMMV.
What tire PSI are you running right now? I don't want to advise running PSI too low for safety reasons, but if you are not hauling loads nor 1000lbs worth of people at a time, you can also drop the tire PSI some to have a bit softer ride, whether it be the current 20" tires or 18" replacements. Personally, my '14 has 18" wheels and I wouldn't drop below 30PSI in a low load state, and normally shoot for 35PSI, then air up closer to tire max rated PSI when I have a significant load.
Lastly, you did not mention your most demanding uses, nor what tires you presently have, but LT (light truck) rated tires tend to have stiffer sidewalls and ride harsher than P (passenger) rated tires, and touring tires tend to have less vibration than all terrain tires.
