It could be an intermittent with the relay, I have had that happen twice, basically random operation. My sense is that it is the switch in your case though. Fairly easy to test, here's how it's laid out:
The motor gets +12V from the blower relay, which gets power from fuse 5 of the underhood power distribution box.
Battery +12V -> Fuse5 -> Relay -> Motor
The motor gets ground through the blower switch and resistor pack.
Ground -> Blower switch -> Resistor pack -> Motor
The switch itself is (gratefully) one of the simpler switches in the Explorer. I forget if you have to get the dash bezel off to do this, but get to the back of the switch, whatever it takes. You will find two connectors to the switch. One is a 4-position in-line connector with two wires. That's for illumination, you can ignore that. The other is an oddly shaped 7-position connector with 7 wires. This is for the switch. The color codes are:
common ground - Black
Lowest speed - Red/Orange
Medium 1 - Yellow/Red
Medium 2 - Light Green/White
Highest speed - Orange/Black
The Pink and White/Light Blue wires are for the A/C portion of the switch, you can ignore those too.
So, just to state the obvious, with the switch on lowest speed, there should be continuity from Black to Red/Orange, and so on. There are no special cross connections; it's a simple one-pole 4-position switch.
If you do this test with the connector still attached to the switch, you have to disconnect the resistor pack to prevent false readings.
And, those color codes follow through to the resistor pack, so if you are clever, you could test the switch from the resistor pack connector, leaving the dash entirely intact. You would simply disconnect the connector at the resistor pack and measure resistance at the wires to ground at each of those settings.
This switch controls actual motor current, 10 amps or more of DC power. This erodes the contacts and they ultimately fail, often in unusual ways. I think you will find the switch is bad. Fortunately, they are inexpensive and not overly difficult to replace.