It could possibly be the turn signal/wiper stalk (aka multifunction switch) - do you have any problems with the turn signals? When partially depressing the stalk for left or right turns do you get an electronic/metallic buzzing sound? When the intermittent are working, do they occasionally act erratically? 1 swipe...pause...3 swipes...1 swipe...2 swipes...etc? Do your sprayers work each & every time you press the endcap?
This is a common problem on older Xs and yes, ours are getting pretty old...it's $60 at Ford and an easy swap.