I have the same issue with my '99 XLS.
We had the dealer replace the 1st armrest under warranty, then shortly after the 35K warranty expired, it broke again within about a month. Go figure.
We've been dealing with it for ~75K miles, and the wife is beginning to get irritable about it now...