I had to do this to move the bracket up for front hitch clearance. Its held on semi-permanently from the factory with two plastic/metal rivet-like fasteners. Even with the bumper off there is no easy access to the rear of the fasteners, you could probably drill them out. What i did was just carefully pull, pry, and wiggle till I got it off, but I had the whole bumper off mine when I did it. The fasteners end up breaking and it didn't damage the bumper at all