If all you are doing is replacing the speakers in the rear, then all you have to do is remove the metal speaker cover grill. Put a flat screwdriver on the top or bottom and pry. The metal grill will pop out giving you full access to the speakers.
Now if you have to do some modifying to make non-standard speakers fit then the panel will have to come off. To get it off, you will have to pop all of the plastic decorative rivets out, remove three phillips screws from the floor, remove both front and rear seat belt bolts (one in the panel and one in the floor), remove the plastic carpet trim where it meets the bottom of the hatch, remove the carpet trim at the bottom of the doors, pull the plastic door liner from the top of the doors, remove the bolts from the clothing hooks, and remove the two bolts holding the rear window pop-out mechanism on.
Remove it enough times and you can do it in slightly longer than it took me to type this.