Yes you need to pull it, but this only takes a few minutes. IIRC the tensioner pulley needs a 3/8" ratchet or breaker bar ( no socket needed ) to loosen the belt to get it off the alternator.
How old is the battery? If I were you, and had a multimeter, I would disconnect the alternator charging wire to the battery, and measure current between it and the battery with a multimeter (only with engine off). If there is no current, then I would leave the alternator in for now and take the battery to be tested.
If there is current, the alternator probably has a leaky diode and needs replaced. However previously I had suggested charging the battery. If you can do that, leaving the cables to the vehicle disconnected, and find that the vehicle then starts up good right after connecting them, it means your battery is good and either the alternator is suspect, or you may have some parasitic drain causing the battery to run down.
Left in too low a run down state for too long, this can kill the battery even if the battery wasn't the primary cause. Same for a bad alternator eventually killing a battery, but the way you put it "started to slow down starting", this seems like a problem slowly getting worse while if your alternator was shot I would think it would more rapidly cause a complete failure to start or even cause the engine to die while running, all the while with your gauges going haywire as the voltage from the battery dropped.