Since it started having the issue once the new starter was installed, I'd guess the new starter or the installation is the issue.
Could just be a sticky solenoid on the starter itself, could be cheap, poor quality mechanicals in the starter, could be the starter gear isn't meshing well with the flywheel teeth and so doesn't disengage. Could even be that you were sold the wrong starter for the vehicle. Be sure the part number is correct and that you have the correct starter for the model year and the type of transmission it has.
I'd suggest removing the starter and inspecting the gearing on it and the flywheel, and perhaps even taking it to get tested to be sure of proper operation. When re-installing, check and double check for proper alignment, be sure the starter isn't angled and the bolts and mounting surface aren't causing it to be at even a slight angle. You may also need to check that the starter gear is in the proper position and engaging the flywheel teeth correctly, if not you may need to use spacers or shims, or figure out what the issue is.
There's a chance that the fender solenoid is just old/worn out and replacing it will fix the issue, if nothing is wrong with the starter.