To me that definitely does sound like a master cyl issue. Just because it doesn't have visible leaks doesn't mean there's not an internal leak.
When pressing on the brake pedal, the master cylinder uses your pedal pressure to push a piston via a pushrod which causes the piston to extend and push fluid through the brake lines, causing your car to brake. When you let go of your brakes, the internal spring that is behind the piston does its thing and returns the pedal back to resting position. The internal piston then rests behind the channel that allows fluid to be taken from the reservoir as your brake pads wear down to maintain the same brake pedal force throughout the life of your pads.
I'm leaving out the function of the vacuum booster because it wouldn't produce the effect you're talking about - a failed vacuum booster will cause the pedal to either be super stiff, not release/go back to resting position as quick, among other issues.
The reason I believe it could likely be an internal failure on the master cylinder is because as the piston is pushed forward, supposedly building pressure in the brake lines, you can see through this diagram that there are several seals internally in the master cylinder. If those seals fail at all, you'll have an issue with building pressure to the brake lines as some of the fluid will want to return to the reservior.
I don't believe it's an issue with bleeding air, as you didn't do any work to the system, but it couldn't hurt to double-check and try that. Check your reservior fluid level, make sure there's enough, and try to bleed the brakes of air.
After you have made sure your brakes are bled (if they aren't, it WILL mess with these results) get in your car with it turned off. Pump the brakes until the pedal becomes firm, so as to not let your vacuum booster mess with the test too. After the brakes become firm, hold the brakes down. If they're still spongy but holds you could have another issue (caliper siezed, etc). If the pedal slowly drops down to eventually the floor, or close to, you've almost definitely got a leak somewhere due to pressure escaping.
Good luck! Update me what I said helps, or if you find the problem! Always remember I could be wrong - do your own thinking and take what I say with a grain of salt.
