Your assumption is correct. If you do not intend to use the factory radio at all, then if you could find your own power, ground, and accessory wire, you're good to go - that is, with the assumption your head unit is a basic one that isn't tied into other options. I'm not familiar with anything above 94, so I'm making a guess here, since I think some 95s and up have integrated features, but I could be wrong. Not sure how this would relate to people with power antennae and stuff or how you'd go about hooking that up, thus the reason most people just break down and get the $15 adapter. There's a point to "Do-it-yourself" but there's also a point where it might not be worth your time like finding new grounds, and wire tapping. After that, you'll just wire up your own amps and run new wire to each speaker.
Later,
Jon