When it is cold outside(20F or less), my engine never quite gets up to temp.

I noticed that today, when the temp outside was -10F, that when idling or coasting, you could watch the temp go down. Also when the heater was on, it would stay down(about a quarter of the way up). This is not normal, because when it is warm out this does not happen.

Is this a thermostat problem, because it amazed me when I drove around for an hour and it never got warm?

Any help appreciated :)

Sure sounds like you need to replace the thermostat.
Its not a bad job if you can stay warm doing it.

How much does the t-stat cost, and how easy is it to replace?

If you have an OBD II scanner, or access to one, you can set it to monitor actual coolant temp and see if its a gauge problem.

I might have to do that before I buy a t-stat.

Now I just need to find someone with one.