Dear Jay J,
I agree with your comment that my PVC pipe shouldn't freeze if there is not water in it. Actually, I'm not sure why it is freezing at all, but I've poured water on it twice and that got the water pumping again.
As far as the piping arrangement, the pump is in the basement and pumps vertically, through a check valve up to the foundation level. It exits the foundation horizontally and protrudes from the foundation wall by about 6". It then makes a right angle downward. About 1.5 feet of pipe is visible before it submerges underground. The pipe is then gently slopes as it extends about 20' horizontally from the house.
Due to the check valve in the vertical section in the basement, I'm sure there is water in that part of the line when the pump shuts off. However, there shouldn't be water in the section that sticks out of the house. Without looking as I write this (its dark), my guess is that the horizontal section that protrudes from the house is somewhat sloped back toward the house. It's that horizontal section that freezes. If I can somehow, slope that horizontal section away from the house, then whatever water is in that horizontal section should drain away from the house.
Thanks for giving me the idea to check this.