MJM5679 - hi, it'll depend on how comfortable you are with poking around to find what sounds like a nest. If the realtor's been seeing wasps all winter long, I'll bet they've gotten into a wall, up in the attic, into the crawlspace, or between the floors and built a nest. The minimal heat that was probably kept on inside the house was enough to keep them alive.
If you think they're coming in by a window, you can arm yourself with wasp/hornet spray and go looking around for cracks/holes where they're exiting from. However, they may be heading for the windows because they think it's a way OUT of the house (they're attracted by the sunlight). I had this problem in the last rental house I was in, and I found the wasps were coming down around the chimney and getting in around the fireplace insert. They hadn't built a nest in there, and all I could do was go up and put a couple overlapping layers of wire mesh (like for screen windows) over the chimney opening to keep them out. (When I wanted to use the fireplace in the winter, I had to go up and remove the mesh...I found that out the hard way!)
You may have to spend some time just watching to see where they might be coming from. It may be that they're coming out of an air vent, up through the floor, down the chimney, etc. That's how I figured out where mine were coming from. Then, if you can get access to follow their flight path back, you should find a nest that can be treated to kill them and then be removed. You may want to call in an exterminator if you feel it's beyond your capability...and if it helps you avoid getting stung, there's no shame in calling the experts!
The worst situation I had was in a different house I was renting. It was a 102-year-old farm house, no insulation in the walls, etc. One evening I heard something banging into the screen over the kitchen window while I was eating dinner - it sounded like hail. It was a small swarm of red-faced hornets, trying to head to the light! I tried using a fresh can of wasp/hornet killer on them and it didn't even slow them down, so I ran back inside and waited until early the next morning. I ended up spending a couple hours the next afternoon/evening watching for them and saw they were going in and out of the gap around a drain pipe that exited a second-floor bathroom that was added to the house some years back. Over a period of about a week, each morning before I started my lengthy commute to work I'd go up on a ladder and empty about half a can of spray into the gap, and I'd liberally coat the edges of the wallboard and the drain pipe surfaces. It apparently started slowing them down and I found a couple small dead hornets on the ground. (One of the bigger ones I managed to knock down with the spray was about 1.5" long!) I got up there the next morning and used a can of expandible foam to permanently seal the gap around the drain pipe. That drove off the hornets that weren't trapped inside. We never did see any hornets inside the house itself and trust me, we kept that bathroom closed off until we were sure they were dead and gone!
I hope this helps you some and I hope nobody ends up on the wrong end of any of those wasps! Good luck! Jim D/West Point, VA