Bridget,
I'm not an exterminator but I had the same problem. I think they are called "digger bees". I used a granule insect killer with active ingredient Cyfluthrin. Apply to the lawn with a spredder. I then sprayed on a chemical concentrate called Esfenvalerate in a hose end sprayer over it. Killed them all. I applied 3 weeks ago and haven't seen on bee since.
Good luck!!