I live in a three storey end unit townhouse with a gable roof. Whenever there is a fairly strong wind there seems to be wind getting behind the brick of the exterior walls and pushing at the rigid insulation creating a creaking noise. I can't figure out where this wind is getting in - any suggestions?

PS: soffits are not perforated

Wind Problem Gable Roof