Smell

Quick question - we have a bedroom closet that shares it's back wall with our upstairs bathroom. When ever we use the fan in the bathroom the closet starts to stink with a sewer smell. We had a plumber out and he was little help and it wasn't until after he left did we notice the connection between the fan and the smell. Any thoughts on what this might be or how it can be addressed?