How much of a rise are we talking about? I would be inclined to locate the spot where the probable offending bit of gravel is located and place a 4-6" scrap of 2 X 4 over it and give it several good raps with a heavy hammer and see if this doesn't cure the problem. You would then need to see if any mounting screws or nails needed to be snugged up in that area.