I have the same problem, even though not all windows are affected. The worst ones are those that are exposed to sun and rain, and the ones where the sill is made out of particle board material. Never knew until it started to bubble up.

The ones that have eaves above them fared better. Most of the windows are about 10 years old now but the damage started years ago.