Marble Window Sill Problems

Hello, new here.

We've been in our house for five years now, the house was built in 1940. We have beautiful marble window sills in each window in the house. These never show ANY signs of moisture, although they do get very cold in the winter.

At the outside corners of some of the sills, the plaster walls are starting to buckle and crack from some sort of water damage, I presume. We have caulked all of the windows outside.

Can anyone explain this to me or offer any advice on how to fix this problem? We love our home and do not plan on moving for the next 30+ years. Thanks in advance for any help.