I was wondering if I should dig deeper into the plaster area that had water damage and then spackle it? Or, should I just scrape the paint and just prime over it?

You need to get back to a sound substrate and then fill it in. If it is large or deep then you will want to fill it in with a product like Durabond setting compound rather than spackle, then you will want to prime the entire ceiling and apply your finish paint.