Insulating around the window should not include the process of taking vinyl siding off.
All you will find by doing that is exterior sheating which is run under your exterior window casing. You will just end up scratching your head in confusion, then reinstalling the vinyl siding.
On newer wood, vinyl, or aluminum windows, there is no way to insulate from the exterior at all because of the nailing fin.
On most older wood windows, you need to re-insulate by taking off the interior casing, insulating, then re-installing the trim, or if that's not possible for some reason, by taking off the exterior casing. That could be a 1x4, brickmold, or other dimension lumber that surrounds the window on 3 sides and sits on top of the sill. You can not insulate under the sill by doing this from the outside, so my first suggestion would be by doing it from the inside.
If you have wood windows with stops butting the sashes, these and the stool normally would not have to be taken off to insulate, just the casing and apron.
Lastly, if these are window weight boxes you are trying to insulate, you can drill an exterior hole and blow them full, or remove the interior trim and fill them with fiberglass. Stuffing insulation up through the window weight box cover will not be effective in filling the chamber properly.