If you are removing the window completely, then by all means insulate it.
If you are just covering up the window, and leaving the window with glass there, (not recommended) I would think you would run the risk of water vapor condensation forming on the window and being soaked up by the insulation.
Dodgeman