Previous owners of our home painted over varnished woodwork. Paint is now peeling off woodwork everywhere, particularly when bumped. The varnish when painted had a crackled, rough surface and looks terrible.

What would be better - to replace the woodwork around doors and windows or strip the one layer of paint and varnish? I am concerned that stripping the varnish may become difficult. I don't think a good sanding will remedy this problem - if any varnish is missed or not sanded properly, paint will just peel again.
Thanks