In my opinion it's not worth trying to repair or remove the damaged drywall at this point...anybody with half a brain would have primered the drywall prior to wallpapering or painting. I would lay new drywall over it with a 3/8" drywall, tape seams, skim coat screw holes, etc...either DIY or have a pro do it. Will cause you the least amount of grief.
Hope this helps.