You need to give more information. What are the walls made of; drywall, paneling, concrete block? How were the shelves attached; screws through vertical shelf supports? Where is this wood filler you want to remove and what is it?
Why not post again so your question will be near the top of the page.