You will need to work your way down the grits of sandpaper. Start with 100 and progress to say 300 and that should take care of the fuzzy wood and be sure to sand with the grain of the wood.

Im sure what you are talking about is the little slivers along the edges mostly on the sides of the panels.
I use either 220 grit or steel wool,and they come off real quick,but where gloves be cause these are a bugger to get out of the fingers or palm of your hands if it happens to stick you.