In your case, I'd guess that you're looking at having the driveway asphalt removed. How much of the gravel base, if it exists at all, that has to be removed will depend on it's condition. I'd guess that it's far too thin...or it sits on an organic sub-base...a combination of the two...or it doesn't exist.
Organic soil left under even the best of gravel bases will be subjected to settling, compression, and future compaction just be the weight of the vehicles that use the driveway (or parking lot). They really should be removed down to mineral soil before coming back up with good gravel.
Any existing gravel under your pavement that has been contaminated by old organic soil underlying it should be removed and replaced, too.
A good base, well compacted, will give long life to your asphalt.
And don't skimp on the asphalt, either.