Nonsense!
Just prime everything so the paint will adhere. If the ductwork has been there a while, the manufacturing oil will have evaporated and you can use Zinsser's Cover Stain (oil) for everything.
However, if there is condensation on cold water pipes, it would be best to insulate them (also the hot pipes) before painting. You will have to experiment with the primer to be sure it doesn't bother the insulation. You may have to use a latex primer on it.