Like previous posters, I'd suggest using a fine sandpaper or
some other abrasive.  I'd follow up by using a lacquer pen of
the appropriate color to cover the bare areas on the flange.

Rust has a habit of coming back, so anything you can do to keep
air off the bare metal will help.  Lacquer pens can be found in
craft stores and hobby shops.

Ed Chaban