Cleaning of musical box cylinders is of course to be taken with the
same delicate treatment that is given to handling them, but it's not
a difficult task.

I use Brasso or Wright's or any other specific brass polish (not
a generic metal polish!).  Test it on one first because once I used
a brass polish that reacted with the steel pins, leaving marks that
looked like copper around the pins.  It was awful, but an additional
polish took it right off.

Use a good amount of polish and scrub it to get the grime off.  Use an
old tooth brush for this.  You can simply wash it off with water, but
dry it well and use a blow dryer to make sure that there is no residual
water.

Finally, once you've gotten it clean enough, polish it one last time
with a thin amount of polish and let it dry.  Use a clean toothbrush
to remove the polish this time -- they should clean up nicely.

Eli Shahar