Art, I'm Robbie's brother.  I work at Hewlett-Packard as a mechanical
engineer designing inkjet printers.

Neoprene is a poor choice for a friction drive.  As you have
discovered, it crumbles readily.  It was developed to have high
resistance to petroleum products, and to ozone.

I'd look at urethanes for excellent friction and wear properties.
These are used for synthetic soles on shoes, and for the wheels on
the in-line skates (roller blades).  It might be that a shoe repair
shop would have urethane blanks or slabs which you could shape to
the proper size.

Machining the urethane will be tricky as it is very tough.  It can be
trimmed with a sharp knife, and can probably be ground to an accurate
cylinder if it is first frozen with liquid nitrogen (or possibly with
CO2).  It may be possible to grind it at room temperature.

Good luck.

John Rhodes