I have been using "Rainbow Loom" non-latex bands made of
silicone elastomer. They are most conveniently available to me
from Amazon, but there may be other distributors. Ref.
https://rainbowloom.com/collections/rubber-bands-opaque/products/rl-band-opaque-white 

The silicone polymer may lose some elasticity over time, but it
will not degrade in the way latex-based elastomers and even some
polybutadiene (neoprene) elastomers will over time.

Marshall Jose