You can try to clean the straps daily with a mild soap and water to remove any oils, lotions, etc. Avoid using any brushes on a daily basis as it can shorten the lifespan of the straps.
Here's How to Clean Dirt and Grime from Rubber Watch Straps shared by HowToCleanStuff.net:
Things You Need:
Baking soda
White toothpaste
Rubbing alcohol
Cotton ball
Water
Toothbrush or other soft brush
The Cleaning Process:
This process involves a series of steps to tackle the removal of dirt and grime. Once you successful remove the dirt, any additional steps can be disregarded.
Begin by applying some regular white toothpaste to the area.
Rub gently with your fingers or a soft brush (only use brushes in cases with heavy build-up as the abrasion from the bristles may damage the surface).
Rinse with clean water and dry thoroughly.
If the dirt and grime still persists, moisten a cotton ball with rubbing alcohol and wipe the dirt away.
Rinse thoroughly and allow it to dry completely.
If the dirt still remains, mix baking soda with a small amount of water to create a paste. It should be the consistency of toothpaste.
Use your fingers or soft brush to gently rub the paste onto the rubber strap. Avoid using too much pressure as the abrasives from the baking soda and the soft brush could damage the surface of the strap.
Rinse with clean water and dry thoroughly.
Even with the best care, rubber watch straps typically last a maximum of 18-24 months. After this time, it is usually necessary to replace them.