It’s meant for wearing 24/7, so no need to worry about that stuff. Bedding and towels are way too gentle to buff out gold. Think about it this way: if it’s doesn’t “buff” out your skin, which is much more fragile than metal, then it won’t “buff” out your bracelet.
All you need to buy is a simple gold polishing cloth and it will clean your bracelet and make it shine almost like new. It’s extremely gentle and will only polish, not buff out any gold.
https://www.amazon.com/Connoisseurs-Gold-Polishing-Cloth-inches/dp/B000RB5S72