I never wear any of my jewelry. I always buy it but hate wearing any of it.
That being said, I wear my Hermes Apple watch in stainless steel all day every day and it coincidentally matches my bags. All of them have palladium hardware.
This probably will seem odd, but if the bag is PHW I wear whatever, but have so much silver/WG that it’s usually that. But I do have a few GHW bags that I almost always wear matching jewelry. Funny but that‘s how it winds up. GHW is less forgiving in terms of mismatch, in my opinion. If I have to wear a certain silver piece, I will wear a multi/mixed stack of rings to blend the look.
I don’t feel it’s important for my bag hardware to match my jewelry (mostly yellow gold). But once I add in hardware on my shoes or belt, I prefer to keep it consistent with my bag. Not sure why that would matter though..