Nope, never even thought of matching bag hardware to jewelry until I read about it here on TPF.
Much of my jewelry (inc. wedding ring set) is both white & yellow gold, so I like to mix it up.
But I'm one who doesn't care what color the hardware on my bags is, period.