Just think how easily you'll find it in your bags, specially the black ones! I think that if you love it and can afford it, then why not. Every piece of Hermes is beautiful as a 'stand alone' item. I would always buy for each individual piece rather than matchy matchy. My only caveat would be, have you been tempted by colour before, say in a top, only never to wear it, if you are strictly a 'neutrals' girl, then perhaps you should hold back (this applies to me!)
I use my Rouge croc wallet with everything.....Hermes gold, black, and my LV Speedy....
Do you have a pic? Would love to see it with the gold...