I'm with the same dilemma, but I already have a black bag so I'm trying to decide between the medium collége in grey or deep red ( burgundy)
I love burgundy shades, they are like my favourite but lately I've been loving the greys and the neutrals. I do find grey quite neutral and versatile and since I'm like you and use either plain clothing or bright colours and/or patterns I find that maybe the grey will be more versatile!
I just experienced my first instance of color transfer (on a pink leather bag) when I pulled it out this morning so I think that eliminates grey for me. I do find burgundy to be 99.9% as versatile as black, and so beautiful. For some reason neutrals are a huge struggle for me and I never reach for my neutral bags, they just don't really seem to suit me.