Hi Croissant!  
I have a white 30cm birkin in Clemence.... white is white, so you will get a little paranoid with it.  However having said that mine doesn't look so bad (to the outsider) and I have had it for like 18-24 months now (and I am not a careful user).  The inside of my bag is filthy though (because its lined in very soft leather... something like kid skin)... I have pen marks, dirt etc. and the handles doesn't look so hot either.  If only I knew earlier... I would definitely recommend those purse linings.  Apart from that, I really love my white bag, it's so easy to go with everything...(I use it so much more over my BJ).
By the way... Kate Moss, Sarah Jessica Parker and Victoria Beckham all have white birkins I believe.
At the end of the day you have to decide... if you want something that will last to the next generation, then white Clemence will definitely not stretch as well versus a black box Kelly for example.  However if you want something quite stunning and more unique (versus the blacks and browns)...
I think you should go for it... since you sound so in
with the colour.  It's daunting at the beginning but you will get over it and it's so stunning!  A complete black outfit with a crisp white Hermes?!!  
   Me I figured, I might as well make the most of it... use it as much as I can and get my money's worth... it's only a bag after all, it's meant to be used and I cannot let it rule how I live!  
   Good luck with your decision!