I carry mine - but mine are funkier, slightly distressed styles in leather, not logo, and would probably not be considered 'designer' anyway by many people at my office.
Sure. I don't do logos though, so I don't think anyone knows that they are even designer, to be honest. Where I live,
Coach and Dooney are what is considered designer for the most part and if you don't care those, people assume it's just a normal bag from wherever.
elizat, I think we work in the same office! I'd get blank stares or an "mm hmm isn't that nice, now" response if I said RM, Kale or LP, etc. Honestly, even if I carried a Balenciaga or Marc Jacobs, I'm not sure how much recognition it would get by even the very fashionable and well-dressed in the office. It's just not their style, I guess.
Neat story though: two of the women in my office went gaga over my RM Morning After Bag in Night Blue, in a really genuine way, so I was happy to introduce them to the RM world

One even went to the sample sale and came away with a few goodies!
As far as office politics, I am one of the office plebeians (seriously, there is such a crazy social hierarchy there), and it would be shocking for me to be seen carrying a pricy recognizable logo bag. This has happened to a friend of mine there.... seriously, anything for gossip in that place!
In certain non-profit environments, I would feel uncomfortable wearing anything flashy and recognizably expensive (which is why this month's "foolproof outfit" in Lucky scandalized me a bit) ... but I also think it is silly for adults to criticize other adults in the way they spend their disposable income. That is reason alone for me to wear what I enjoy anywhere, haters and gawkers be da**ed.