Well the lovely thing about have the freedom of speech is we can say what we want. We don't always have to sugar coat things. This isn't a perfect world and never will be. Wear a womens bag thats fine don't expect everyone to accept your choice. Whether that be a straight man carrying a purse or a gay man carrying a purse. If you spend so much time defending your choices it sounds more like an internal battle of continuously having to give rhyme or reason for your decisions.
If an open ended question like this is asked everyone should have expected different views to come up. Just because we don't have "culture" doesn't necessarily mean we are wrong. In some cultures being a feminine guy is highly frowned upon and those individuals are ostracised if not damned because of their cultural beliefs. Using others "cultural beliefs" as a scapegoat is a way of showing you aren't comfortable in your own skin so you find a way to attach yourself to something else for that approval.
If you are a guy rocking that Watercolor Speedy awesome, just know that there will be whispers and pointing fingers ALWAYS.
The baby in your pic is really ugly. I love freedom of speech.