Bags do not have genders! They may have been designed to suit what's considered to be a feminine taste, but that's very much cultural. We're all more or less conditionned to think that a bag designed for women would look better on a woman - but only because our eyes have always been accustomed to associate this or that design or color for women's fashion.
Have you ever looked at portraits of male, European aristocrats from the 18th century? Powdered wigs, heels, rouge galore on cheeks, laces and frills, tights - all of those would be considered effeminate today, but was very much in fashion and considered "masculine."
Perhaps a more relevant example would be male Asian beauty and fashion. With the spread of K-pop, Western audiences have been introduced to an aesthetic that would be considered effeminate. However, back in South Korea, a country where traditional gender roles are much more defined and rigid there than it is in the West.
Chanel in particular has always borrowed heavily from men's fashion. What made it so avant-garde was because Gabrielle Chanel would use materials, shapes and cuts which had been previously been reserved only for men's fashion. We have a tendency to associate tweed, for example, to an old rich lady, but that material made headlines back then because it was considered appropriate only for men. Not to mention the classic tailleur which is more or less a women's version of a suit (this wasn't considered appropriate during the first half of the 20th century - blazers and pants on women, oh my!). Why couldn't/shouldn't the reverse happen now?
Fashion is very much changing these days with more androgynous looks. It wouldn't look out of a place in a big city. And most importantly, if you like it, get it! Anyone else who thinks badly of it can go moan about it in their own little corner somewhere. These people probably would've been the same ones back then, belittizling women who would wear pants and not a dress, because it wouldn't be "gender appropriate."