The wealth question is always so interesting to me. No matter how affluent you are, someone will be wealthier than you. I work in tech in Silicon Valley and see all sorts of crazy money -- someone was literally telling me over lunch today how stressed out he is he doesn't have his own private jet and has to do a jetshare.
The problem with Birkins / Kellies is that most newcomers to the brand only want them (and then, in a certain size / color combo, depending on trends). Hermes is very good at capitalizing on that, especially at a larger, established store like Toronto, where there are a lot of consumers better knows to SAs. If you only want a Birkin or a Kelly, go the reseller route. If you want to explore the brand, there is no such thing as too wealthy or too poor. Just go for it.