I think it's usually rude, but depends on the person and how they say it. I mean, if my best friend asked me the price I wouldn't think much of it, because she's probably just genuinely curious and not asking to judge me. But I'd feel uncomfortable if, say, my future MIL asked me. I guess it has to do with their intent.
I'm not going to feel guilty about what I spend on my bags, because everyone has their things that they like to spend money on. It may be purses for me, but it might be shoes for my friend, electronics for so-and-so, etc. A lot of people just don't understand it, and that's fine because they don't have to. I don't understand some of the things that others spend their money on, but that's fine, it's their money. Like me and my mom, she doesn't understand the purse thing (and why you need more than one or two of them), but I don't understand one of her hobbies that she dumps $ into. Oh well, we joke about it.