I keep small evening bags but other bags, I would sell or give away when no longer in use. I find it depressing to have an old "friend" sitting unused in my closet, I'd much rather pass it on to somone else who might love it.
I just rotate them - I only buy bags that I really love therefore I tend to not have masses of bags from the same label. I find I use a different bag every day according to the weather, where I am going, what i'm wearing etc-.
I do have some bags that I am convinced will appreciate in value and I tend to actually display those ones in my house almost as museum pieces! I have two Lulu Guiness collectables for example that are editions of only 500 and a Moschino "chocolate" bag that is actually a museum piece (its in the V + A museum in London) so those almost never go out unless i'm going to a really posh restaurant or wedding.
The vintage bag scene is huge and if you keep track select ones go for huge prices so tell your husband that when he moans about how much money you've spent! ( I do!)
I tend to rotate my bags depending on how I feel, the season, and what I am wearing. There are definitely some bags in my collection that I hardly use, but I don't really consider them "retired" since I occassionally bring them out.