I’ve been in the same position. Then I read something on the Bottega Veneta forum that started to change my thinking. A poster mentioned advice her SA had given her: “don’t buy bags in colors you wear, buy them in colors that compliment the colors of your clothing.” At the time, I was purchasing bags in various shades of blue, as I wore a lot of blue clothing. But none of the bags were working for me color-wise. I stopped trying to make blues work and started adding colors that weren’t colors of my clothing, but colors that complimented my clothing. I first added a Barolo (wine colored) bag and found that it worked perfectly with my blues. While I kept wearing my bright colored tunics and sweaters, I stopped buying any bright colored bags.
I think the change worked for me, in part, because I don’t wear black clothing (except for An occasional pair of black pants). I know....I’m a freak of nature to not own any black sweaters or shirts.
Now, I am moving quickly to black, off-white and reddish/maroon/wine colored bags. Those three neutrals work with my entire wardrobe. Actually, black would work with everything, but I think that would be too limiting in my hot climate...and in my thinking right now.
Anybody else have a ‘color theory’ for an edited bag closet? I’m always searching for more information to try....