That's exactly how I do it. I was trying to find a bag that matched most of my work wardrobe, I tend to wear black and brown only. But I love bags, and let's face it, some bags just make an outfit! So, now I switch a few times a week and dress around it.