It does matter some. There is one bag that I have bought and my hubby disliked, but it has grown on him. He gave me his reasonings for not liking the bag, but he wanted me to be happy carrying a bag I loved. On the other hand, the last time I had my heart set on a particular bag, when we saw it in person - again, my hubby wasn't impressed and gave me his reasonings for not liking the bag. I was torn between 2 bag that day and I ended up going with the one he liked better. His reasons for not liking the bag actually made me think more about what he did not like and whether it would bother me later on.
My hubby is my best friend and I like having his opionion. Luckily most of the time, he likes the bags I have picked out anyway.