Yes, it matters a great deal, and it rarely has anything to do with quality. As a consumer, I don't want to support Chinese manufacturing. They've cheated to get a lot of the business, they're an environmental and human/animal-rights nightmare, and I greatly resent any company that takes part or all of their manufacturing over there only to charge the same amount--or worse, represent themselves as a "luxury" line when they cut as many corners as possible. Kooba started out being made in Italy for less than half of their current prices, and now that demand is up, they've cut their costs to the bone by going to China, but surprise surprise, the prices have more than doubled.
I highly recommend reading "Deluxe, How Luxury Lost Its Lustre" as it will open your eyes to a lot of what's happening. The author says that Hong Kong knows it's a Chinese holiday when they can see the sun because they're downwind from most of the manufacturing, and it's THAT bad for them air-quality wise.
China matters. A lot.