I have a damier azur zippy wallet and I absolutely love it. I purchased it pre-owned, now I think about it, I believe the previous owner didn't take much care of the wallet. I was new to Ebay and LV back then, so thinking back, the deal at the time wasn't exactly a good one (could've gotten it for cheaper). Still, at the time there was still a lot of life left to the wallet, and upon receiving it, despite some disappointment, I fell in love with the wallet.
This wallet is now going to be 5 years old. The previous owner used for about 2.5 years, and I've used for another two years now. The state of my wallet is exactly what it looked like two years ago when I first received from the previous owner. When I received it, glazings at the bottom corners were gone, there were scratches on zipper pull, and overall surface wear. But the point is, after using it daily for the past two years, its condition stayed about the same, never have gotten worse. I've not changed the zipper pull nor had it reglazed, because I didn't know about this back then.
I plan to purchase a new DA zippy wallet for myself, because after knowing LV for more than a year now, I am a bit disgusted at what the previous owner did to the wallet, and want to get a new one and wear it in my own way. I will continue to use this old DA zippy for international trips, because it easily holds two persons' passports along with cash and coin.
The point of my story is, I don't think you should worry about your DE zippy at all. If my DA zippy can withstand so much wear, your DE will definitely be able to withstand much more. The zipper pulls, like others said, can be changed in-store free of charge. And if glazing is a problem, you can always get it reglazed too. Use your wallet and enjoy it!