Metal on metal increases the rate of wear and tear, so metal pitting and oxidation. It's a cosmetic issue that has nothing to do with the zipper pull's function and strength, so you won't actually break the zipper. I used to attach my locks to my some of my zippers. I stopped doing it after I noticed that it was pitting in under a year. My non-charmed zippers still look new, years later.
Now if you pull on the charm moreso than the pull tab itself, it might cause the top of the slider to lift too much (the part that hooks into the top hole of the pull tab). Too much lift and the pull tab might just slip off. This happened to me in the LV boutique actually lol I grabbed the leather ends instead of the pull tab, and it came out. Whoever assembled the slider + pull tab part didn't crimp them enough.
If you know how to fix a slider and pull tab, then go ahead and attach charms. If you don't, then ask yourself if you're willing to learn, or take it to LV for a possible repair.
You know what the risk is with charms, so ask yourself if you're ok with the fix afterward, should it come apart. I don't think you have anything to worry about, as long as you keep an eye on the gap between the slider base and hook. If it starts to get a little too big, crimp it closed. Pull tabs separating from a slider don't bother me, since I can fix that by myself. A zipper that has seen corrosion is something I can't fix, since it would require a replacement/new parts.
do you by chance know how much it would cost to repair the zipper and/or the zipper pull itself on the PSM in the situation that it does wear/break over time ? was planning on putting the fleur de monogram charm on my PSM too !! i only assume its alright to do because they always display PSMs in the boutique with charms