I used to throw them away, now I regret it. Some bags are complex enough in design that they look better without the tags. Some need a little oomph, so I'll keep the tags on or maybe add contrasting tags. I have some charms that look better with a tag in the background.
So now I am keeping all the tags and matching chains, and designing my bag for the day depending on what I wear. I will probably keep my unused tags and charms in my "purse stuff" drawer so they are available when I feel like dressing up a purse.