I'm a changer. I do have certain fobs that I use repeatedly on certain bags. Like on my black Tattoo Tote, I seem to always use the skull & crossbones fob. I don't know why, it just seems to go.
On my other bags, if I don't feel like changing bags that day, but I feel like some kind of change, I'll change up the fob I have on it. I used to use scarves, but I got tired of them very quickly as (for me) they get dirty and/or come untied.
I like the charms with the lobster clasp because that makes it easier to change, but what I've noticed is that certain charms match perfectly to certain bags so I'll just keep the charm on that particular bag. I probably have enough charms to have one on each bag, but there are a couple I like to put on any bag because they match everything.
I buy fobs for particular bags, and leave them on. My Pisces fob is ALWAYS on my glam tote, Ladybug on Sabrina, Legacy Lipgloss on Peyton. Maggie and Hailey are fob-less, but there really isn't a good place to hang one.