I think it depends on the wrist piece... that should be the determining factor. For example.. I have one wristlet.., but the wristlet is a bracelet connected to the purse. I've seen some wristlets that look like makeup bags if that makes sense, because the bag is tiny and the zipper detail, etc.
I think you should do what makes you happy, and I know wristlets are safer because you don't have to hold it like a clutch, but clutches definitely look better.