To each their own. To be quite honest, I adore my Trim (I) and use it quite often but that's just me. It was literally a 5 minute purchase from the moment I walked into the boutique.
The great thing about Hermes is its wide range of handbag styles. Some may suit the majority due to media or just broad appeal - e.g. birkin/kelly. While others may require a bit more selective/individual tastes - e.g. constance, plume, bolide, trim, himilaya, etc.
So to answer your question, yes the Trim is a less popular bag for whatever reason. But I'm 100% sure that if say one of the Olsen twins, Posh or Kate Moss were to sport it, the popularity of this bag would definitely soar overnight!