It hasn't bothered me at all. I think the venetia is really easy to carry and functional (b/c it's structured, it's easy to keep things nicely organized). The flap (at least on mine) has softened over time, so it's really easy to just place the buckle on. When i know i'm gonna need to get in/out of my bag,sometimes I just leave my my bag unzipped, and place the buckle on the flap (without even placing the flap in the loop). or just use the zipper...the flap doesn't look messy or anything...(esp. when i'm in jeans/tee , the casual-ness works...)
Above all, it's a nicely structured bag that you'll appreciate.