I think the classic flap works best with casual simple clothes better than dressed up, actually! My favorite looks are classic flaps with great jeans and simple tees or sweaters that really make the bag stand out, or slightly more dressed up dark pants for the black or darker colored flaps. Blazers and good coats also look great with the classic flap.
On the other hand, with more formal dresses I just don’t think it looks right, and a clutch would look better. With sun dresses, I tend to prefer other more casual looking bags that go with the look better like Chloe. I also don’t think it looks good with athletic wear, that is just way too casual. But wear whatever you want with it that makes you feel good, regardless of what anyone says! Pinterest is a great place for lots of inspiration, here are some I grabbed.