I'm of the belief that medium flaps work best for going out or formal occassions, so if that's all you will be using it for, I would go for lambskin. I also believe jumbos are great for day bags but are too big for formal events, so go for caviar in those. However, if you really plan to use the bag alot, don't want to have to worry about scratching it or getting it wet, I would go for caviar. Another option is to get a vintage lambskin flap second-hand. I truly feel as though the older lambskin holds up alot better than the newer lambskin.
Either way, you will have a gorgeous bag. Have you made a hardware choice yet?