MJ: tough choice, I like them all! The brown looks like a tan in your picture... a medium/light neutral. So, if you only pick 1 other, I'd go for the red one.
I like the yellow one also, so I don't think you can make a mistake. After you carry the brown one around for a while, you will know how much you love the bag. At the price, if the style works for you.... go for all 3 colors.
My first choices would have been red and yellow, then tan. But I love color in handbags.
I have the pebbled E/W Derby tote... I think that's the bag you have pictured. I have the Grass green, and just changed into it last night. I like the bag, but for me, the lock, which is beautiful and easy to open/close, adds weight to the bag, and I think I might have a problem with that. Also, the straps are flat, not rolled, and I don't care for the rustic look of the underside of the straps. Those are just personal preferences, but I've used them to decide that this is a style I don't need in multiples.... as much as I love the look of the leather and the lock. I did get a yellow pebbled leather tote (Layla) at the end of last year. I was obsessed with getting another yellow handbag.