I have several bicolor Birkins, but they also don’t have bicolor exteriors - they do, however, have contrast stitching. They are all pop colors and I wear them all the time.
i think of it this way - if I’m wearing black or a similar neutral, the colors will go. If I’m wearing colors, I chose bag colors that I wear or are similar to what I wear, so they go. It’s fine to do fun colors, just make sure that what you pick goes with your wardrobe. All my interiors are shades of pink (so for me that’s a match). Exteriors are colors I wear - blue, yellow, green. If you have any of the scarves, that can tie your bag in with your outfit as well.