Here are two threads dealing with this question in different ways that may give you some more insight in what to wear with a navy bag (it does look like whatever you want to wear with it; the notion that you can't pair certain colors is no longer a 'thing'):
Cognac is my favorite with navy or dark denim. But if I want to stick to cool colors, there are lots of options - anything in a medium or light tone works like olive, taupe, gray, ivory, beige, chambray blue. Pops of color like yellow or red would work. Metallics work. Black is ok if the outfit is balanced.
I guess anything works.
Even my turquoise tote goes with navy. Then it becomes more about what style you are going for: playful or conservative or casual or nautical.....