I'm of the opinion that height doesn't enter into this decision nearly as much as what type of look you prefer and what your personal preference is as far as bag size.
As an example, I'm about your size and love 35 Birkins just as much as I do 30's, but I feel perfectly comfortable carrying them and like the look. It's much more about whether "you're" going to feel like it's too large for you and only you can answer that.
Obviously though, you asked about the 32 HAC and to me, this is a perfect compromise between the 30 and the 35 when someone feels the 35 is a little large and is debating between the 2 sizes. As far as a bright color in the 32 is concerned: yes, it might look a little larger than a dark color would in the same size, but that's only an issue as far as your feelings on color are concerned.
The decision really comes down to your own preferences, but in and of itself, the bag isn't too big for you.