Not at all... I also got my first Rolex in my 20s, and prior to that, I've been wearing designer watches (Omega, Tag, etc...) since I went to university. The style you describe is completely not flashy, and it's great that it has sentimental value.
IME, people hardly ever notice when I wear my Rolex since it's a model similar to yours, stainless Datejust, so it's sporty, but not blingy.