The 38mm is definitely okay for a man or a woman! I am a female and I have 2 38mm j12s. I love them so much! I like that its an automatic, can't live without the sweeping hand. If you want something more many you can get the 41mm chrono, its a lot bulkier but very nice.
Here is a picture of my 38mm white with diamonds next to my bf's LE sub. So you can see size wise it would be okay on a man.
I would say DEFINITELY get at least the diamond bezel. both of mine have the diamond bezel, but my black one also has diamonds in the center circle. Chanel uses great diamonds, so its very eye catching. Diamonds are worth it for this watch every time!
The white is a lot more versatile than a lot of people think. I wear mine year around too, and would even if I didn't live in Hawaii, land of one season!
If you want something a little different, a pink sapphire bezel with diamond markers j12 just came out. I am waiting for the 38mm version, the boutique only has the smaller one right now. I'm getting either that or the diamond bracelet for either of my j12s.
Its a fun watch, I'm sure whatever you decide will be fab!