I would feel comfortable purchasing only at a few places in the caribbean - and none of them sell loose stones (it's all branded jewelery). If you were looking at a specific brand and not studs, I would send you to:
Jewels by AH Rise (St Thomas) - they have a concierge services for shopping from home and will meet you at the store with your items set aside, or mail to you..I did this with my watch.
The Art of Time (watches) or Artistic Jewelers in St Martin (Dutch side, but also artistic on french side)..penny preville, mikimoto, luxury watches..owned by the same people who NO LONGER own artistic in St Thomas.
Expect the discounts on branded items to be approx 20 percent and no tax.
I know other stores have "cruise line guarantees" but I've never shopped there. Jewels is a huge store and we have known the owners of artistic/art of time for 15+ years. I would trust one of those places to recommend a shop to you for loose stones - but try to find someplace with a cruise line guarantee (usually if something appraises for less or gets damaged, they will help you deal with it). Definitely be careful - I am just so skeptical of the random jewelery stores out there. Do you have any friends who have someone in the NY diamond district?
And no, I am not affiliated with any of the listed places..I just really like to shop!!