Well most things can be found in NYC so hard to know what to recommend here. We have some local clothing brands that are nice but they are in the upper market and not what I would call souvenirs. They are quite pricey and most you'll be able to find in NYC anyway.
First, I would say Candy. Really, we do have the very best Candy. Specially salty liquorice. Its acquired taste but you might wanna try it

I would go to a large grocery store and look around. Do you know where you'll be staying? Also, what kind of Christmas market are you visiting? We're not really famous for them so to speak
Anyway .. I recommend you visit
Designtorget. The web site only choose 1/100 of what they have. You'll see a few of these stores around the city but the
big one is right in the middle at Sergels Torg and that is the one you should visit. A great place to pick up small, odd things from local designers.
Also, since you'll be here during Christmas time I would pick up a bottle of Glögg, Our version of Glühwine. You can only buy this a
Systembolaget. There are lots of different kinds and I don't really drink myself so can't recommend you which one to get but ask the staff. You may be able to find it at the airport too.
Each year Blossa makes a Glögg that is limited. This years bottle looks very swedish. (I'm actually thinking of buying more bottles to decorate with at home.)
Forgot to ask, are you here during a weekend? In that case you might wanna visit Hornstull's market. Its a arts & craft market we're people put up a table and sell their own stuff.
Hope this helps a little.