Yes, you only avoid duty if you are leaving the place, whether a country or economic union, where the tax applies - pretty normal. I hold US and one other non-EU passport and I can't detaxe until my last stop in the EU for same reason.it's depends on whether the shop is located before or after the immigration, because within the continental Europe there's usually no passport control, once after the passport control, the prices at the duty free shop usually is detaxed, the one before passport control will be normal city price, then they can offer you a detax form if you live outside EU, it's the same ritual as if you buy from the city.
More on topic, also worth noting that some US states (California and I think NY as well, probably others) will levy use tax on items bought in duty-free shops when those items are brought back to the States. This tax is the same as state sales tax.