Also, has anyone tried shorts from J Crew Factory? I'm kind of between sizes and didn't want to spend a bunch on them just yet, but I wonder about the quality.
If you are talking about the chino shorts, I have them. They are the lightweight chino, so they are not like the 3" Chino Shorts from retail, they are much thinner (stay away from the white!) and feel different. It can be nice in the summer because they are cool, but they do seem to wrinkle more. I would say that in terms of sizing, they are pretty similar, but the factory ones might run a smidgen smaller than retail (which is pretty true of all factory stuff). If you are in between sizes at the retail store, I'd go with your bigger size at the factory store.
The quality is not bad, I've been buying them for years and I normally get a new pair every year and make last year's pair my more bummy, casual pair. I put mine in the dryer so that probably doesn't help. I wouldn't think twice about getting them at about $25, but full price is a bit much for what they are.
FWIW, this year I've switched to the Retail 3" Chino shorts for my navy and white pairs, which are my workhorse items of the summer, just because I like them slightly better. I am thinking about getting a colored pair at the factory store, though, because they have better colors and are cheaper for something I won't wear a lot.