Think about how much detail is in a wallet, how small they are and how much work has to go into them. All those sleeves, pockets, id windows, snap closures, zippers, what have you, are all being sewn into a small item. People don't think about this: you have as much detail (or more) as a bag, going into a smaller amount of space. This ups the difficulty and the time spent, therefore, wallets are more expensive.
Makes sense to me. Besides, finding a wallet that suits your needs perfectly can sometimes be harder than finding the right bag. I'm willing to pay for a wallet that is the perfect fit, but I won't say no to a good deal either