Well, I have a B.S. degree in Textile and Apparel from the Univ. of MD. I went into fashion merchandising and was an assistant buyer for Hecht's (it's now part of the Macy's chain, but it wasn't in the 1980's when I worked there.)
The buyer that I worked for did all the ordering for the pre-teen department. All her purchases were approved by her superivsor, though. She decided which merchandise to put on sale, which things to put in ads, which stores got which labels (there 20+ stores), which things to return to the vendor b/c they didn't sell, visited branch stores, etc.
She traveled to NYC a couple of times a month to go to showrooms. The hours were long (even for me as an assistant) and I found the atmosphere unpleasant. Deadlines were always looming and problems with shipments were never ending. Maybe it was the personality of Hecht's but nothing was ever dealt with calmly. Slamming doors, yelling and banging down telephones was on-going.
Some math was involved, but no more than basic arithmetic. Anymore, I suspect the computer would do it all for you. I'm sure it's a much more sophisticated system than when I worked in a buying office.
For a person who enjoys traveling and devoting tons of time to a job, it's ideal. I think, though, for a person who has hobbies, a family, and doesn't like to travel it's not a good match.