I think you've said it all in that you'd rather a bag in better condition. Because of that I think you should return it and wait for another option, because if it bothers you now, it will probably always bug you. In my opinion, it will be very hard to find a ML in black caviar in better condition with the box and authenticity card for only a few hundred more. If you are very patient, I think you can get one for ~$4000 ballpark, but you will have to be patient and search aggressively.
If it were me, lacking the box would not bother me (I have too many as it is and I don't store my bags in their boxes, so they just take up space). I prefer the authenticity card, but it is definitely not a deal breaker, it's mostly helpful if you think you will want to sell it later. The little marks by/on the hardware would not bother me either, it goes with buying a bag that is more than 5 years old and at a big discount, there will be little marks, little dents, little scratches etc. The scuffs inside the flap probably would bother me though, if they are quite obvious from an armslength away, it is hard to tell since the photos are zoomed in so far.
In any case, since it bothers you, that is all that matters, and I certainly agree with you that ~$1,000 more to swap out a bag you aren't crazy about to one that you love is well worth it!
I hope that helps.