The point of the felt is to prevent the sangles from leaving indentations on the flap, and to prevent the flap from leaving indentations on the body of the bag. Also, prolonged contact between two pieces of leather may cause the surfaces to stick together, and then peel when separated.
These are the important considerations here.
I would say that your concern over stretching the sangles is unnecessary.
Do you limit what you put in your bag for the same reason?
If the sangles stretch a little, fastening and unfastening the bag will be easier and the I doubt, in use, the change in appearance would be negligible.
Do not worry - keep using bag and enjoying it - don't store it for long enough for any problem to manifest itself.