You aren't wrong, but please remember that not ALL fake labels peel off. However, if you CAN peel it off-- it's definitely a fake.
Also, some ks bags have metal labels... but it has to be a specific sort. There is a VERY common metal label that is only used on fakes. It has a different look.
Thanks! Good to know. Of course this makes it even harder to authenticate things!