If you dont get a lot of replies, ask a mod to move it to fitness
I think if you want to do it, you should absolutely go for it. I have friends who have started WAY WAY older than you. It is never to late to do something you love.
As for finding a good facility, the best thing to do is visit some, or if there is a horse show in your area, go as a spectator and walk around the barns and trailers and see if you can pick up a good vibe from one of the trainers. You wont want to bother them too much when they are getting their riders ready, but you can ask for their name or get it off their trailer, and perhaps get a chance to talk to them a the end of the day or at lunch.
A show program will also have some advertising in it. You can also google for any riding clubs in your area and get a feel for the members who also give lessons.
good luck!