I think you need to see the interior of the home on the lot you love in order to be sure.
Without that key piece, it is hard to say.
And older home can be a wonderful, or it can be an endless job of throwing good money after bad. i have a friend who just bought a 1913 house and nobody saw in it what he saw, and now it is amazing...a wall knocked out here or there can make a world of difference.
If you are not in love with your house any more, you will always be looking around for something different.
I suspect it is more than the yard because you mentioned that there was a lot you would have done differently, and thats OK. Life is for learning and if you will be happier in the other property, after DUE DILIGENCE and lots of thinkng and planning, go for it.
Please make sure you check thoroughly the costly things that cannot be put off if they are failing: electric, plumbing, septic, well, etc.
One of my relatives bought a house from 1971 and it is as solid as anything built today, and looks like a smashing retro open plan home that needs just a coat of paint, new counters, and carpet. The rest is brilliant. But other homes of that era need the wrecking ball, KWIM. It all depends on what was put into them at the start.
At the same point, gilliana is right, the yard is not as important at the feeling of home-y-ness and comfortability that you have when you are there.
You will know more after seeing the interior
good luck!!