A baby horse, infant/newborn, is regarded as a foal.
In horse years, 5-16 (2 years, I think) is when you can regard it as a 'pony' or a 'foal.' If a small child can indeed ride it with a bright pink and rainbow saddle, then it's a pony.
And then a fully grown horse is a horse.
"you don't say."