I personally agree, praying isn't actively harmful. Go for it.
In most situations, yes.
But the problem occurs when people use prayer alone as a substitute for an actual solution.
Like the select few number of parents who insist "oh no we're not going to bring our sick kid to the doctor, we're just going to pray for him instead" and then surprise, surprise, their kid dies from a disease that would have very quickly and easily been treated, had they chosen to provide him with scientifically proven medical treatment.
And note I'm not talking about people in poor undeveloped areas who can't afford or don't have access to healthcare. I'm talking about people fully capable, but deciding not too.
That stuff
is actively harmful. And while the amount of parents who actually do that is small, it's unfortunately not zero.
But it most cases, yeah, there's nothing wrong with it.
Now, regarding the question of "does prayer affect your morality?" my answer is that it depends on the content of your message, not how you deliver it. For example, when I interviewed for my first job, a lot of family texted me "good luck!" or something similar. My grandma instead said "I'll pray for you". Her message of good will was still the same, she just chose a different "method of delivery"
Or, if you desire harm to fall to someone, that is immoral. How you deliver that desire (telling them to their face vs privately praying for harm to come to them) does not change your morality