I think I've got it down, but can someone check this for me?
You're father grew up in a stuffty environment which didn't offer any sense of comfort to him.
Occasionally whether he meant it or not, he would call your grandfather a "Dumb kid" or he would say "I hate you".
Your grandfather would then make him walk with him to a shed to get a well deserved ass whooping.
(C'mon, his father called his grandfather a dumb kid. :c)
This mentally scarred you? The living conditions are sad and the "If you keep crying I'm gonna keep hitting you" is sad too, but I don't know about mentally scarring.
speaking of getting scarred, I've seen both my grandmother's boobs and my grandfather's snake before
I've had the unfortunate pleasure of seeing most of my relatives naked at-least once, yummy!