The only problem with your logic is that you are not directly affected by your great grandfather today. Blacks still have never gotten official compensation for anything their ancestors went through, and are still in the poorest areas of the country enduring racism and hardship today.
Regarding reparations for slavery:
I actually am affected by when and how my greatgrandfather died and what he did, but that's irrelevant. Dead people aren't paying reparations, we would be. I'm not going to let my tax money go to a bunch of random people easily because they had some great great relatives who got royally forgeted over. If this stuff was like 20 years ago or something then fine, but it wasn't, it's been like 3 generations.