It's fine as long as you don't overdo it, like only drink on special occasions and not in stupid amounts
Yeah it's really only when you get to repetitive binge drinking and full alcoholism dependence that you really damage your liver.
Either way it's still a respectable choice to not drink, nobody's going to hate you for it.