It never happened to me, but luckily it seems like if you ever log in from another state (or some potential hacker tries to), google does a great job at stopping random dudes from getting into your accounts and stuff. I'm kinda surprised at this though, since I used to be much less security conscious not too long ago.
Anyways, a easy way to make sure you'll be safe is to link a phone to your email for two factor authentication, so if they somehow get past the first layer, they'd have to have your phone to get in or somehow guess a random 6 digit number (which won't happen).