If a server is private, and the build is private, and meant to be private, you shouldn't have access.
That's like someone going into your personal information or something and you catch them and are all "HEY GET OUT OF THERE, THAT'S MY STUFF"
And they just say "So what?"