This happens with servers that are using the disable upnp method. Only some people can join servers like that.
IF the case is that instead of forwarding ports you're using another upnp toggling method, with which i am not that familiar with, like half of the players, including me can't join the server.w
Are your ports forwarded?