I forwarded ports already and I'm having the same problem, except much worse. About 1 in 20 people that try to join get into my server, and every once in a while, I get a lucky day and my server works for 17 in 20. I never have lucky days anymore.
Sigh.
P.S. What is UPNP?