I have a limit of like 390 and I'm only level 28. So I've never saw a reason to delete anyone.
I use tags to keep the people I actually talk to at the top, like...
1 Family
2 Friends
3 Buds
4 Online Buds
5 Online Associates
And then followed by the titles of the games I met the people in.