from the context, you can tell he's referring to recreational "illegal" drug users.
I mean yeah, if you have good inference skills you could probably just skip the generalization and think, "Oh yeah, he is most definitely talking about the recreational drugs group". But from how he came off with it, it was general and he needs to watch how he explains himself. Because once you generalize, you've made the most foul move you can ever make in an argument.