My sister had this problem. If you have wax build-up, the doctor's will give you a prescription for a liquid that helps get the crap out. In a few days, the softened wax'll come right out.
Does your ear hurt any? I've had infections in my life, and it hurts if it's an infection.