There are a couple of possibilities here. Any user that goes into a refusal state quickly either has their Refusal Timeout set to low on the ACD user section. This will turn the user to Refusal state since they will not have a chance to answer. The other item to confirm is that they have not entered an invalid phone number at login. This will create a refusal state as well.