You were drunk. He took advantage of you while you were weak. You said no. He forced you to have sex with him anyway. That is RAPE.
I can understand your not wanting to report him to the police, despite what he has done. However, please think about this: It may not have been the first time, and it may not be the last time, that he will do this to another woman. I think that frightening thought is reason enough. Stand up for your rights, and for the rights of potential victims. Send the message across to him that this ISN'T okay, and that he CAN'T get away with it. All too often, rapists do.