I'm sorry, best thing to do is try to talk to him. Make him understand how you're feeling. If he throws a fit or tells you off, then cut all ties with him.