Hey, Desiree
It's alright that you made a mistake; everyone can slip up while they're talking and if he's ok with it then just move past without worrying. If you absolutely wanna explain to him, tell him that his name came to mind NOT because of any positive feelings, but just that he was bothering you lately and while you wish you could wipe your memory that only happens with time. He should understand this. You have plenty of time to show him that you care about him waaaaay more
You didn't ruin anything, dear. Just go with it and have fun, it's the beginning of something good. You're both human and will make mistakes along the way.