“I did tell him, I even told him if it was just us, me and him, that maybe I would change my mind…”

(Yeah. ‘Maybe’. That’s going to clue him in to anything, or make him confident you even would. And no, you never did say why at the time, just that you didn’t want to in front of Heiji. And to be perfectly frank, without the knowledge of what you’re upset with exactly, it just sounds like the childish ‘I don’t like him, so no’. Not something that would keep Kai from trying. He’s smart at times, oblivious at others, so if you’re getting upset, tell him outright, else he may not get it until later, and then he gets guilty at making you upset. You aren’t the one who has to deal with a sullen thief, most of the time.)