"You'd have to work a lot harder than that to make me hate you," he said, in an oddly depreciating tone. "You're not even all the way over the line to irritation, yet."

That's joking and playing a bit, though, and once he's caught up he puts his hands in his pockets and heads for that diner.