“I would just look the other way when he had lines to do,” the actor said.