I would prefer not to speak with you. Please remove yourself from my curtilage. Good bye.
Seriously. Watching this video pissed me off. Notice, too, how precise the agents were about waiting for him to specifically invite them in while making it awkward if he did anything else, hoping he would.
No. You don't want to walk out and chat with them. You want to collect their business cards and call your lawyer and tell him about it. "Am I free to go back inside?" "Well, because I've got nothing to say." "Yes. Completely sure." "No, like I said I've just got nothing to say. Am I free to go?" "Okay, have a good one."