This is one of those naïve "defenses" only a foolish feefighter would allege.  Yet it's among the most popular, because candidates fall all over their paychecks to help.

There are actually two levels of defense here:

1.  The candidate was your agent and therefore could waive (relinquish) your right to the fee.

This is a sophisticated argument, since you didn't actually deputize the agent to act for you.  However, ostensible (apparent) or implied authority can sometimes be credibly argued if the candidate represents (states) that you authorized him.

2.  The candidate was a witness to your waiver of the fee.

This is the safest route.  The candidate doesn't have to lie (much), and can even say he  thought you waived it.  Usually the words he "heard" were, "Don't worry about paying the fee."  He "mistakenly" interpreted this as meaning no fee would be charged to the client.

Supposedly that's what he said during the interview.  The client "reasonably" believed him, so hired.  No agreement to pay, no communication with you (since it believed you weren't involved), therefore no contractual liability.

Don't budge.  There's no legal way a client can prove an actual agency relationship existed with the candidate.  The candidate's thoughts have nothing to do with the obligation of the client under the legal objective theory of contracts.  It simply says, "Look at  the parties' agreement to determine their intent.    

"Let the facts speak for themselves."  Don't back down.

That client will pay!