Duplicate Check not working on Custom Relationship Field

Hi all.

I've added a new relationship on leads linked to Contacts (contacts_leads_1) which is One to Many relationship. On the lead this creates the standard relate field linked to the relationship.

When going into the "Search and Select" box and hitting "Create". If I create a new record, the duplicate check doesn't fire like it would if creating the record directly, it seemingly just skips it.

Has anyone had this before and knows of anyway to fix it?