Duplicate detection rules: 8 checks across 4 CRMs

Key takeaways for how to configure duplicate detection rules in crm tools

• HubSpot requires Data quality tools access, Edit permissions for All Contacts or All companies, or Super Admin permissions to use the duplicates manager tool, according to HubSpot.
• Microsoft requires system administrator or system customizer roles to create, enable, and publish duplicate detection rules, according to Microsoft.
• Microsoft limits published duplicate detection rules to a maximum of five for one table type at one time, according to Microsoft.
• Pipedrive checks for matching organization name, person phone, and person email fields during imports to identify duplicates, according to Pipedrive.
• Microsoft excludes duplicate detection when users merge rows, save an activity as completed, or change the status of a row, according to Microsoft.

Configure duplicate detection by defining match fields and setting permission scopes

HubSpot requires specific data quality permissions or Super Admin status to access its duplicates manager tool, as stated in HubSpot.

Before configuring match fields, verify that your team members hold the correct administrative roles. In HubSpot, users need Data quality tools access and Edit permissions set to All Contacts or All companies, or they must have Super Admin permissions to use the duplicates manager tool, according to HubSpot. Without these specific scopes, the tool remains inaccessible regardless of other user settings.

Microsoft Dynamics 365 enforces a different hierarchy for rule creation. You have to be a system administrator or a system customizer to create, enable, and publish duplicate detection rules for your organization, according to Microsoft. These roles control the lifecycle of the rules from initial setup through final publication.

Audit HubSpot users for Data quality tools access and Edit permissions set to All Contacts or All companies, or Super Admin permissions, and audit Microsoft users for the system administrator or system customizer role. Check each user’s profile against the requirements above before proceeding to field configuration. Without the HubSpot permissions, the duplicates manager stays inaccessible, and without the Microsoft roles a user cannot create, enable, and publish rules. Confirm that the people configuring HubSpot and Microsoft hold the roles named above.

Partial reference of duplicate detection capabilities by publisher

The following table summarizes the eight specific rule categories for duplicate detection across HubSpot, Microsoft Dynamics 365, Pipedrive, and Close. Every cell below is a capability or limit documented by that publisher.

HubSpot also notes that records won't appear in the undo list if you make property value or other changes to the records during the 14-day period, according to HubSpot. Microsoft Dynamics 365 supports having more than one duplicate detection rule for each record type, according to Microsoft. Pipedrive merges records if it finds that any of the specified fields matches another person within the database or the import file, according to Pipedrive. HubSpot documentation covers how to review duplicate records, take action on duplicate pairs, manage custom rules, and export merge history, according to HubSpot. Microsoft Dynamics 365 also states that duplicates aren't detected when you merge rows, save an activity as completed, or change the status of a row, according to Microsoft. Pipedrive maps unique phone numbers onto existing records during the merge process, according to Pipedrive.

Define the match criteria and field logic for your specific data types

Selecting the right fields is the first step in building effective duplicate detection. In Microsoft Dynamics 365, you can have more than one duplicate detection rule for each record type according to Microsoft. This flexibility allows you to create specific logic for different data structures. However, you can publish a maximum of five duplicate detection rules for one table type at one time according to Microsoft. Keep this limit in mind when planning your rule architecture.

When choosing which fields to monitor, consider common identifiers that are unique to your data. For example, you detect duplicates by matching the record fields, such as email address, first name, and last name according to Microsoft. These fields provide a solid baseline for identifying potential matches across your database.

Pipedrive uses a specific logic for imports to prevent data corruption. Pipedrive recognizes “Jon Snow” in row 3 and checks for matching organization - name, person - phone and person - email according to Pipedrive. This example illustrates how the system scans specific columns to find existing records. If Pipedrive finds that ANY of those three fields matches another person within the Pipedrive database or the import file, it will merge them together and add any new data included in the import according to Pipedrive. This merge behavior ensures that new information is captured without creating redundant entries.

To apply this logic to your own environment, map your critical identifiers to these platforms. Identify which fields are most likely to be duplicated in your specific workflow. Read Stage duration rules across 5 CRM platforms when you also set pipeline timing. Aligning your field logic with your broader data management strategy helps maintain consistency. Review your current field mappings and ensure they align with the capabilities described by each publisher.

Understand the limitations and exclusions of automated detection

Automated duplicate detection in Microsoft Dynamics 365 does not trigger for every action within the system. According to Microsoft, duplicates aren't detected when you merge rows, save an activity as completed, or change the status of a row. This exclusion list applies specifically to the merge duplicate records functionality documented in the Power Apps user guide. IT managers must account for these specific actions when designing their data integrity workflows, as the system will not flag potential conflicts during these operations.

A separate administrative guide confirms similar exclusions for the broader platform. According to Microsoft, duplicates can’t be detected when a user merges two records, converts a lead, or saves an activity as completed. The "convert a lead" action is a distinct exclusion mentioned in this Power Platform admin documentation that is not listed in the previous user guide. Sales ops leads should note that converting a lead bypasses the detection logic entirely.

The scope of the detection dialog itself is limited to specific user interactions. According to Microsoft, the duplicate detection dialog is only displayed for the records created or updated in the user interface (UI). This means that records created or modified through APIs, integrations, or background processes will not trigger the standard duplicate detection prompt. Teams relying on automated data entry from external systems must implement their own deduplication logic, as the built-in UI dialog will not appear for these non-UI interactions.

Manage the merge process and review history for audit trails

Once potential duplicates are identified, the merge behavior varies significantly by platform. In Pipedrive, when the system detects a conflict during an import, it adds the new Phone number to row 1’s record, as well as updates row 3's organization - name with “The Wall” according to Pipedrive. The tool also maps row 2’s unique phone number onto the same phone number created in row 1 according to Pipedrive. This specific mapping logic ensures that unique data points are consolidated rather than discarded during the import process.

For bulk operations, the available options depend on the specific CRM tool. HubSpot allows users to reject, merge, or merge in bulk potential duplicates identified by the HubSpot default rule according to HubSpot. Close offers a different workflow for bulk merging; if you want to merge all duplicates in bulk, you send the generated CSV file to their support team along with your preferred merge rule and they will handle it for you according to Close. This manual handoff in Close contrasts with HubSpot options to reject, merge, or merge in bulk.

Auditing the results of these merges is critical for data integrity. HubSpot provides specific properties to track these actions. You can use the Merged [record] IDs properties to view which records were previously involved in a merge according to HubSpot. Additionally, the platform allows you to review duplicate records, take action on duplicate pairs, manage custom rules, and export merge history according to HubSpot. This export capability supports long-term audit trails for compliance and data governance. See Field history limits: 8 checks across 3 CRMs for field-change history next to a merge export.

Be aware of the limitations on undoing these actions. Records won't appear in the undo list if you make property value or other changes to the records during the 14-day period according to HubSpot. To catch more subtle duplicates, HubSpot provides more similarity options, such as 30% and up, so operations managers can review potential duplicate contacts that aren't immediately identifiable according to HubSpot.

Illustrative example of checking one import

Picture a team cleaning one people table before a file load. The table holds 10 rows. The reviewer matches email address, first name, and last name. Row 1 and row 3 share an email address, so they count as one duplicate pair. The phone number from row 2 is kept on the surviving row, and the organization name from row 3 is kept as well. The reviewer writes 4 match rules and publishes all 4, staying inside a maximum of 5 published rules for one table type. During the next 14 days, nobody edits a property on the merged rows, so the pair can still show on an undo list in this illustration. A fifth row arrives only through a background load, so no detection dialog appears, and the reviewer checks that row by hand today.

Review your current merge history exports to ensure your audit trail is complete.

FAQ: How to configure duplicate detection rules in crm tools

Who can use the HubSpot duplicates manager?

Data quality tools access and Edit permissions set to All Contacts or All companies, or Super Admin permissions, are required to use the duplicates manager tool, according to HubSpot.

What happens to duplicate detection when a record is created by a workflow?

The duplicate detection dialog is only displayed for the records created or updated in the user interface, according to Microsoft.

How does Pipedrive handle new data when it merges duplicates during an import?

Pipedrive will merge records together and add any new data included in the import if it finds that any of the specified fields matches another person within the Pipedrive database or the import file according to Pipedrive.

Can I merge duplicates in bulk using the Close support team?

Close confirms that if you want to merge all duplicates in bulk, you should send the generated CSV file to their support team along with your preferred merge rule, and they will handle it for you according to Close.

Where can I view the history of merged records in HubSpot?

HubSpot instructs you to use the Merged [record] IDs properties to view which records were previously involved in a merge according to HubSpot.