SFMC Duplicate Contacts Query

Question

I am trying write a query that will result in duplicate SFMC contacts in a SFMC list.

More specifically, I have 3 brands and each has a shared publication list then there is the All subscriber List. Let's say I have a subscriber with the email address [email protected] who is in the lists as follows.(note that each subscriberKey is unique)

EmailAddress ListName SubscriberKey [email protected] BrandA 0031500001dpVxEAAU [email protected] BrandA 0031500001dpVyjAAE [email protected] BrandA 0031500001dpVy7AAE [email protected] BrandB 0031500001dpVxUAAU [email protected] BrandC 0031500001dpVxNAAU [email protected] All Subscribers 0031500001dpVxNAAU

I need a query that can identify that this emailAddress exists on List BrandA 3 times and will give me all but 1 of those subscriber keys.

(The goal is that I will delete the results of the query from the contacts keeping only one contact per brand list or all subscriber list if they are not in any brand list)

Any suggestions?

This is an example of what does NOT give me what I need:

SELECT 
  EmailAddress
  , SubscriberKey
  , ListName
FROM _ListSubscribers 
WHERE EmailAddress IN (
    SELECT EmailAddress 
    FROM _ListSubscribers 
    GROUP BY EmailAddress 
    HAVING COUNT(*) > 1
) 

Answer 1

Row_Number() is one of the more useful functions for everything relating to deduping.

This will number your duplicates and from the result of the numbering, select all those that do not have the number 1, i.e. are duplicates.

Code is untested, but extrapolated from a working example. I hope it gives you a starting point.

SELECT a.EmailAddress,a.SubscriberKey
FROM (
  SELECT EmailAddress,SubscriberKey, ROW_NUMBER() OVER(Partition by EmailAddress, ListName ORDER BY EmailAddress DESC) AS Row_Number   
  FROM _ListSubscribers) a
WHERE Row_Number != 1