Understanding Unique Fields

What are Unique Fields?

When setting up a Smart Connector, the system needs to be able to interpret your data correctly. Unique Fields are used by the Smart Connector to distinguish one row from another in your dataset.  You can choose one or more fields in a row to determine uniqueness. 

Whatever field you choose, the system will concatenate the Date field to it to create a uniqueness key. If you choose multiple fields, the system will create a concatenated string of those fields and the Date field. See examples below.

Sample table data

If you choose the Campaign field in this row, the uniqueness key will be 7/20/22~July_2022

If you choose the Campaign and Ad fields, the uniqueness key will be 7/20/22~July_2022~It’s_HOT!!!

How do I determine which fields to use as my Unique Fields?

NOTE: If you cannot determine the Unique Fields on your own, the provider of the data may be able to help.

The best way to determine which field(s) to use as your Unique Field(s), is to look at the data and ask “What makes these rows uniquely distinguishable from each other?”. Your choice should include just enough field(s) to make the row distinguishable, without adding any unnecessary fields.

NOTE: When possible, try to choose an ID field for your unique field, as they are less prone to changing over time. Fields such as “Campaign Name” could change over time causing your data to split.

Here is an example where only one unique field is needed:

In this case, only Campaign ID is required. That's because the combination of the Date field and Campaign ID field is enough to distinguish the rows. There are two rows with July 22 campaigns, but they occur on different dates, so the rows are uniquely distinguishable.

Here is an example where multiple unique fields are needed:

In this case, both Campaign ID and Ad ID are required to be a Unique Field. That's because multiple campaigns share the same Campaign ID for the same date. Therefore you’ll need to add both ID fields for the system to create a unique string for each row of data.

How does a Smart Connector use unique identifiers to process and write data to the platform? 

In TapClicks, the Smart Connector uses “upsert” operations. If a record exists, it’s updated; if the record doesn't exist, it's inserted. A unique identifier like a Campaign ID helps the system prevent duplicates and keeps the data current and accurate.

How Smart Connector Uses Unique Identifiers for Upsert:

• Data Integration: Unique identifiers like Campaign IDs or Ad Group IDs are used when connecting a data source.

• Data Fetching: TapClicks pulls data using these identifiers to retrieve the correct records.

• Matching Records: When TapClicks receives data, it checks if a record with the same identifier already exists in its database.

• Update: If a match is found, TapClicks updates the record.

• Insert: If no match exists, TapClicks inserts the new record.

• Data Integrity: This ensures records are current and avoids data duplication.

• Handling Updates: Incremental updates allow TapClicks to refresh only changed or new records, making data management efficient.

What happens if I configure my Unique Fields incorrectly?

Not enough Unique Fields

If you do not include enough Unique Fields to establish the uniqueness of every row, some of your data maybe discarded. In the example below, if only Campaign ID is selected as the Unique Field, the system will read both rows as having a uniqueness key of “7/20/22~J22'' since they both share the same date and Campaign ID. This will cause the system to only keep one of them because it will decide the other is a duplicate row. 

Too many Unique Fields

Having too many Unique Fields isn’t really an issue, unless those fields can change over time. In the example below, the Ad field has been added as part of the Unique Fields. The problem is, that can change over time. This causes the system to interpret that row as different from the other rows from the same campaign.

Today:

Uniqueness key:  7/20/22~J22~IH134~It’s_HOT!!!

Tomorrow:

Uniqueness key: 7/20/22~J22~IH134~It’s_Really_Realy_HOT!!!

How do I fix my Unique Fields if something's wrong?

The easiest way to tell if you’ve configured your Unique Fields wrong is that you’ll see discrepancies in your data, as well as receive a notification in Data Load Status regarding duplicate rows. If you experience either of these, do the following:

1. Delete the data for the Smart Connector (you may keep your mappings)
2. Check the raw data to determine the best Unique Fields
3. Reconfigure the Unique Fields(s)
4. Assign your clients
5. Click the Fetch My Data button
6. Check your data after it been loaded