How Data Fetching Works
The TapClicks system has been engineered to work in BATCH MODE, meaning that when you click Fetch My Data, the system updates ALL connected data sources and campaigns at once.
Best Practices
Fetching data may take longer than expected, depending on several factors. This is due to the way the system updates all your data. To ensure a timely update while you are building content in your dashboard, we recommend considering the following actions when you fetch your data:
-
Connect all data sources that you know you need.
-
Map all the clients you want.
-
Then click Fetch My Data and allow the system to process the request in the background.
-
If you are an enterprise user with hundreds or thousands of clients, then it’s best to use the two bulk features to speed up the process!
Why Data Fetching Takes So Long
When you click Fetch My Data, all connected data sources and campaigns are updated with the data. The following factors (and more) contribute to the time it takes to fetch data:
-
The amount of data: The more campaigns you have connected, the longer it takes to get the data.
-
The depth of data: The more levels of data you have connected (e.g., Ad Creatives, Ad Placements, Keywords etc.) the longer it takes to get the data.
-
The type of Data Source: The quality of a vendor's API varies a lot from Data Source to Data Source.
-
The number of times you click Fetch My Data: If there is a heavy load of fetch clicks, the updates must go in a queue, causing some additional delay.
-
New connections: Whenever you hook up new campaigns, the delay will be longer due to the history factor. Generally speaking, the initial fetch for most data sources goes back one year, but for existing campaigns, only three days. So, hooking up a lot of new campaigns will mean significantly longer update times.
-
API request limitations: Some data sources have API request limitations, requiring us to spread out the fetch requests over time meet those limitations.
How to Make Data Fetching Go Faster
Below are some options available to you.
-
Determine if an update is necessary: The TapClicks platform automatically updates every morning at about 6 AM EST or a time based on your instance's fetch strategy. Most third-party platforms update only once in 24 hours, so you only need to update if you are loading new campaigns.
- Streamline your data requests: If you are pulling data you or your clients don’t need, consider pausing the data flow connection to speed up your fetches. Learn how to pause data flow.
-
Optimize your fetch process: If you manage large data volumes and need assistance improving fetch times, our Customer Solutions team can evaluate optimization strategies. Simply reach out to your Customer Success Manager or customercare@tapclicks.com to learn more.
-
Run individual fetches for SmartConnectors: As of November 2023, we have added the ability for users to run individual fetches for SmartConnectors, rather than running a fetch for all data sources. This is important, especially when creating or editing a SmartConnector because it dramatically improves fetch performance when you only have to fetch a single connector (see image below for how to do it).
- Run individual fetches for Instant-on connectors: Users also have the ability to run fetches for individual Instant-On connectors, rather than triggering all of them simultaneously. This allows for quicker assignments and targeted updates, helping users avoid unnecessary load on data limits and reducing the chance of hitting API quotas.
What's next?
To view the status of your data fetches as a whole, click Data Load Status in the Data Management menu on the left side of your dashboard. For more information about Data Load Status, please see this article.
If you need more assistance, our Customer Care team is standing by to help at customercare@tapclicks.com.