This document contains frequently asked questions that arise about Data Exporter. You can quickly navigate to a particular FAQ by using the table of contents below.
Client Issues/ Troubleshooting
I need historical data, how do I get that?
Is there a limit on the number of days of data I can request through Data Exporter?
General Questions
What data or fields can be exported through Data Exporter?
Why does TapClicks need DELETE permissions for warehouses and databases?
Is Leads data available via Data Exporter?
Are metrics modified by calculations available via Data Export?
Can I filter my Data Export to campaigns instead of clients, client groups or business units?
What happens when "Merge data to the same table on each export" is enabled?
Can I choose to deliver an alternative file format when scheduling exports, maybe a .txt file?
Why can’t I find some fields (e.g., Smart Campaigns)?
What do the file names look like when they are delivered?
Who can access the Data Export?
How many days are included in the exports?
Can I export data that is aggregated to a week or month?
Does scheduling work for Data Exporter?
Can I use Smart Connectors in Data Exporter?
Can I use Channels in Data Exporter?
When setting up a Data Exporter, is there a max number of columns for each export?
What timezone are the log timestamps represented in?
What are flexible columns in the Data Exporter?
Why are flexible columns useful?
Data Destination Questions
What is the Optional S3 credentials section for?
Why would someone use the Optional S3 credentials section?
Approximately how much faster is it to use the Optional S3 credentials section?
I'm using the Optional S3 credentials section, why don't I see any data in the bucket?
What if I need data to be in the S3 bucket and also in my target location?
Can TapClicks host an S3 bucket for me so I can use the Optional S3 credentials section?
Can I send data to Microsoft Azure SQL?
Pricing/Contract Questions
Will failed rows count against my Data Exporter row limit?
Client Issues/Troubleshooting
TapClicks changed one of my data export fields or I changed my calculation formula, but I still see the same old data in my data warehouse. What’s happening?
If new code was pushed to production or if you changed your calculation formula, but you haven’t run another export, it’s possible that the data hasn’t been refreshed on the target system. A quick way to check this is to change the Data Exporter to “On Demand”, push the export again, confirm the data, then change the Data Exporter back to normal schedule.
If you still have problems with what you’re seeing in your target system, feel free to reach out to your Customer Success Manager or customercare@tapclicks.com.
After a successful data export, I can see my data in the TapClicks dashboard, but that data isn’t where the exporter sent it. Why is my data missing?
It’s possible that your Data Exporter ran before your data fetches completed. To troubleshoot this, look at the following checkpoints:
- Open your Data Exporter and check the logs to see the last time the export ran. Confirm that there was no failure.
- Then, navigate to the Data → Data Load Status page → Data Load Status tab and check to see what time your fetch for that Data Source completed.
If the export ran before the fetch completed, then it’s likely that the data for yesterday was not included in the export. To resolve this issue, schedule your export to run later in the day, approximately when the fetch for the Data Source is complete. This will not cover 100% of cases, but it will cover many of them.
I need historical data, how do I get that?
At present, you can do this by setting up a schedule for 365 days, running it once, then either turning it off or changing it to another delivery cadence.
Is there a limit on the number of days of data I can request through Data Exporter?
Yes there is a limit. You cannot request data for more than 2000 days in the past, but please note that increased days will lead to increased export time.
General Questions
How does data get delivered?
For data exports to databases like MySQL and PostgreSQL, and warehouses like Snowflake and BigQuery, you can either merge (i.e., overwrite) data to the same table or create a new table each time the export runs. The details are explained in the tooltip (see image below).
For Amazon S3, FTP, MSFT Azure and email, you’ll notice this option disappears. That’s because TapClicks delivers a new CSV file each time we export them.
What data or fields can be exported through Data Exporter?
Almost all data in TapClicks can be exported via the Data Exporter. This includes channel data, calculations, Smart Connector data and more. If it can be displayed in a grid widget, it can likely be exported via Data Exporter.
The Data Source fields that TapClicks exports can be seen on our connectors page. At this time we do not export on-demand connector data and we do not export Leads data.
Why does TapClicks need DELETE permissions for warehouses and databases?
In order to export data in an efficient and timely manner, TapClicks first loads data into a temporary table and then the system deletes it at the end of the data export process.
Is Leads data available via Data Exporter?
Not at this time.
If you need to export Leads data please either submit feedback to our product team or reach out to your Customer Success Manager or customercare@tapclicks.com.
Are metrics modified by calculations available via Data Exporter?
Yes.
Can I filter my Data Exporter to campaigns instead of clients, client groups or business units?
Not in the Data Exporter. However, you can use second-level assignments to connect your TapClicks clients to specific campaigns and then filter your Data Exporter by client.
When I have an FTP, SFTP or Amazon S3 Bucket location, do I need a backslash before the directory name?
No, the field assumes that if it’s filled in, a proceeding backslash needs to be used. See the example below:
What happens when "Merge data to the same table on each export" is enabled?
When you enable the “Merge data to the same table on each export” option, each new export will be merged into an existing table instead of creating a new, unique table for each export.
Here’s what happens when this option is enabled:
-
Data Merged into the Same Table: Instead of creating separate files for each export, TapClicks will merge new data into an existing table in your delivery destination.
-
Overwriting Existing Rows: If there are any rows with existing data for a specific date, those rows will be overwritten with the new export data. This ensures that the most up-to-date information for those dates is retained in the table.
- New, Unique Tables if Disabled: If this option is disabled, a new and unique table will be created for each export, without overwriting or merging with any previous exports.
This feature is useful if you want to maintain a single, consolidated dataset that is continuously updated with new information while ensuring any older, outdated data for specific dates is replaced.
Can I choose to deliver an alternative file format when scheduling exports, maybe a .txt file?
Data Exporter supports CSV only at this time and cannot deliver any other file types.
If you need to export data in other file formats, please either submit feedback to our product team or reach out to your Customer Success Manager or customercare@tapclicks.com.
Is the order of fields in the exported CSV file governed by the order in which you select fields in Data Exporter?
Yes.
Why can’t I find some fields (e.g., Smart Campaigns)?
Some fields you may be used to seeing when building widgets (e.g., Status, Country, TapClicks Smart Campaigns) cannot be easily broken down on a daily level and exported.
If this data is important to your needs, please either submit feedback to our product team or reach out to your Customer Success Manager or customercare@tapclicks.com.
What do the file names look like when they are delivered?
When TapClicks generates a new CSV file for Data Exporter to Amazon S3, FTP, etc., the system ensures a unique filename on each export by appending a timestamp to the filename. The system appends the current date and time to the filename in the format, [YEAR][MONTH][DAY]_[HOUR][MINUTE].
Can I change the file name?
Yes. You can use the “Prefix filename” field to add a string that makes sense for your needs (see image below).
Who can access the Data Exporter?
At this time, only Super Admins can access Data Exporter.
How many days are included in the exports?
TapClicks will export data for the number of previous days you choose, along with the data on the day the Data Exporter runs. For example, if you choose seven days of data on a Data Exporter that runs today, the system will include data from today, and the seven previous days (see image below).
Can I export data that is aggregated to a week or month?
No. Data in the resulting files will always be broken down by day. You can aggregate that data as you see fit after it’s been delivered.
Does the Data Exporter respect data profiles and RBAC for fields available, and who can set it up?
Currently, the Data Exporter feature does not check data profiles before shipping data because the feature is available to Super Admins.
Does scheduling work for Data Exporter?
Yes.
Can I use Smart Connectors data in Data Exporter?
Yes.
Can I use Channels data in Data Exporter?
Yes.
Does TapClicks support SQL?
SQL is a query language. MySQL and PostgreSQL are databases which you can query using SQL. Data Exporter can ship your data to a warehouse where you can use SQL to surface the data you need.
When I add a new assignment to a client included in an export, will that data automatically be included in the next Data Exporter schedule, or will I need to update the Data Exporter?
Yes. Newly assigned data will be included in data exports that contain that data source and client record.
When setting up a Data Exporter, is there a max number of columns for each export?
The larger the query (i.e., columns and lookback), the longer the query is likely to run and the more susceptible it is to time outs. That said, there’s no reason we shouldn’t be able to send all available fields if you keep your lookback window in the 7-30 day range.
Can I delete a Data Exporter?
Not currently, but if this functionality is important to you, please either submit feedback to our product team or reach out to your Customer Success Manager or customercare@tapclicks.com.
What timezone are the log timestamps represented in?
This is the same timezone as the Data Load Status page. Timezones respect the timezone setting of the logged-in user. You can see this timezone on screen, just above the logs (see image below).
You can also change your timezone by going to Administration → Users and pulling up your user record. Here you can edit your timezone.
What are flexible columns in the Data Exporter?
Flexible columns allow Super Admins to customize the fields shown on the Data Exporter page. Along with standard fields like Name and Status, additional fields such as Created Date, Creator, Last Modified Date, and Last Modified By can be displayed to provide more context.
Why are flexible columns useful?
Flexible columns give Super Admins the ability to see important details about data exports, such as who created or last modified them and when. This is especially helpful for managing instances with similar or cloned Data Exporters, allowing you to display only the information you need at a given time.
Data Destination Questions
I changed my Data Exporter destination table but I don’t see that table name updated in the target system, why is that?
This is because when the Data Exporter is updated to have a new destination, you need to run the export for the new table to be created. This process does not change an existing table in the target warehouse, nor does it delete the original table. It creates a new one.
What is the Optional S3 credentials section for?
This section is available for certain delivery types and acts as a staging location between the Data Exporter and the target location. The flow is as follows:
- Data is pulled into the TapAnalytics database (through Instant On connector or Smart Connector).
- Data Exporter pushes that data out to a target location.
- If the optional S3 section is used, then the data goes from TapAnalytics → S3 Bucket → Snowflake.
- If the optional S3 section is not filled out, then the data goes from TapAnalytics → Snowflake.
You can use the S3 bucket simply by filling in the fields shown below on the Data Exporter. When the transfer is complete between the S3 and Snowflake, the data on the S3 is purged.
Why would someone use the Optional S3 credentials section?
Snowflake is optimized to pull large data sets from Amazon S3, so this optional section works best for transferring large, granular data sets as it makes the transfer faster.
Approximately how much faster is it to use the Optional S3 credentials section?
Based on customer feedback, it’s about 10 times faster to use the S3 bucket:
- Transfer time with the S3: 00:00:12
- Transfer time without the S3: 00:12:47
I’m using the Optional S3 credentials section, why don’t I see any data in the bucket?
You will only see data in the bucket while the transfer is in progress.
What if I need the data to be in the S3 bucket and also in my target location?
In that case, you should build another Data Exporter that has Type = Amazon S3 (see image below).
Can TapClicks host an S3 bucket for me so I can use the Optional S3 credentials section?
No, not at this time. You will need to host your own S3 buckets.
I tried to connect to the Optional S3 credentials section and was hit with an error that my credentials are not working, what does this mean?
This means that we cannot export data to your S3 bucket because your credentials don't allow us to (see image below for example error message).
One way to resolve this is to go to the Amazon S3 Identity and Access Management (IAM) page, click on the user that is authenticated to the Data Exporter and on then click the Add permissions button for “AmazonS3FullAcess” (see image below).
Can I send data to Microsoft Azure SQL?
Microsoft Azure SQL is not listed as a data destination directly to help you maintain database integrity and security. The recommended way to get data into Microsoft Azure SQL is to use the Data Exporter to send data to Azure Blob Storage and then copy data from Azure Blob to an Azure SQL Database using Azure Data Factory. Learn more here.
Pricing/Contract Questions
Will failed rows count against my Data Exporter row limit?
No. Rows are only counted if the export is successful. If the export fails, the export is marked "Failed" and rows are not counted towards any amount of row usage.
How can I see how many rows my Data Exporter is expected to produce against my invoice?
Create a grid widget on a dashboard with the same configuration settings as your Data Exporter. Make sure the widget's "Styles" is set to "Paginate Records" and display all records for reports. From here, you can get an idea of the number of rows in your table and therefore in your export. To further review your data, you can export this data into a CSV file (see image below) and review the rows there.
This will give you an idea of how many rows to expect in a single export. If you intend to have your Data Exporter configured to run every day, consider taking this number of rows and multiplying it by the frequency that your Data Exporter will run. This will give you an estimate of how many rows you can expect from your Data Exporter.