Background

Advertising is changing fast, and so is the data that comes with it. Our agency combines data from services such as Facebook Ads, Google’s Campaign Manager 360, Google Ads, Bing Ads, Snapchat, and others so we can track and optimize advertising campaigns across channels together in Domo. 

However, some of the services listed above delete data from their system after 24 months. Additionally, because advertising data can change up to 90 days (or longer!) after an action has occurred (as attribution modeling updates or fraudulent clicks are removed), we are interested in a rolling window of data.

When pulling data, we can easily change date ranges in pre-built Domo connectors and use the append functionality to capture historical data. We can then create a safe haven within Domo for our data and know that it won’t get deleted. 

However, while Domo has 99% of the connectors we need, sometimes we need to work with the Domo API to create a custom solution.

Problem statement

Let’s say I’ve developed a solution that pulls data from 91 days ago until yesterday into Domo. I want to save the historical data after that 90-day rolling window (the 91st day) but I don’t want to spend time redeploying my solution with a different date range in the pull. 

Solution statement

We can leverage the DataSet Copy Unload Connector to create a loop within Domo that backs up data. This connector is easy to use. All you have to do is set up Domo API credentials, then specify the Dataset ID you want to copy and the Domo instance you’re grabbing the data from. 

Demo

First, we set up an ETL that uses a filter-formula to grab the information from 91 days ago from our custom connection. Then, we use the DataSet Copy Unload Connector with the dataset ID from the dataset produced by the ETL, set the update mode to append, and set it to run on a schedule to update before the original connection runs again. 

Just like that, you’ve got your historical data from a rolling window of data safe in Domo. I hope this post was helpful and introduced you to something new in Domo.