How to Do Different Import Schedules for the Same Project? Tags!

The Import Scheduler supports multiple schedules. As each schedule is setup separately, different settings can be used per schedule, like different FTP servers, different update intervals, different Meta Data transformations, etc. To get full support in the data activation process when using multiple import schedules a Tag logic can be used.

So the Import Tags are used for activating needed data files when multiple data sources with different behaviors have been used. 

This makes it possible to use different activation logic for different data sources.

In an example where project contains survey data and sales data:

Survey data should be imported to the project once a week. Each survey data file contains the new respondent only (e.g. the respondents from the last week). Sales data is imported once a month. Each sales data file contains new and historical data. During the automatic data activation process, the following must happen:

• Survey data import: The new survey data file is appended to existing survey data (all existing survey data sets and the new data set should be activated).
• Sales data import: The existing sales data set should be inactivated, and the new sales data set should be activated.

Example 2

A project should be loaded with data once a week.

Each data file will contain both new and old respondents (data from 2014 and forward).  In the initial setup of the project, historical data was loaded in one data file (data before 2014). During the data activation process, the following must happen:

• The existing data sets with data from 2014 should be inactivated. Only the new data set should be activated.
• The historical data set should be activated just like before the import.

By using the Tags in both examples above, the defined activation behavior in the “Import Scheduler” only effects the data sets with the same Tag. Other data sets will not be effected (the ones that were activated will still be activated, and vice versa). So in both examples above, two different tags are used to solve the needed process.

Tags can be created in two pages within Forsta Visualizations: the Import Scheduler page and the Data Import page (in the tab “Import” only, not in the “Quick Import” tab). The logic is the same in both places. Click “Edit Tags” and a popup window will appear for creating, editing, and deleting Tags.

Step by step

1. Hover over the arrow at the bottom of Import data icon to access Scheduled import setup.
2. Press Edit tags button to access the Import tag options.
3. To create a new Tag, enter the name, and then click Add new tag.
4. To rename a Tag, select the Tag in the list, enter a new name, and then click Rename.
5. To delete a Tag, click the Delete icon on the Tag.

Tags can be assigned to data sets in three places:

1. In the Data Import screen
2. In the Import Scheduler screen
3. In the Activate Data screen

To assign a Tag to a data set that is going to be imported, select the Tag from the selection lists shown in the image below. If no Tag should be assigned, select “No Tag.” To change the Tag of an already imported data set, or to add a Tag afterwards, go to the Data Activation page and follow these steps:

1. Click Edit in the farthest right column
2. Select new Tag in Import Tag column
3. Click Update in farthest right column

Note: New Tags cannot be created in the Data activation page, only in the Import Scheduler page and the Data Import page (in the Import tab). If the Tags should be used during data activation in the Import Scheduler, check the option highlighted in the image above. If this option is checked, only data sets with the same Tag as the current schedule will be effected during data activation.

Note: Setup of multiple import scheduler

The setup of multiple Import Schedules is made in the existing screen in the top area. As default, all projects will have one empty Import Schedule created from the beginning.

•          To select an Import Schedule, use the selection list Selected import schedule.

•          To create a new Schedule, click Create, enter a name in Schedule name, and continue with setup.

•          To delete an Import Schedule, simply click Delete.

Some restrictions are present in using multiple Import Scheduler support. These restrictions are relevant if you use multiple Import Schedules where data should be imported at the same time in two or more schedules: Two or more Import Schedules cannot be setup for the same start time. So, in cases where two data files should be imported at 00:00 every night, one needs to be scheduled at 00:00, and the other one scheduled at 00:15. If reports should be pre-generated in the example above, this should only be setup in the second Import Schedule. There is no need to pre-generate reports in the 00:00 schedule if more data is supposed to be imported. If the first data import/data activation runs for more than 15 minutes (in the above example), the second one will not be started, since only one can run at a time. This is solved by using the function Retry if unsuccessful. In case one data fails to import or any other step fails in the import, the scheduler will stop and not complete the remaining steps.