In Harmoni, you can replace or remove existing data sources from your project. It is essential that you do this with caution to avoid losing any existing project definitions.
In this article
1. Overwrite
Overwrite allows you to replace a data source with updated data.
Some scenarios include:
- New tracker update, where the project has been created using a cumulative file, and a new wave of data is available.
- When there are changes in the original source, like adding new variables or data cleaning.
Harmoni will only identify an updated source when the name and source type are the same.
- Select view/add sources from the action pane.
- Select ADD/REMOVE from above the source names.
- This opens the Sources area and displays a list of the data sources linked to your project.
- Choose Upload or Connect.
- Navigate to the location of your updated data source(s).
- Harmoni will ask you to overwrite the original file with the new updated one.
- Make sure that all sources you wish to include in your project are selected.
- When ready, click the OK button.
Prior to loading your project, if there are any original project items that are no longer present in any of the sources, Harmoni displays a warning as they will be removed completely from the project. If you choose to Continue, the project will load.
See Preview Removed Items in section 3 below.
2. Remove Data Sources
You can replace data sources with new data or completely remove specific sources from your project. It is important to do this with caution to avoid losing any existing project definitions that could break your project and stories.
You must add the new data file to your project as a new data source and harmonize it before removing the old source(s).
To conserve project definitions, before removing sources, it is important to add the new sources, load the project, check what is new, harmonize axes and elements as relevant and then remove the sources you no longer wish to use.
a. Add New Sources
For example:
You have created a project with two separate sources for one market (i.e., New Zealand 2017 and 2019), you now have a cumulative file (Brand Monitor_NZ), so you want to use this one instead.
You will need to upload your new sources and load your project, including the sources you wish to remove.
- Select view/add sources from the action pane
- Select ADD/REMOVE
- This opens the Sources area and displays a list of the data sources linked to your project.
- Choose Upload or Connect
- Navigate to the location of your new data source(s).
- Make sure that all sources you wish to include in your project are selected. It is important at this stage to keep the existing sources, even those you wish to remove.
- When ready, click the OK button, and your project will load. The project will include new and old sources; this step is required so the logic that is defined by the old sources can be transferred to the new sources.
b. Harmonize
After loading your project, Harmoni automatically identifies what's new. New items are shown in bold in the project tree. You can also identify new items by using the New icon under the Project tree filters.
Make sure you apply all harmonization changes required; after this, you can go back and remove the sources you no longer need.
- Identify NEW items (using the Project tree filters) and harmonize as required.
c. Remove Sources
Once you have harmonized axes and elements as needed, you can remove the sources you no longer need in your project.
- Select view/add sources from the action pane
- Select ADD/REMOVE
- This opens the Sources area and displays a list of the data sources linked to your project.
- You can now unselect the sources you no longer want to use by unticking directly on the source tile or using the X on the source name in the source diagram.
- Make sure that all sources you wish to include in your project remain selected.
- When ready, click the OK button.
Prior to loading your project, if there are any original project items that are no longer present in any of the sources, Harmoni displays a warning as they will be removed completely from the project. If you choose to Continue, the project will load.
See Preview Removed Items in section 3 below.
d. Delete Sources
Once you have removed the sources from your project, you can delete them.
Go back to the Sources area and on the source tile, use the three-dot menu to access the delete option. Note that you will not see the delete option until after the source has been removed from the project.
3. Preview removed items
Before Loading a Project
After overwriting or removing a data source and before you load your project, Harmoni displays a warning message when existing project items are not present in the new data source.
The ‘Preview removed items’ tick box is enabled by default. When ticked, Harmoni displays a warning when there are original project items no longer present in any of the sources and will, therefore, be removed from the project. At this point, you can choose to continue or cancel the action.
For example:
You have created a project with two separate sources, one for Australia and one for New Zealand. You now have an updated data file for Australia; however, this new file no longer contains the variable Brand Drink Most Often, which was only present in Australia. By overwriting the data source and updating your project, the variable will be removed from the project and will no longer be available as a reference for any constructions in which it has been used.
- Make sure that all sources you wish to include in your project are selected
- When ready, click the OK button
- As we are replacing a source file here, Harmoni asks if you want to Overwrite
- Prior to loading your project, if there are any original project items that are no longer present in any of the sources, Harmoni displays a warning as they will be removed completely from the project. If you choose to Continue, the project will load.
Harmoni will identify what is new in a project. It also identifies any constructions that need to be redefined due to items being removed from data sources.
After Loading the Project - Invalid/Broken Constructions
After the project loads, you may have broken constructions. Harmoni automatically identifies broken constructions as a result of a data source being removed or overwritten. Invalid constructions are shown in red in the project tree. You can also identify invalid constructions using the ERR icon under the Project tree filters.
As well as seeing the axes and element(s) turn red when a construction is broken, Project Owners, Creators (non-owners) and Explorers see the word Missing included where the source element is no longer available.
If an element that is used in construction is removed, Harmoni will list the axis in the construction definition and show the element as missing.
When the whole axis is removed, the definition shows both the axis and element as missing.
You can choose one of the following options:
Fix the Construction
After the project loads, find and update all the elements that are broken.
Important: When fixing a broken construction, ensure that all elements are fixed. You will not be able to save the construction if there are any elements that are broken, so please review every element in the construction.
Delete the Construction
If the construction is no longer required, you may delete the construction, but ensure that you discuss this with your team before proceeding.
If items break, so too will Stories created using these items.
It is important to discuss with your team and/or researchers using the project before proceeding.
- A source file is overwritten
- Harmoni warns that items will break
- You can choose to Continue or Cancel at this point
- Only continue if you know that removing and breaking items is expected and understood by all users of the project
- Identify invalid constructions by using the ERR option
Where to from here?
Learn more about data sources
- New Project - Managing data sources
- Add data sources to your project
- Append data sources to your project