Deduplication

Deduplication

Deduplication, as the name implies, searches for and creates a Report of duplicates in a file location. Deduplication can be run against any Window’s file path including local, network or shared drives. By default, the Deduplication tab configuration can be added in the Tab Editor.

 Location Selection

 The location selection, by default, will be a dropdown list of Electronic Profiles that have been setup in the repository. Selecting an Electronic Profile from the dropdown and hitting Run, will generate a report of duplicates in that profiles location.

 Advances Location Selection

Apart from defined Electronic Profiles, specific Window’s (or UNC) paths can be entered. For example, “My Documents” in the local “C:” drive can be deduplicated by entering:
“C:\Users\<USERNAME>\Documents”

Multiple locations can also be entered separated by a semicolon(;). For example to deduplicate “My Documents” from the local “C:” drive as well as the “Finance” area of the NAS drive “nas2” the following text would be entered:
“C:\Users\<USERNAME>\Documents; \\nas2\Finance”

 Report Area

 After running against the desired location(s), the main Report Area will populate with all duplicate files found. These will be grouped by what is known as an MD5 hash of the file. This compares the files at the memory level, to ensure files that are found to be duplicates are complete duplicates.

Note: file name, path or metadata are not considered for duplication, only the file itself.

 Columns

The Report Area displays six main columns: File Name, Size, Type, Folder, Created and Modified.

Grouping

 

 

The main Report Area will group files by the file MD5 hash. The blue headers will be the generated hash.

 

Check-boxes

There are check-boxes at the left side of each file in the Report Area. These allow the user to specify which files are duplicates and/or which files are to be moved. When moving files, all checked files will be moved out of their original location to a safe duplicate area described in Removing/Restoring Duplicates below.

 

Removing/Restoring Duplicates

 Move

 When clicking Move, you will be prompted for a location to move duplicate files to (the safe duplicate area). This adds a layer of insurance in the deduplicate process. Files that are checked in the Report Area will be moved to this location.

Note: The folder structure is preserved when Move is performed. This eases the restoration of these files outside the application.

 Merge (Restore)

 Merge re-locates files that have been moved as duplicates and prompts for a location where these duplicates were previously moved (the safe duplicate area). Enter the location of the desired files and click “Confirm” to restore a previous move.

 

Reporting

After a Deduplication Run is complete, the Report can be exported to Excel. Under the Report menu at the top of the application, select Generate Report. This opens excel and populates a new spreadsheet with the current information from the Report Area in AgileIM.