Remove duplicate files (lite)

From KCLUG Wiki

This does not replace the media scrub script, and is not as thorough. This script presupposes the filename not changing during archival. The purpose of this snippet is to recurse the tree of an archive of files, and for every file in that tree, check for a matching file in the current directory. When a match is found, the file in the current directory is deleted.

find /path/to/archive/root/ -type f | sort | while read file ; do diff "$file" "./`basename "$file"`"&>/dev/null; if [[ $? -eq 0 ]]; then echo Deleting ./`basename "$file"`;rm -f ./`basename "$file"`; fi;  done