For most of your problem, rsync is the classic solution. It will
notice that files actually differ (e.g. a corruption from an
interrupted copy) and update it with a fresh copy. Likewise, if a
file has been altered, it will update the original version with the
new version. There are also options to delete files that aren't in
the source tree, so if they are renamed or you have older versions
you can prune them to make it an exact mirror of your source.
Thus you might do something like
$ rsync --dry-run -avr ~/Music/ /media/player/my_music/
to make /media/player/my_music/ contain all the files in ~/Music/
I'm adding the "--dry-run" flag which doesn't actually do the copying
but shows you what it would do. If everything looks kosher, then you
can remove the "--dry-run" flag to actually perform the copying. For
this first example, it's more optional. But for this example:
$ rsync --dry-run -avr --delete ~/Music/ /media/player/my_music/
it will delete files that aren't in the source ~/Music/ directory, so
it's nice to get a report of which files would be deleted before you
potentially nuke valuable data.
Searching for *exact* duplicates (i.e., renames) isn't too hard. I
wrote a little Python script
https://github.com/Gumnos/dedupe
that will find exact-match duplicate files regardless of name and (if
you don't specify the "--dry-run" flag) will also deduplicate them by
making either hard-links or symbolic-links depending on your
options. But you can use the output of a dry-run to find
exact-content-match duplicates.
However, finding partially-matching duplicates poses a much hairier
problem.
-tim
On August 14, 2017, Jeffery Mewtamer wrote:
Okay, so using the cp command with the -Rn switch is convenient for===========================================================
say copying my Music folder from my home directory to an external
hard drive or the SD card for my portable Media player when the
destination already has an older copy without needing to copy
everything already present in the old copy, manually determining
what's been added and copying manually, or dealing with a bunch of
prompts. It also allows an aborted copy to more or less be resumed
from where it left off.
This method is simple enough to not require scripting or complex
command syntax, but it does have a few downsides:
1. It won't overwrite corrupted files left by an interrupted copy,
and such files are too rare for manual searching.
2. Files that have been altered don't get copied. unless they've
changed filename. Not a big issue for copying my Music folder since
those files are seldom altered, but copying say, my writing folder
can lead to the backup media containing only older drafts of some
documents.
3. It tells me nothing of files from an older copy that have been
deletd/renamed since the last copy. jdupes can find old files on the
destination media if the new file is just a rename, but it can't
help with files that have been altered as well as renamed.
I suppose what I'm looking for is a command line utility or script
that executes the following pseudo code:
Given directories source and destination:
for every file found in both source and destination:
if file.source != file.destination
prompt user whether to overwrite one version of the file with the
other or to add the files to a list for later examination.
For every file only in source:
search for match in destination.
If match found prompt user towhich to rename to match.
If not match found, copy to destination.
For every file only in destination:
search for match in source.
If match is found, prompt user to which file should be renamed.
If no match is found, prompt user whether to copy of delete.
If anyone knows of command line utilities to help with this task, it
would be greatly appreciated.
===========================================================
The raspberry-vi mailing list
Archives: //www.freelists.org/archives/raspberry-vi
Administrative contact: <mike.ray@xxxxxxxxxxxxxx>
-----------------------------------------------------------
Raspberry Pi and the Raspberry Pi logo are trademarks of the
Raspberry Pi Foundation.
This list is not affiliated to the Raspberry Pi Foundation and the
views and attitudes expressed by the subscribers to this list do
not reflect those of the Foundation.
Mike Ray, list creator, January 2013
