Backup and Transferring Images: From the Field Laptop to Home Workstation
Related: backup, data integrity, diglloydTools, Eastern Sierra, IntegrityChecker, laptop, Lundy Canyon area, peak bagging
Out in the field, I download my camera cards each night to my MacBook Pro Retina laptop. This acts as one level of backup*.
When I return home, I copy the files over to my Mac Pro workstation. But how do I know with certainty that the files are transferred without errors and that all files are really copied?
The odds are low of an error (corruption) but I want to know, since before long those files will be erased from the laptop. The same principle applies to any file transfer. The only thing that cannot be known is whether the original copy off the camera cards is correct, though even that is possible with a 'diff' program.
To preclude file corruption, I use diglloydTools IntegrityChecker to create SHA1 hashes** on the files on the laptop. I wrote IntegrityChecker some time ago for this very purpose and similar ones.
Since the SHA1 cryptographic hashes exist within each folder once this is done, over on my Mac Pro, I can verify the copied folder(s) to see that all files are there, and that they are unaltered***.
See also How to Safely Transfer Data or Verify Backups; the issue of file integrity is a general one applicable to any data-transfer situation.
* I insist upon large-capacity cards (32GB or 64GB) so that I rarely ever erase the images on the cards during my trip: this is another level of backup. Just prior to downloading, I rename the folders on the card to have the the form YYYY-MMDD-camera-lens-subject, e.g. 2013-1107-NikonD800E-Zeiss135f2-LundyCanyon. This both organizes my work and prevents the camera from seeing the images for the next day’s shoot (to avoid confusion while shooting).
** SHA1 is a type of cryptographic hash. Single bit errors are detected with a extremely low probability of missing anything (e.g., your odds of winning the lottery jackpot 10 times in a row are higher).
*** The salient difference between hash verification and a 'diff' is that a 'diff' always requires that two copies exist; the IntegrityChecker (".ic" hidden file) hash file within each folder allows verification of all files in that folder at any time using only that copy. This is both faster and more resilient and semi-permanent for archiving purposes.
Example
I use the command line version preferentially, but a simple GUI is available.
diglloyd:ZeissGuide lloyd$ ic verify 2013-1107-NikonD800E-Zeiss135f2-LundyCanyon IntegrityChecker(tm) v1.2.1 64-bit, diglloydTools 2.2.2, 2013-10-26 13:20 Copyright 2006-2013 DIGLLOYD INC. All Rights Reserved Use of this software requires a license. See https://macperformanceguide.com/Software-License.html ... details omitted ... Processed 7.63GB in 13.7 seconds @ 569.7MB/sec ================================================================= ic verify 2013-1107-NikonD800E-Zeiss135f2-LundyCanyon Monday, November 11, 2013 at 1:44:58 PM Pacific Standard Time ================================================================== # Files with stored hash: 178 # Files missing: 0 # Files hashed: 178 # Files without hashes: 0 # Files whose size has changed: 0 # Files whose date changed: 0 # Files whose content changed (same size): 0 # Suspicious files: 0
