Large digital library advice...cloning, copying and/or backing up

So like many of you I have a large digital library. I have been slowly re-ripping my entire CD collection losslessly over the past year and I have about 3TB taken up of a 4TB drive.

The scenario is that I have that drive (G-Tech) hooked up VIA firewire to my home computer in my listening room a mid-2011 iMac.

About every 60-90 days I copy the contents of that drive to a second Seagate backup plus that I keep offsite in my recording studio. Both drives are exact copies of each other and once I do my manual backup I also copy my itunes library file and take that with so in effect I have the exact same music library on both computers, playlists and all.

The problem I am wishing to solve is this. Right now I simply wipe the backup drive and re-copy all of the files fresh which takes almost 16-20 hrs of chugging and churning . Recently I got a program that compares folders on multiple drives and I can manually replace the files that have been added, changed and/or upgraded. Of course that takes a good long afternoon to do as well, especially since I am a compulsive tinkerer, I add and change artwork, split greatest hits collections into the original albums and with jazz box sets I put all of the songs onto the original albums, 10" releases and such.

My question is, is there an easier way to do this? In other words is there a backup/copy program that can compare the one drive to the other (backup drive) and just copy and change only the new or altered files?

I have tried reading about carbon copy cloner and superduper, but they keep mentioning boot partitions and such which I dont care about as this is not my system drive...I use Time machine for that.

Also I need the secondary drive to look, act and function just like the parent drive so it can be a plug-in play swap. I can't have files encripted, zipped or encoded.

Will one of these programs do this or am I dreaming, anyone else here mirror your drives for use on two separate systems at different locations?

Thanks in advance!

XJ

 

Part of the problem, as I see it, is that new music files should be copied, but altered files are usually a problem, and should be replaced with the original version. I get that with the iTunes library, you would want to keep the newest version, though. What program are you using to compare folders?

 

Its actually called "Compare Folders" I got it from the Appstore

 

I can't recommend ChronoSync highly enough! I've used it for years. It's do a quick comparison and sync two folder, either bi directional or one way or another. Has lots of filtering and scheduling options available as well. Compare Folders is likely a simpler more limited option.

DeltaWalker also seems pretty popular. They both have trials so you can give one a shot.

 

I've used Karen's Replicator to do this sort of thing for many years. It's easy to use and it's free. It's normally available from www.karenware.com, but the site is down at present, as friends of the late and much-missed Karen move it to a new host. You can find it on other download sites though.

Hope this helps.

Phil

 

Unfortunately it looks like Karen's is PC only and I am on a Mac

 

Thanks for the responses so far!

 

The key to backups is redundancy and simplicity.

One copy is not a complete backup. You need two.

Get Carbon Copy Cloner from the App Store. It is fast, powerful and super easy to use. Get another 4TB external drive. If you get another FireWire drive they can be daisy chained.

Now, configure CCC to clone the original drive onto the new one, and set it to do so at whatever intervals make sense to you - I would choose nightly. The first backup of the whole drive will take a while. The rest will be super fast, as they are incremental. CCC does this automatically, and keeps a backup of deleted files on the target drive as well. It's cool.

Now, get the off site drive and plug it in. Configure CCC to clone the original onto it, choosing the option 'whenever the drive is mounted'. This way, whenever you get the drive from your studio and attach it, CCC will do its thing, no intervention needed. Just remember to grab it every couple of weeks.

Boom. You're done. Two backups, one nightly, one fortnightly. All incremental, for speed. All automatic, for foolproofness.

 

Of course with a Mac and firewire, you could just do a Time Machine backup which is truly incremental fast and on the fly. Contrary to some beliefs, you can use two backup drives and cycle them in and out, it just takes awhile for the first backup when they are swapped.

 

Time machine doesn't fit his requirements of having non encrypted/hashed/etc file-by-file backups. I prefer this myself as well.

I'm curious in trying replication software as well, as currently I'm doing pretty much what the OP is doing, which is incredibly time consuming. Adding complexity to the mix is my use cases involve having my music on my laptop for some cases, and having it on my desktop/server for serving to Sonos and the like, and then a cloud backup. Ideally all three of these would be mirrored. That's going to be tough to pull off, and I think I'm going to have to settle for updating the cloud every couple of months, which is fine if it's just another layer of redundancy I suppose.

 

...thanks for the heads up on this software - it looks like exactly what I need.

Just this week I started to run out of space for music on my 3TB Seagate drive so I purchased a 4TB replacement. The 4TB arrives and I start copyng the 3TB stuff to it (it seems slower than normal). After a couple of days of copying the 3TB drive is quite warm and then dies !

Fortunately I have most of my stuff copied on to the 4TB drive and what is not is mostly on a previous 2TB drive.

The 3TB drive was still in warranty so once I get the replacement back from Seagate I will set Karen's Replicator up to copy back changed files to the 3TB drive.

Remember folks - the work involved in copying a large amount of data off a previously good drive can kill it !!

 

And to think some think playing vinyl is too much bother.

 

There are some really cool syncing programs out there. Basically allowing you to roll your own Dropbox type setup on your machines.

I haven't tried Sparkleshare yet, but intend to give it a whirl at some point. Might be worth a look.

http://sparkleshare.org

 

I use Get Backup2 to sync my ITunes library to two USB drives. Seems to work well. Not quick though.

 

Ditto recommendation of CCC. Saved my royal bacon last month when two drives failed on me simultaneously. I also started with CrashPlan; $60 a year for unlimited backups.

 

I've used Crashplan in the past, the way it offers differential backups makes it especially nice for music libraries as it will only backup your new albums.

 

I've used SuperDuper! in the past to do this and it handles incremental updates well. I currently have everything on an external Synology server, which has it's own incremental backup software.

Even doing an incremental update it can take up to a day. It's a pain, but it saved my a** last month when I had to restore my entire library (don't ask why, my fault) from my last backup. It took almost four days to restore everything, but it was a lifesaver.

 

I use SuperDuper to run a nightly incremental backup of my media drive. Once every couple of weeks I bring home a HD that I keep at work, run an incremental backup, then take the drive back to work the next day. This gives me an off-site backup and access to my entire library while at work.

 

I suggest you investigate Syncovery. I use it to do what you say, with about the same sized library that you have. Because it only copies changed files, I can do it every time I make a change to the master library.

I use it on Windows, and a Mac version is available. I've been using computers since 1967 and PCs since 1983, and I think Syncovery is one of the best programs I've seen for this kind of thing.

 

I use Super Duper and Backblaze.

 

I use Allway sync and it works very well in Windows. You would need to install sync gateway for a MAC. Free SW.

 

Just downloaded the 30 day trial of Carbon Copy Cloner. Gonna start with that. Thanks everyone.

 

Theadmans, sounds like what you need is raid.

 

Raid is not a backup.

 

That's true. RAID simply protects from drive failure.

I've only had two drives fail in 10 years of having a digital collection. Once it was a laptop drive that was backed up daily using Apple's Time Machine software. The other was the gradual failure of a drive in my RAID that I simply swapped out.

There have been several times where I've made a catastrophic "oops" and been glad that I had a separate backup to go back to.