How to keep your photos safe

[morts]

Mods, please don't delete my placeholder posts as this is going to be a long post and I thought it better to break it up into sections instead of having to fight with the 10000 character limit


Time for another thread about that bane of us digital photographers: keeping your data safe.

As with my web design thread, feel free to comment and post your suggestions!

Contents:
1. Your disk and how it can fail
2. Steps to prevent drive failure
3. Steps to prevent data loss
4. Miscellaneous tips

Your Hard Drive

Unless you have money coming out of your ears and can afford a set of SSD drives, you'll have a straightforward steel box with some spinning platters inside. The read/write heads are mounted on robotic arms that whizz across the surface of the platters as the drive spins at a typical 5400-10000 RPM.



These connect to your machine through IDE or SATA or iSCSI or SAS or possibly even through the magic of unicorns.

Types of failure

I'm going to rate each instance with a number to signify how easy it is to recover yourself. A lower number means it's likely a professional job whereas a higher one means you could do it in a couple of hours.

A hard drive is a mechanical device so it's bound to fail due to wear and tear eventually. That said, I have drives that I've been using since 1999 and they still show no signs of stopping. Manufacturers will include a Mean Time Before Failure rating in the specs of the drive. This will tell you how many hours of operation it's expected to be able to do before dying a noble and honourable death.

Drives will either fail after a good while of use, or very soon after purchase. If it's the latter, then it's a manufacturing defect and you can safely RMA the drive and get a replacement.

Recoverability rating: 3

A hard drive can also fail electronically - i.e. the controller can malfunction, causing the drive to stop reading/writing correctly. This isn't the end of the world as if you have another identical drive, you can power both on and then swap the controller boards across. The new board will take over running the disk and you should at least be able to recover your data.

Recoverability rating: 6

The motors inside the drive can burn out. This can take the form of either the arm actuators or the motor that spins the platters biting the dust. In either case, the data is recoverable, but not without spe******t kit, as you need to have the platters spinning up again and the arms moving to do anything.

Recoverability rating: 3

Another common cause of hard disk death are bad sectors. This happens when the drive is unable to read a specific location on its surface. Sometimes this is caused by the OS drivers messing up and sometimes it's a physical glitch on the surface of the disk. In either case, a bad sector isn't necessarily a huge problem and for the most part, the drive will carry on working around it. But there will come a time when the drive needs to read that sector and cannot. This will likely cause it to crash.

To prevent this, keep an eye on your SMART monitoring status. 99% of drives made this century have SMART monitoring which keeps track of a myriad different variables, one of which is possible dodgy sectors. The drive's controller picks them up and keeps track of them for you. You can download Active@ Disk Monitoring to do just that here

If it flashes up warnings about any of the health checks, now is the time to run like wildfire down to your local electronics store, buy a USB drive and backup.

To recover a bad sector, you often need to do a low-level format of the drive. This will physically reset all sectors of the disk to 0 instead of clearing the "used" flag on them, as regular formatting does. You will lose everything and there is no chance of post-format recovery then. On the upside, your drive will live.

Recoverability rating: 7

Finally, the worst type of failure is a head crash. These are nigh-on impossible to recover from at home.

During a head crash, the read heads that skim the surface of the drive, mere nanometers above, make contact with the platter surface. If it's a brief contact, it'll cause a bad sector and you'll be spared the aftereffects that are this bastard's trademark. In most cases, however, it will score a groove along the platter, ruining the data on there and making the platter unreadable. Occasionally, the vibration in the arm will make the head bounce across the surface like a skimming stone. Either variation is a bad thing.

On the nanometer scale that disk sectors are packed together, that is the equivalent of a crack the size of the Grand Canyon suddenly appearing where a smooth surface filled with data should be.

When this happens, your system will likely freeze and become totally unresponsive. You might also hear a repetitive clicking from the drive. That's the sound of the heads flicking back and forth across the platters as it tries to work out where a whole chunk of its data is gone.

Shut down and get thee to a data recovery monkey

Recoverability rating: 1

[morts]

Steps to prevent your drive failing

As I outlined above, there are a few ways a drive can fail.
Some drives are more prone to certain types of failure than others by virtue of the environment they are in. I will explain in greater detail:

• Laptop drives are more prone to failing due to head crashes and bad sectors
• Desktop drives are more prone to software errors

Those are obviously gross generalisations but my experience leads me to believe that's fairly true.

Let's start with laptops.

The reason for those drives having a greater likelihood of failing mechanically is simply that they get more abuse than a drive in a tower case. Carrying the laptop around, bumping it on things and occasionally dropping it does no favours to the hard drive.

Unless you have a MacBook of some kind, then you are likely to see this type of failure sooner or later if you own a laptop. I single out MacBooks because they have an accelerometer (think orientation sensor in iPhones) built in which detects if the laptop is subjected to any irregular forces. The instant this happens, it will park the heads of its hard drive to avoid damage.

So, to prevent disk failure when using a laptop

• Avoid knocking it especially when it's on and so forth.
• This may be really obvious to some but NEVER unplug the drive when your machine is on. Even if it says "hot-swappable". Chances are, your laptop is a single-drive machine (even if it has multiple partitions for recover and so on). Unplugging its only disk would be like pulling out your heart for a transplant without first hooking yourself up to a life support machine.
• Avoid powering down the laptop by pulling the battery out.

Desktops

Here you're more at risk of damage through unexpected power cuts than dropping your case. Unless you play frisbee with it.

If you really care about your system uptime, invest in a UPS (Uninterruptable Power Supply) unit and run your machine through that. It also has the dual benefit of absorbing and neutralising any power fluctuations like spikes and dips in the line, thereby "cleaning" your power feed. This means you are less at risk of blowing out something vital in your setup. It's the setup that is used on a much larger scale in all major datacentres around the world, so it's proven.

Conversely, not providing enough power to your drives can also cause them to malfunction so ensure your PSU is up to the task of powering your machine. If you have a beefy graphics card like a GTX 2xx or ATI 48xx series, you need at least a 650W PSU to run it along with the rest of your machine.

As with laptops, avoid shutting down the PC by removing the power cord. Even a forced shutdown (holding down the power button for 5 seconds) is healthier as the BIOS will send a command to the disks to park their heads.

Fun fact: the disks don't stop spinning for several seconds after you shut down your PC. Those "clicks" that you hear when your PC shuts down for good are the heads parking and then you get a "neeeoowww" noise as the platters spin down.

A final point: when installing or removing disks, try to avoid dropping them. That's never good.

[morts]

Preventing data loss

I gave this one a section all of its own because physical failure is one thing, but losing data is another. And there is, as they say, more than one way to skin a cat.

While you can't guard against some types of physical and electronic cockups, you can ensure your data remains intact. Here are some good ways of doing so:

Use a USB drive for backups ONLY

This is the cheapest and easiest way of running backups. I'll explain in greater detail later on in the post.
I wanted to draw peoples' attention to the "ONLY" part. If you say "hey, I have loads of space on this drive!" and use it as a primary datastore then you're running as big a risk as you would with an internal drive. If the USB drive fails then you lose whatever was on it. If that data was just backups then there's basically no data loss. Sure, you lose redundancy but you don't lose any data.

Remember: when backing up and when removing USB drives, ALWAYS STOP THEM FIRST BY USING THE "REMOVE SAFELY" button

USB disks are NOT hot-swappable. You need to disconnect the drive at the OS level first before physically unplugging it.

KEEP AN EYE ON YOUR DRIVE(S)

Refer to my first post about S.M.A.R.T. monitoring software and install some. It will give you warning when a drive is about to die on you so you can get all your data off it ASAP.

Separate your OS from your data

A lot of you have your OS and your files, documents and photos all on one drive because you bought your PC from Dell and they said "yeah 500GB is plenty!". And you believed them.

They are, technically, correct. However if you look at it another way, that's a whole load of data to potentially lose if you

• Get a virus
• Have to reinstall Windows
• Accidentally mash the Delete key
• Spill beer on your computer
• Have a disk failure

So the first thing I would do as a step to ensure data integrity is move your OS to a separate 60-80GB SATA II drive. You can change the location of My Documents and associate folders easily and the windows shortcuts will all work and get along! You can thus reinstall Windows with impunity without worrying about making sure you don't whack "FORMAT" by accident and lose everything.

Image your windows install

This is less of a data protection tip and more of a "how to get back up and running when shit hits the fan" tip.

When you install Windows for the first time, apply all updates, install Office/Photoshop/Firefox/Antivirus/Porn and nothing else. Just the core system bits that you use often.

Then download DriveImage XML and UBCD4Win. Burn the UBCD stuff to a CD (or better still, two), label them and KEEP THEM SAFE.

I'll explain what both tools do.

DriveImage

This app will make an image of any one of your drives (or partitions) and store it wherever you tell it to. I'd recommend a USB drive that you use for backups.

If your Windows install breaks and you end up reformatting, you can simply use DriveImage to reformat the system drive and then restore the image it took after install. This skips having to do all the pointless buttonmashing that the XP/Vista installation is and you'll be back to your "default" install in no time.

UBCD

The Ultimate Boot CD was originally made as a suite of command-line tools like memtest86 and various diagnostic utils for CPUs, motherboards and hard disks. It included things like SeaTools for working with Seagate drives and so on.

The Windows one is prettier and comes with a bunch of diagnostic and data recovery apps (too many to list), one of which is DriveImage. This means that if your system has gone tits upward, you can simply pop in the CD, boot from it, run DriveImage and restore your image!

Backup your data

This is obvious and you should all be doing it. I personally have over 280GB of photos in my main directory. If I lose the drive they are on, that's a lot of work gone out the window.

Lightroom also has a function for backing up its catalogs which you should all be using.

For God's sake, don't put the catalog backups on the same disk as the catalog. If the disk fails you have neither backups or catalogs.

So, the easiest way of backing your stuff up is to go out and buy a USB drive. Buy big. You'll be surprised how fast it fills up. Also, if you intend to use it for backup, don't put working files on it just because it's free space. The drive should be purely for backup.

You can either use the Windows Backup app, download a 3-rd party app, or use Robocopy which comes with Windows to copy files over to another location.

I do this in the following manner:




Select "Idle"



Now, for the arguments, enter the following:

D:\Where\Your\Photos\Are E:\Where\You\Want\To\Backup\To /S /E /COPY:DAT /Z /XO /R:0 /W:0 /LOG:"C:\My Documents\Logs\Photos.txt"

Replace the source, destination and Log directories with your own

This will copy any files and folders from the source that are newer than the destination or don't exist at the destination. And it will do so with no need for any work on your part as the process will run when your PC is idle.

Optical Media

Some of you have expressed the opinion that optical media (DVDs) is a good way to keep your data safe. Unfortunately, I would strongly discourage you to do this. A few years ago when USB drives were prohibitively expensive, maybe. Not now.

It also relies on your remembering to back up your archive to DVDs, label them and store them where they won't get scratched, thereby ruining your backups. Use Robocopy and a USB drive instead.

The Danger Zone or NAS/DAS/RAID/Oh god what do I do?!

If you have more money than sense or are paranoid, a USB drive or 2 is fine. But who wants to rely on that when you can harness the power of RAID and network-attached storage solutions?

That's right, buy a NAS. Look it up on your local PC retailer. They start at about £150 for an entry-level single-drive solution and go up. Get one with 2 drives and make sure they are running in RAID 1. Stay the **** away from RAID 0. Don't ever touch it. Ever. EVER.

The difference between the two is that in RAID 1, your data is mirrored across however many drives are in the array. This way, if any drive were to fail, then the array will still be intact due to the replication. You lose storage space but gain N+X redundancy where X is the number of additional drives.

RAID 0, however, spreads your data across the array, making it into one big single drive. If a drive in RAID 0 fails, it's like suddenly losing a chunk of your hard drive to the ether. The array will not be able to recover from a failed drive and you will lose EVERYTHING.

The next step up is investing in a home server. This can be any old box you have lying around (preferably with computer parts in it) which has some drives and a network card. Install your favourite flavour of Windows and create some shares. Then robocopy/backup to those shares. If your server has multiple drives (good idea), set up a robocopy job nightly to mirror the contents of specific folders across the drives for RAID 1-style redundancy.

[morts]

Miscellaneous Tips

Here's where I'll add useful posts people make. Post more, get more!

Originally Posted by Kn1nJa

Another way I have seen people back up photos is on a flash drive which is then kept in a safe location. While this may sound like a good idea, it is not the best way to archive files, especially important files. I work in a computer lab at my college and I see flash drives fail a LOT. Most often the drive just quits working and cannot be fixed (cheaply). While flash drives are great for transporting files, I would not recommend them for long term storage..

Can't argue with that. Flash drives have a limited number of read/writes by design and are about the same in longevity and reliability as using optical media for backups.

Originally Posted by rediguana

Another thing I'd suggest with respect to laptops, if you can, get them a stand to sit on so that air can flow over and under them. Whilst not proven, I've had a number of hard drive failures in laptops, and I'm fairly convinced that these were caused by the laptop running hot on a flat desk with inadequate ventilation. The additional heat buildup in the laptop dries out the circuit boards, and could eventually lead to failure.

I'm sure you've noticed how hot a laptop can get - just touch the keyboard or underneath surface. My laptop now hardly ever overheats and both keyboard and underneath surface are normal room temperatures when you touch them. It may be a small thing, but I believe that ensuring the laptop doesn't run hot all the time will extend its life, and reduce the risk of failure.

Disks CAN fail through overheating because the processor on the chip will start throwing errors, much like your CPU will when it overheats, leading to bad sectors. It can also cause internal components to expand beyond their operating tolerance, leading to malfunctions.

That said, drives will happily run at 70 - 80 degrees without too much complaint.

Originally Posted by Jamesc359

I think DVDs and CDs have gotten a bad reputation for backup because of inadvertent abuse. I've got cheap discs that I purchased and burned many years ago that are in perfect condition, while discs off the same spindle that I burnt for some family members had catastrophically failed within a couple of years. It doesn't take much effort to protect your optical discs, but it does take some.

I think the most important thing to remember is that optical media is made possible because of a thin layer of organic dye that is smashed between two plastic discs. While this dye is significantly more advanced than the pigments Leonardo used to paint the Mona Lisa, they still share one very common trait - exposure to light slowly degrades it. Ultra-Violet light in particular exacerbates the aging process. How do you protect your discs from UV light?

1. Never leave a disc in the sun, this can destroy even the best of discs in a matter of weeks. Short term exposure can also warp discs.
2. Always store your discs with the data side down so the label can absorb the majority of light.
3. Place your discs in a case with a opaque bottom.
4. Store in a cool area that receives as little light as possible or in a opaque container.

Because I handle them by their edges and always lift and lower my discs into the drive tray and case, scratches has never been a problem for me - with the exception of discs I've lent. Sliding a disc into the case or tray will cause scratches.

On laptop coolers:

They make laptop coolers that sell for anywhere from $20-$30 on up. These are a thin case you place your laptop on that contains fan[s] to circulate air under it, metal coolers also act as a giant heat sink. Most also have a USB hub and some features drawers for storing small items. Given the size of a stand, coolers are more convenient when traveling, plus active cooling is also far more effective.

One word of caution though, unless your plugged in they will drain your battery quicker.

If you care for your discs then I see no problem with using DVDs as a backup medium. I just think that most photographers would opt for the option that involves the least work, since backing up to DVD requires actually doing it :P

[Nicole]

Great posts so far (and I think the placeholder posts are a great idea so that if silly people like me reply before you're finished typing it doesn't break up the tutorial throughout the thread )

[Sime]

Indeed... Great post - I just bumped down the image to 740px... I just looked better, I hope you don't mind!

Looking forward to seeing the rest of the series! Thanks.

Sime

[Jim Bryant]

Good post Morts. Bout time a techie helped put

[LoveDSLR]

Please consider including some of the following if they fit the topic.

All PC
Backup (disk imaging) the boot partition.
Separate partitions between boot/programs and data.

Desktop
well-ventilated box.
Use 24/7 enterprise drives for critical stuffs.

[kirbinster]

Great info. Might I add that my Compaq notebook also has accelerometer capability to protect the hard drive.

[Kn1nJa]

It may not be the best option, but burning your images to a CD or DVD is a good way to make a backup. They last a long time and can save your rear in the event of a system crash. The only downside is the risk of scratching the data layer (the label side .... the shiny side is actually looking through the tough plastic to the data layer. If you get a scratch that you can see light through, the disc is essentially ruined).

Another way I have seen people back up photos is on a flash drive which is then kept in a safe location. While this may sound like a good idea, it is not the best way to archive files, especially important files. I work in a computer lab at my college and I see flash drives fail a LOT. Most often the drive just quits working and cannot be fixed (cheaply). While flash drives are great for transporting files, I would not recommend them for long term storage.

Sorry for the book .... I'm a computer geek and I tend to ramble on about stuff I feel passionately about and/or love a lot.