Why data-mirroring is not data-backup

Collapse
This topic is closed.
X
X
 
  • Time
  • Show
Clear All
new posts
  • radhak
    Veteran Member
    • Apr 2006
    • 3061
    • Miramar, FL
    • Right Tilt 3HP Unisaw

    #1

    Why data-mirroring is not data-backup

    This is so wrong on so many fronts that I have nothing to say...

    JournalSpace was one of the many blogging/journal sites, in operation since 2002. Something or somebody gave them a body blow, and destroyed their data : Why Mirroring Is Not a Backup Solution

    They have been trying to fix it for some days by recovering the disk(s), but they have finally given up the ghost :

    Tuesday:

    Journalspace is no more.

    DriveSavers called today to inform me that the data was unrecoverable.

    Here is what happened: the server which held the journalspace data had two large drives in a RAID configuration. As data is written (such as saving an item to the database), it's automatically copied to both drives, as a backup mechanism.

    The value of such a setup is that if one drive fails, the server keeps running, using the remaining drive. Since the remaining drive has a copy of the data on the other drive, the data is intact. The administrator simply replaces the drive that's gone bad, and the server is back to operating with two redundant drives.

    But that's not what happened here. There was no hardware failure. Both drives are operating fine; DriveSavers had no problem in making images of the drives. The data was simply gone. Overwritten.

    The data server had only one purpose: maintaining the journalspace database. There were no other web sites or processes running on the server, and it would be impossible for a software bug in journalspace to overwrite the drives, sector by sector.

    The list of potential causes for this disaster is a short one. It includes a catastrophic failure by the operating system (OS X Server, in case you're interested), or a deliberate effort. A disgruntled member of the Lagomorphics team sabotaged some key servers several months ago after he was caught stealing from the company; as awful as the thought is, we can't rule out the possibility of additional sabotage.

    But, clearly, we failed to take the steps to prevent this from happening. And for that we are very sorry.

    So, after nearly six years, journalspace is no more.

    If you haven't yet, visit Dorrie's Fun Forum; it's operated by a long-time journalspace member. If you're continuing your blog elsewhere, you can post the URL there so people can keep up with you.

    We're considering releasing the journalspace source code to the open source community. We may also sell the journalspace domain and trademarks. Follow us on twitter at twitter.com/jsupgrades for news.
    It is the mark of an educated mind to be able to entertain a thought without accepting it.
    - Aristotle
  • Alex Franke
    Veteran Member
    • Feb 2007
    • 2641
    • Chapel Hill, NC
    • Ryobi BT3100

    #2
    Wow. That's just awful...
    online at http://www.theFrankes.com
    while ( !( succeed = try() ) ) ;
    "Life is short, Art long, Occasion sudden and dangerous, Experience deceitful, and Judgment difficult." -Hippocrates

    Comment

    • BobSch
      Veteran Member
      • Aug 2004
      • 4385
      • Minneapolis, MN, USA.
      • BT3100

      #3
      Another example of why you should

      ALWAYS DO BACKUPS!

      Don't rely on RAID, make sure you have your data stored on some other media, preferably off-site.
      Bob

      Bad decisions make good stories.

      Comment

      • JSUPreston
        Veteran Member
        • Dec 2005
        • 1189
        • Montgomery, AL.
        • Delta 36-979 w/Biesemyere fence kit making it a 36-982. Previous saw was BT3100-1.

        #4
        I'm sure others will back me up (no pun intended) on this, but never trust your backups completely, either. Have multiple backups off site. Test your backup plans. Do test backups and restores on a periodic basis.

        I had a tape drive to fail on me after the RAID controller on a file server went south about 10 years ago. Never was able to recover the server 100%. Since then, I've never really trusted tape. I use it, but I also have a set of backups on disks that I prefer to use. Tape should be and is the absolute last resort, at least in my shop.
        "It's a dog eat dog world out there, and I'm wearing Milk-Bone underwear."- Norm (from Cheers)

        Eat beef-because the west wasn't won on salad.

        Comment

        • nothing
          Forum Newbie
          • Sep 2007
          • 55
          • Central MS
          • Delta 36-979 with 30" Biese

          #5
          Originally posted by BobSch
          Another example of why you should

          ALWAYS DO BACKUPS!

          Don't rely on RAID, make sure you have your data stored on some other media, preferably off-site.

          QFT

          Back in my admin days we had 30 tapes for each server. We had the tapes in sequential order so if something got overwritten we could go back 30 days to recover the data.

          Comment

          • sparkeyjames
            Veteran Member
            • Jan 2007
            • 1087
            • Redford MI.
            • Craftsman 21829

            #6
            To bad for them. Maybe next time they will figure out that multi-gigabyte flash drives for use in backing up data is cheap as are DVD backups.
            Last edited by sparkeyjames; 01-02-2009, 01:10 PM.

            Comment

            • Kristofor
              Veteran Member
              • Jul 2004
              • 1331
              • Twin Cities, MN
              • Jet JTAS10 Cabinet Saw

              #7
              I don't know how large their DB was, but when you're talking multiple terabytes neither flash drives nor DVDs are the media of choice, but for home use they would normally suffice.

              Of course these folks were running for 6 years without an actual backup which is even more amaturish...

              Comment

              • crokett
                The Full Monte
                • Jan 2003
                • 10627
                • Mebane, NC, USA.
                • Ryobi BT3000

                #8
                Drive mirroring (or any RAID level) is not the same as backup. RAID is designed to keep a system up and running. I work with customers every day who have terabytes of data - one of them is implementing a petabyte installation. Even the ones who mirror entire storage systems offsite do not consider that a backup. Most all of them use tape. Some use disk but that gets expensive in a hurry, plus has the disadvantage of no convenient way to store it offsite.
                David

                The chief cause of failure in this life is giving up what you want most for what you want at the moment.

                Comment

                • cabinetman
                  Gone but not Forgotten RIP
                  • Jun 2006
                  • 15216
                  • So. Florida
                  • Delta

                  #9
                  We've replaced 2 HD's. LOML has since set up EHD's.
                  .

                  Comment

                  • sparkeyjames
                    Veteran Member
                    • Jan 2007
                    • 1087
                    • Redford MI.
                    • Craftsman 21829

                    #10
                    Originally posted by Kristofor
                    I don't know how large their DB was, but when you're talking multiple terabytes neither flash drives nor DVDs are the media of choice, but for home use they would normally suffice.

                    Of course these folks were running for 6 years without an actual backup which is even more amaturish...
                    With only 2 large HD's in a mirror array I doubt they were running multi-terabyte databases. You can also utilize hot swap HD's for backups as well. A restorable compressed SQL database dump is often far smaller than the database itself. Is a moot point anyway their own incompetence killed them.
                    Last edited by sparkeyjames; 01-02-2009, 09:06 PM.

                    Comment

                    • LinuxRandal
                      Veteran Member
                      • Feb 2005
                      • 4890
                      • Independence, MO, USA.
                      • bt3100

                      #11
                      This is always a problem when people don't pay attention. They don't look for a solution even when there are issues with backup size, verses drive size. Even then, not foolproof. I know of one instance, where a tape backup (don't see them used so often anymore), erased itself on the rewind after verifying.
                      She couldn't tell the difference between the escape pod, and the bathroom. We had to go back for her.........................Twice.

                      Comment

                      • crokett
                        The Full Monte
                        • Jan 2003
                        • 10627
                        • Mebane, NC, USA.
                        • Ryobi BT3000

                        #12
                        Originally posted by sparkeyjames
                        You can also utilize hot swap HD's for backups as well..
                        I've fixed too many problems with hard drives in RAID arrays to recommend this as a regular backup solution. You can do this in a pinch but it puts extra load on the RAID controller (especially with a RAID level that requires parity like RAID 5). It also requires extra churn on the drive(s) that stay in the system, thereby shortening drive life.
                        David

                        The chief cause of failure in this life is giving up what you want most for what you want at the moment.

                        Comment

                        • LinuxRandal
                          Veteran Member
                          • Feb 2005
                          • 4890
                          • Independence, MO, USA.
                          • bt3100

                          #13
                          Originally posted by crokett
                          I've fixed too many problems with hard drives in RAID arrays to recommend this as a regular backup solution. You can do this in a pinch but it puts extra load on the RAID controller (especially with a RAID level that requires parity like RAID 5). It also requires extra churn on the drive(s) that stay in the system, thereby shortening drive life.

                          Machine mirroring, may work with hardware failure, not software/sabotage.

                          With today's larger (half a terabyte or larger) drives, what do you recommend for home backup?
                          She couldn't tell the difference between the escape pod, and the bathroom. We had to go back for her.........................Twice.

                          Comment

                          • Wood_workur
                            Veteran Member
                            • Aug 2005
                            • 1914
                            • Ohio
                            • Ryobi bt3100-1

                            #14
                            Originally posted by LinuxRandal
                            Machine mirroring, may work with hardware failure, not software/sabotage.

                            With today's larger (half a terabyte or larger) drives, what do you recommend for home backup?
                            An external Hard drive.
                            Alex

                            Comment

                            • nothing
                              Forum Newbie
                              • Sep 2007
                              • 55
                              • Central MS
                              • Delta 36-979 with 30" Biese

                              #15
                              Originally posted by LinuxRandal
                              Machine mirroring, may work with hardware failure, not software/sabotage.

                              With today's larger (half a terabyte or larger) drives, what do you recommend for home backup?
                              Also doesn't work for fire/storm damange. This comes back to off-site backups. It really comes down to how much time and money your data is worth. I would think if you base a business on data storage it would be top priority. Redundancy is the key.

                              Comment

                              Working...