What is my method to keep my data healthy for a prolonged period? I want to share my methodic in this blog post. Quick summary: In my opinion, is a NAS (network-attach-storage) with a next-generation file system the best solution to store my data. I will explain this.
In 2021 the two most used methods to store our information are: Put the data on a magnetism surface and or as an electric charge in a chip. Both storage methods have their disadvantages and advantages.
Magnetic storage is the technology used in hard drives and with backup tapes. Hard drives deliver a cheap way to store terabytes of data easily. You can buy them everywhere and they are easy to connect. They have a chance to malfunction because of sensitive moving parts. Most hard drives will function for five years. Big corporations are using tapes as a backup medium. They are a reliable and the cheapest medium to store a lot of data. A disadvantage of tape is that it takes time to read a specific part of the tape. Finding one document can take minutes. The magnetic field on hard drivers and backup tapes will slowly lose its power. After ten-twenty years, it will get harder and harder to read the information.
Electric charge is the method to store data in a memory chip. Popular uses are Solid-state drives(SSD) and camera memory cards. But also USB thumb drives and storage build-in a device like a media player.
Engineers are pushing the limits with the data cells in memory chips. Data cells are designed to hold one bit (SLC). With the help of modern controllers and advanced algorithms, the engineers have pushed this to four bits. (QLC) (Five bits memory chips are coming (PLC)) This technological leap has made memory chips cheaper. The downside is a higher chance of data corruption. The engineers have tackled this with an algorithm that checks the data regularly. However, a disconnected SSD can become unreadable after a few years. Another downside is that memory cells are wearing out faster. With one bit, you can store 100.000 times information in one memory cell. With four bits, this drops to 500-1000 writes before the cell breaks. Combining more bits in one cell is more stressful. If you write a lot of data, the four-bit memory chip can fail in a few years. Or even months.
What if you store all your information on one storage device? For example, an external hard drive or a USB thumb drive. You don’t touch this device for a given time. Slowly your storage device will reach a Schrodinger state. You still have your data, or all your data is gone. You don’t know which state your data is in. The only way to find this out is to connect your storage device to a computer. A question that keeps popping up on the internet is: “I connected my external storage device to my laptop. But it doesn’t work anymore. Can someone please help me? It contains all pictures of all my vacation trips.”
I can’t handle the truth that certain people store all their valuable information on one single storage device without a way to know their data is still there. I need to know what is the state of my data. I need a device that can tell me how my data is. Yes. I care about my data.
I store my valuable information on a NAS (network-attached storage device). The combination of hardware and advanced software in one package gives me that extra feeling of security. For the following reason:
My NAS has two magnetic hard drives. The NAS writes the data to both drives. You can say that the two drives are a mirror of each other. If one drive malfunctions, all the data is still on the other drive. I can replace the broken drive with a replacement drive. The NAS will copy all the data from the working drive to the new drive. When that process finishes, all my data is mirrored again.
Next-generation file system
Synology has adapted and improved the BTFRS file system. It’s is a next-generation filesystem with checksums and snapshots.
Next-generation file system: Checksums
A checksum is a little bit of extra stored information regarding the existing data on the NAS. Every time I open a document, the NAS will first run a quick check to see if the checksum is correct. In the worst case, the data deviate against the checksum. In other words. The data is damaged. The software on the NAS will now self-repair the data with the data from the second hard drive in the NAS. If this still fails, the NAS will notify me about the data corruption. I can take action to recover this data from a backup. Every three months, the NAS will run a scrub. This process will check the whole filesystem against the checksums. It takes 12 hours, but it gives me the secure feeling that all my data is healthy. The disadvantages of checksums are: It takes more space and is a tiny bit slower. You can enable it specifically for your valuable data.
Next-generation file system: Snapshots
Snapshots are a great way to protect my data against my stupidity. I have enabled snapshots with a one-hour interval. The NAS will freeze every hour, specified data on the NAS. The second thing I have configured is how long the data is stored. One year for my important data. Seven days to a month for non-important data. Snapshots are like a time machine. I can go back in time to get my data at a specific time. Why is this a great way to protect my data against my stupidity? What if ransomware gets access to my network shares and starts to encrypt my data on the NAS. The ransomware can keep its activities undercover for a few weeks before it makes it releases its demands. I can pay some bitcoins to get access back to my files, or I can log in on my NAS with my secret administration credentials to restore a snapshot from the past before disaster struck. Snapshots are also great if I completely fucked up one document. Restore the latest correct document from the past, and I can continue with my tasks. The disadvantage is that the snapshots will take extra space on your NAS. However, this extra space is kept to a minimum because BTFRS will only store what has changed between every snapshot. Why would you buy a big hard drive only to keep it empty?
My NAS is just a tiny black box connected with two wires. One to the power supply, the other to the network. Sometimes I get the idea that some people think mirroring your data in a NAS is enough. To have this thought is dangerous. A NAS is a fragile device. Dogs; cats; meteorite impact; rats; lightning; floods; burglary; asteroid impact; hackers; karma; children; zombies; power spikes; earthquakes; fire; building collapse; cleaning service; sinkholes; mothers-in-law; the hand of God. A list of random things that can be a danger to your NAS. For the peace of your mind, it’s highly recommended to backup your data to an offsite location. Every night my NAS makes an encrypted backup to a remote location. I don’t need to think about it. Is it an impact on my internet upload speed? No, I’m sleeping. I don’t need to think about my backups. Also, I make offline backups because my NAS and my remote backup location are so-called online backups. An offline backup is a backup I can store offline: Everywhere, like in my backpack or a desk drawer. All my backups are encrypted. Once a week the NAS takes some time to check the reliability of the backups. This is another failsafe that Synology has build-in to make sure the backups are healthy.
No news is good news. Nowadays I feel the same about e-mails. If I receive an e-mail for every successful backup, I will get crazy. So the NAS only sends me an e-mail if the backup wasn’t a success. Also, I get an e-mail on every first day of the month with a Drive Health Report. Mostly the e-mail has good news: “No problem detected with the drives in DSM.”
The NAS is connected to your home network. This gives you the freedom where to install the NAS. There is no limitation. You can locate the NAS at any place in your house or office that feels safe or has the best environment. The NAS doesn’t need to be close to your computer. If your notebook is on a kitchen table, the NAS can be in your office in the attic. You don’t need to connect a USB cable. You can reach your files over the wifi network. Do you want to install the NAS at a safe location in a locked cupboard? Do it. Do you want to install the NAS in the basement because it has a stable environment temperature. Do it. You can keep your data fysical out of sight.
This blog post explains in a nutshell, why I use a NAS to store my long-term and short-term valuable data. I treat all information I store on memory cards and USB thumbsticks as non-important data.
It is a balance you have to find with how much you want to invest and how much your data is worth. Do you want to invest in buying as much data as possible, or do you choose for less data storage? But in a more reliable configuration. It’s your call.