Linux Software-RAID 1

This is a short guide on setting up Linux software RAID-1 (mirroring) on a running system. Please test everything in a virtual machine or on some unused hardware before you try this on production servers. There could be errors and typos that could wipe your disks, you have been warned. If you find any errors, please let me know :)

Requirements

• Two or more disk-drives
• Kernel with md & raid1 built-in (or as modules in initrd)

It is recommended to use two identical disks, but it is not required. If you are using IDE disks, you should place either disk on separate controllers for best performance. This is also true for SCSI and other devices.

Install Linux

If you already have a working Linux system running, you can safely skip this step, but make sure you have the md-driver built-in to the kernel and the mdadm tools installed. I prefer Debian-based distributions, but any recent Linux distribution should work. Install your favorite Linux on the first disk, and make sure you do a minimum installation (this will shorten the copy process later). During the disk setup process, I spend time creating a though-out partition-scheme. If you don’t do this now, it is possible to fix later, but it takes more work. When your Linux system boots, make sure you have mdadm installed (Debian: apt-get install mdadm) and a kernel with md+raid1 built-in or in initrd. Most recent distributions have md and raid drivers in initrd.

Setup Disks

If you made all the proper partitions when you installed the system, you just need to clone the partition-scheme from the first disk (we use /dev/hda) to the second (we use /dev/hdc). We use the sfdisk program to dump the partition-scheme from /dev/hda and later write it to /dev/hdc. The sfdisk program takes care of changing /dev/hda to /dev/hdc, so we don’t have to edit the hda.out file.

$ sfdisk -d /dev/hda > hda.out  # Dump hda partition table
$ sfdisk /dev/hdc < hda.out      # Make hdc look like hda

After changing partitions, it is always a good idea to reboot. When booting, make sure the system comes up in single user mode. We make sure (with fdisk -l /dev/hda & fdisk -l /dev/hdc) that the disks looks alike.

Create RAID Devices

We need to create a RAID device for each partition (including swap!), but I will only write examples for a few. It is a good idea to write down raid-devices and partition mappings (eg. /dev/md0 = /dev/hda1 + /dev/hdc1) on a piece of paper, to avoid any mistakes later on. Below we create two raid devices, one for the root file system (currently on /dev/hda1) and one for swap (currently on /dev/hda2). It would be very stupid to add our primary disk (which is still in use) to the RAID, so we use the word "missing" instead of the corresponding partition on /dev/hda. I love this feature :)

$ mdadm --create /dev/md0 --level=1 --raid-disks=2 /dev/hdc1 missing
$ mdadm --create /dev/md1 --level=1 --raid-disks=2 /dev/hdc2 missing

This should leave us with two broken mirrors, /dev/md0 and /dev/md1. We can check this by reading the /proc/mdstat file: cat /proc/mdstat 

The output should show that both raid-devices are online, but with failed devices (because we created them with one part of the mirror missing). 

We need to create file systems on the raid-devices, before we are able to mount and copy our data.

$ mkfs -t ext3 /dev/md0
$ mkswap /dev/md1

Normally you would create a RAID-device for each partition on /dev/hda and create a file system on the RAID-device.

Populate The RAID

We should have all of our RAID-devices working now (still in broken/missing mode). We copy data from the real disk (/dev/hda) to the RAID (/dev/md...) one partition/mount-point at a time. Repeat the following for all partitions (except swap) with corresponding dirs.

/ partition: 

$ mount /dev/md0 /mnt
$ cd /
$ find . -xdev | cpio -pm /mnt

/usr partition: 

$ mount /dev/md2 /mnt/usr
$ cd /usr
$ find . -xdev | cpio -pm /mnt/usr

and so on ..

Change configuration on RAID

We should have a complete copy of the original disk on the RAID-devices now. Before we continue, we need to change some configuration files, to reflect our new device names. Most important is /etc/fstab which should be modified to use our RAID devices instead of the /dev/hda partitions. Our partition-to-raid scheme on paper comes in handy now. The GRUB configuration file should also be changed, so our root file system is /dev/md0 instead of /dev/hda1.

Before we reboot and test our new RAID, we setup RAID auto configuration... 

RAID Autoconfiguration

To enable the RAID auto configuration feature, we use the fdisk utility. Start fdisk /dev/hda and for each partition change the partition-type to 'fd' (Linux RAID-Auto configuration). Do the same for /dev/hdc.

When we reboot the system in a minute, we need to pay special attention to the boot loader. After the GRUB prompt appears, press 'e' to edit the highlighted kernel. On the line containing the path to the kernel-image we should change root=/dev/hda1 to root=/dev/md0, to make sure we mount /dev/md0 as our root filesystem. While we are at it, make sure we boot into single-user mode as well (add the word single to the end of the same line). 

Repair broken mirrors 

If all went well, you should have your system up and running again. If you type mount, you should see md-devices being used instead of /dev/hda. If something went wrong you still have your old-system intact, just reboot again without changing the GRUB prompt. If everything looks right (double check!), it's time to repair the broken mirrors. Our original disk (hda) is not being used anymore, so we can safely add it's partitions to the md-devices.

$ mdadm /dev/md0 --add /dev/hda1
$ mdadm /dev/md1 --add /dev/hda2

I like to watch the raid-recovery process with the command watch cat /proc/mdstat. 

Save RAID Configuration

When we are using the RAID auto configuration feature, the mdadm.conf file is not strictly needed, but it will be useful for some utilities and in case of recovery. To generate the mdadm.conf file, type the following:

$ echo "DEVICE /dev/hda* /dev/hdc*" > /etc/mdadm/mdadm.conf
$ mdadm --detail --scan >> /etc/mdadm/mdadm.conf

Now is also a good time to tell grub about the second disk, in case the primary disk should fail.

$ grub-install /dev/md0

Have fun :)