I have a firewall running OpenBSD 3.7 installed on a SATA disk known to the system as wd0. An identical disk, wd1, is available and not being used. For added data security, I would like to mirror the content of wd0 to wd1 without re-installing or taking the box offline for more than a couple of minutes. After searching the net for HOWTO’s and information I finaly found a NetBSD guide, that helped me in the right direction. The missing pieces was found in the raidctl manpage. Below are the steps that I have taken to achive my goal. You should be able to use my instructions on your own system by changing device names and/or create labels according to your own needs. If you decide to use this guide, please make sure you have backed up all data. Proceed at your own risk!
Modify the Kernel Compile and reboot a new kernel with RAIDframe enabled:
option RAID_AUTOCONFIG # RAID Autoconfiguration pseudo-device raid 4 # RAIDframe disk driver
Initialize the second disk with fdisk This is only needed on i386 systems:
$ fdisk -i wd1
Setup disklabels With the disklabel -E wd1 command, you can add appropriate labels. I have added the following:
a: for complete minimal installation, 2G b: for swap, 512M c: unused e: for RAID rootfs, 2G f: for RAID swap, 2G g: for usr,home,etc, the rest
Remember to set FS type to RAID for e,f & g (FS type: [4.2BSD] RAID)! If you fail to do this, the system won’t find the raid devices.
Copy OpenBSD to wd1a It is not possible to load the kernel from a RAID device, so we have to make a small partition with a full OpenBSD installation and your RAIDframe aware kernel. Maybe it’s possible to skip this and load the kernel from a floppy or something… i don’t know.
Create filesystem on wd1a and mount:
$ newfs /dev/wd1a $ mount /dev/wd1a /mnt
Copy required data to wd1a (I have /, /var & /usr as seperate mounts)
dump -0 -f - / | (cd /mnt && restore -x -f -) dump -0 -f - /var | (cd /mnt/var && restore -x -f -) dump -0 -f - /usr | (cd /mnt/usr && restore -x -f -)
Modify /mnt/etc/fstab to look like:
[root@firewall ~] $ cat /mnt/etc/fstab /dev/wd1a / ffs rw 1 1
Umount wd1a:
$ umount /mnt
wd1a should contain a working copy of your OpenBSD installation including your new kernel with RAID support build in.
Create RAID configuration :
The disk /dev/wd9 is non-existing and is used as a dummy for wd0 during setup. Remember to create the device node for wd9:
$ cd /dev && sh MAKEDEV wd9
Create RAID configuration for raid0: (Read the raidctl man page for more details on the syntax.)
$ cat < raid0.conf
Create RAID configuration for raid1:
$ cat > raid1.conf <
Create RAID configuration for raid2:
$ cat > raid2.conf <
Create the RAID devices (use -c if re-creating):
$ raidctl -C raid0.conf raid0 $ raidctl -C raid1.conf raid1 $ raidctl -C raid2.conf raid2
Initialize the component labels each with a unique serial:
$ raidctl -I 200509290 raid0 $ raidctl -I 200509291 raid1 $ raidctl -I 200509292 raid2
Initialize the RAID with verbose output:
$ raidctl -iv raid0 $ raidctl -iv raid1 $ raidctl -iv raid2
Setup RAID disklabels
Setup a label on raid0 for the root filesystem:
$ disklabel -E raid0 a: for /, all available space
Setup a label on raid1 for swap:
$ disklabel -E raid1 b: for swap, all available space
Setup labels on raid2 for the other mountpoints:
$ disklabel -E raid2
I have created the following labels:
- e: for /tmp, 2G
- f: for /var, 8G
- g: for /usr, 4G
- h: for /usr/local, 8G
- i: for /usr/obj, 4G
- j: for /usr/ports, 4G
- k: for /usr/src, 4G
- l: for /home, rest of the disk
This is properly overkill, but I like to seperate things and my box is equipped with two 200Gb drives and i won’t need the space for anything.
Create filesystems and populate
So far so good. Now we need to create the filesystems with newfs and copy data from wd0 to the raid devices:
Create and copy to raid0a (/ -> /mnt)
$ newfs raid0a $ mount /dev/raid0a /mnt $ dump -0 -f - / | (cd /mnt && restore -x -f -)
Create and copy to raid2e (/tmp -> /mnt/tmp)
$ newfs raid2e $ mount /dev/raid2e /mnt/tmp $ dump -0 -f - /tmp | (cd /mnt/tmp && restore -x -f -)
Create and copy to raid2f (/var -> /mnt/var)
$ newfs raid2f $ mount /dev/raid2f /mnt/var $ dump -0 -f - /var | (cd /mnt/var && restore -x -f -)
Create and copy to raid2g (/usr -> /mnt/usr)
$ newfs raid2g $ mount /dev/raid2g /mnt/usr $ dump -0 -f - /usr | (cd /mnt/usr && restore -x -f -)
Create and copy to raid2h (/usr/local -> /mnt/usr/local)
$ newfs raid2h $ mount /dev/raid2h /mnt/usr/local $ dump -0 -f - /usr/local | (cd /mnt/usr/local && restore -x -f -) $ umount /mnt/usr/local
Create and copy to raid2i (/usr/obj -> /mnt/usr/obj)
$ newfs raid2i $ mount /dev/raid2i /mnt/usr/obj $ dump -0 -f - /usr/obj | (cd /mnt/usr/obj && restore -x -f -) $ umount /mnt/usr/obj
Create and copy to raid2j (/usr/ports -> /mnt/usr/ports)
$ newfs raid2j $ mount /dev/raid2j /mnt/usr/ports $ dump -0 -f - /usr/ports | (cd /mnt/usr/ports && restore -x -f -) $ umount /mnt/usr/ports
Create and copy to raid2k (/usr/src -> /mnt/usr/src)
$ newfs raid2k $ mount /dev/raid2k /mnt/usr/src $ dump -0 -f - /usr/src | (cd /mnt/usr/src && restore -x -f -) $ umount /mnt/usr/src
Create and copy to raid2l (/home -> /mnt/home)
$ newfs raid2l $ mount /dev/raid2l /mnt/home $ dump -0 -f - /home | (cd /mnt/home && restore -x -f -) $ umount /mnt/home
Modify fstab, unmount and make bootable
$ cat /mnt/etc/fstab /dev/raid0a / ffs rw 1 1 /dev/raid1b none swap sw 0 0 /dev/raid2l /home ffs rw,nodev,nosuid,softdep 1 2 /dev/raid2e /tmp ffs rw,nodev,nosuid,softdep 1 2 /dev/raid2g /usr ffs rw,nodev,softdep 1 2 /dev/raid2h /usr/local ffs rw,nodev,softdep 1 2 /dev/raid2i /usr/obj ffs rw,nodev,nosuid,softdep 1 2 /dev/raid2j /usr/ports ffs rw,nodev,nosuid,softdep 1 2 /dev/raid2k /usr/src ffs rw,nodev,nosuid,softdep 1 2 /dev/raid2f /var ffs rw,nodev,nosuid,softdep 1 2
Unmount /mnt/var, /mnt/usr, /mnt/tmp & /mnt
$ umount /mnt/var $ umount /mnt/usr $ umount /mnt/tmp $ umount /mnt
Make second disk bootable
$ mount /dev/wd1a /mnt $ chroot /mnt /usr/mdec/installboot -n /boot /usr/mdec/biosboot wd1 $ umount /mnt
The NetBSD guide wrote something about parity errors when rebooting without unmounting the swapfs. I have added the following to my /etc/rc.shutdown file just in case:
swapoff=YES # Unconfigure swap to avoid parity errors
if [ "X${swapoff}" != X"NO" ]; then
echo "Unmounting swap on raid1"
swapctl -d /dev/raid1b
fi
Make RAID autoconfigurable & reboot
Make the RAID’s autoconfigurable with the root filesystem on raid0:
$ raidctl -A root raid0 $ raidctl -A yes raid1 $ raidctl -A yes raid2
Reboot your system and remember to boot wd1 !
$ shutdown -r now
When the OpenBSD boot loader appears, type: boot wd1a:/bsd
During boot you should see something like
Kernelized RAIDframe activated raid0 (root): (RAID Level 1) total number of sectors is 4194176 (2047 MB) as root raid1 (root): (RAID Level 1) total number of sectors is 4194176 (2047 MB) raid2 (root): (RAID Level 1) total number of sectors is 226476160 (110584 MB)
Add your primary disk
Relabel wd0 with same layout as wd1
$ disklabel wd1 > disklabel.wd1 $ sed -e 's/wd1/wd0/g' disklabel.wd1 > disklabel.wd0 $ disklabel -R -r wd0 disklabel.wd0
Add hotspares (wd0), remove dummy devices & reconstruct raid (one at a time is preferred). Check recontruction with raidctl -s raid.
Add hotspare, remove dummy & recontruct raid0:
$ raidctl -a /dev/wd0e raid0 $ raidctl -F component0 raid0 $ raidctl -S raid0
Add hotspare, remove dummy & recontruct raid1:
$ raidctl -a /dev/wd0f raid1 $ raidctl -F component0 raid1 $ raidctl -S raid1
Add hotspare, remove dummy & recontruct raid2:
$ raidctl -a /dev/wd0g raid2 $ raidctl -F component0 raid2 $ raidctl -S raid2
When all raids have been rebuild, it’s time to rebuild the parity:
$ raidctl -P raid0 $ raidctl -P raid1 $ raidctl -P raid2
Check the parity with raidctl -p raid’. Only when Parity Status: clean you’re safe.
NOTE: remember installboot on wd0!
Final Words
I hope my information was of any use. It took me a full day to figure out :)
Please contact me if you find any errors or know a better/easier way to accomplish this.