Bootable-RAID on SuSE 7.3 Step by Step

Introduction

This document describes the setup of a bootable RAID configuration using SuSE Linux 7.3 using the provided standard kernel 2.4.10, initrd (which is used by default), lilo Version V22 and the provided raidtools 0.90.

There are several documents covering Bootable RAID configurations, but either I did not like the concept, the information was outdated, or the information provided was either too general or too specific to convince me to adopt it to my box.

Other documents covering the theme can be found on:
http://www.tldp.org/HOWTO/Boot+Root+Raid+LILO.html
http://www.netsys.com/library/suse/howto/en/Software-RAID-HOWTO.txt
http://www.fokus.gmd.de/linux/dlhp/aktuell/DE-Software-RAID-HOWTO.html
Another Bootable RAID (German only) approach based on a SuSE Distribution can be found
http://www.pl-berichte.de/t_netzwerk/raid1.html
http://www.tldp.org/HOWTO/Root-RAID-HOWTO.html

I finally decided to create my own concept which was finally a "best of breed" based the informations I found all over the web and on:

• lilo version >= 22, which is capable of booting RAID 1 (Mirroring) partitions
• the standard SuSE 2.4.10 Kernel
• the initrd (initial ram disk) subsystem which is used also by default in SuSE 7.3 installations.
• 2 IDE harddisks /dev/hda and /dev/hdb. /dev/hda contained a fully operational linux installation which was migrated to the RAID-Volume

Disclaimer and License:

Der Autor sowie die EvoTegra GmbH übernimmt keinerlei Gewähr für die Aktualität, Korrektheit, Vollständigkeit oder Qualität der bereitgestellten Informationen. Haftungsansprüche gegen den Autor sowie die EvoTegra GmbH, welche sich auf Schäden materieller oder ideeller Art beziehen, die durch die Nutzung oder Nichtnutzung der dargebotenen Informationen bzw. durch die Nutzung fehlerhafter und unvollständiger Informationen verursacht wurden, sind grundsätzlich ausgeschlossen, sofern seitens des Autors kein nachweislich vorsätzliches oder grob fahrlässiges Verschulden vorliegt.
Bei direkten oder indirekten Verweisen auf fremde Internetseiten ("Links"), die außerhalb des Verantwortungsbereiches des Autors liegen, würde eine Haftungsverpflichtung ausschließlich in dem Fall in Kraft treten, in dem der Autor sowie die EvoTegra GmbH von den Inhalten Kenntnis hat und es ihm technisch möglich und zumutbar wäre, die Nutzung im Falle rechtswidriger Inhalte zu verhindern.
Der Autor sowie die EvoTegra GmbH erklärt hiermit ausdrücklich, dass zum Zeitpunkt der Linksetzung keine illegalen Inhalte auf den zu verlinkenden Seiten erkennbar waren. Auf die aktuelle und zukünftige Gestaltung, die Inhalte oder die Urheberschaft der gelinkten/verknüpften Seiten hat der Autor sowie die EvoTegra GmbH keinerlei Einfluss. Deshalb distanziert er sich hiermit ausdrücklich von allen Inhalten aller gelinkten /verknüpften Seiten, die nach der Linksetzung verändert wurden. Diese Feststellung gilt für alle innerhalb des eigenen Internetangebotes gesetzten Links und Verweise sowie für Fremdeinträge in vom Autor eingerichteten Gästebüchern, Diskussionsforen und Mailinglisten. Für illegale, fehlerhafte oder unvollständige Inhalte und insbesondere für Schäden, die aus der Nutzung oder Nichtnutzung solcherart dargebotener Informationen entstehen, haftet allein der Anbieter der Seite, auf welche verwiesen wurde, nicht derjenige, der über Links auf die jeweilige Veröffentlichung lediglich verweist.

This document is copyright(c) 2002 for EvoTegra GmbH and it is a FREE document. You may redistribute it under the terms of the GNU General Public License.

Neither the author nor EvoTegra GmbH is responsible for any contents linked or referred to from his pages - unless he has full knowledge of illegal contents and would be able to prevent the visitors of his site from viewing those pages. If any damage occurs by the use of information presented there, only the author of the respective pages might be liable, not the one who has linked to these pages. Furthermore the author and EvoTegra GmbH is not liable for any postings or messages published by users of diskussion boards, guestbook's or mailinglists provided on his page.

While the given information in this document is believed to be correct, no liability for the contents of these documents can be accepted. No person, group, or other body is responsible for any damage on your computer(s) and any other losses by using the information on this document. i.e.
"THE AUTHORS AND ALL MAINTAINERS ARE NOT RESPONSIBLE FOR ANY DAMAGES INCURRED DUE TO ACTIONS TAKEN BASED ON THE INFORMATION IN THIS DOCUMENT.

Target audience

The target audience are experienced Linux administrators *only*. Try this only when you really know what you are doing.

IMPORTANT *BEFORE* YOUR START

Please note that dealing with RAID in this specific context means dealing with partitions, initrd and lilo.

This can quickly result in complete unrecoverable loss of data.

MAKE SURE YOU HAVE COMPLETE AND READABLE BACK UP OF ALL YOUR DATA!

If you feel not comfortable with trying it yourself, EvoTegra GmbH will be glad to assist you with this task. This support of course can not be for free. We ask for your kind understanding that running a business can not be based on an "all for free" service. Please contact EvoTegra GmbH just in case you are willing to pay for good service or you have any hints or corrections related to this document.

READ THE COMPLETE DOCUMENT BEFORE YOU TRY ANY STEP.

The Maintainer

Tobias Manthey. You can reach me at howto@evotegra.com .

Step 1. Install lilo v22.2

SuSE 7.3. uses a patched version of lilo version 21.7. There is no specific SuSE package, but I just grabbed and installed an rpm package from http://rpmfind.net, which installed without any problems. To make sure everything is working run lilo from the command line after installation.

The first time I now got a message like:

"Fatal: Message to must be <64k".

Checking my lilo.conf revealed the entry:

message = /boot/message

The size of /boot/message was about 100kB. Some enquiry revealed that this message came up due to the patched lilo version used by SuSE.

SuSE has by default a nice graphical lilo menu. I simply commented the line out and lilo updated my box without complaints. The penalty is the loss of your graphical lilo menu, you will be presented the old lilo text style instead. Despite this loss I did not suffer any loss of functionality.

Step 2. Relocate the /boot/ directory

Booting from an md (multiple disks) volume requires the /boot/ directory and its contents on the same RAID volume as the root volume.

On many SuSE installations however, it is likely that the /boot/ directory is located in a small partition at the beginning of the disk.

I therefore tar'd the directory and its contents to my root volume and removed the original /boot directory from my /etc/fstab so it would no longer be mounted on reboot. There may occur boot problems if the root volume is not located within the first 1024 blocks of the disk.

Step 3. Partition /dev/hdb

I now portioned the second disk /dev/hdb exactly like the first one:
The only difference is that I chose the partition type fd (Linux Raid Autodetect)

linux:/etc # fdisk -ul

Disk /dev/hda: 255 heads, 63 sectors, 7476 cylinders
Units = sectors of 1 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 63 112454 56196 83 Linux
/dev/hda2 112455 120101939 59994742+ f Win95 Ext'd (LBA)
/dev/hda5 112518 10602899 5245191 83 Linux
/dev/hda6 10602963 12707414 1052226 82 Linux swap
/dev/hda7 12707478 120101939 53697231 83 Linux

Disk /dev/hdb: 255 heads, 63 sectors, 7476 cylinders
Units = sectors of 1 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdb1 63 112454 56196 83 Linux
/dev/hdb2 112455 120101939 59994742+ f Win95 Ext'd (LBA)
/dev/hdb5 112518 10602899 5245191 fd Linux raid autodetect
/dev/hdb6 10602963 12707414 1052226 82 Linux swap
/dev/hdb7 12707478 120101939 53697231 fd Linux raid autodetect

Step 4. Create the RAID control file /etc/raidtab

To create the RAID-Volumes I used the following configuration:

raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
persistent-superblock 1
device /dev/hda5
failed-disk 0
device /dev/hdb5
raid-disk 1
chunk-size 4

raiddev /dev/md1
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
persistent-superblock 1
device /dev/hda7
failed-disk 0
device /dev/hdb7
raid-disk 1
chunk-size 4

Then I copied the very same file on a floppy disk, because we need it later. Please note that failed-disk parameter. I had a complete and fully functional installation on my first disk and without the failed-disk parameter all my partitions on /dev/hda would have been deleted during creation of the RAID volume.

Step 5. Create the RAID-Volume

Now I rebooted the box and started the rescue system from the SuSE cdrom.

By default, no raid modules are loaded in the rescue system, so we load them first:

modprobe raid1

Once the module is loaded, copy the raidtab configuration file from the floppy disk to /etc/ and run:

mkraid /dev/md0
mkraid /dev/md1

If there was already an existing RAID configuration on the disk your may want to use mkraid's -force parameter.

To make the RAID available we have to start it first:

raidstart /dev/md0
raidstart /dev/md1

To check the status of your RAID volumes type
cat /proc/mdstat

Step 6. Create the filesystems

After mkraid has successfully created our RAID Volume in degraded mode on the second disk /dev/hdb, you can start creating the filesystems. You can use whatever filesystem you want. Personally I experienced once heavy file system corruption on my root volume when I used reiserfs.
I therefore decided to create the root volume with extfs2 and my data volume using reiserfs.

mkefs /dev/md0
mkreiserfs /dev/md1

Step 7. Copy your data

Now I mounted my former root volume /dev/hda5 and the new created /dev/md0 and tar'ed all my data to the raid volume. Do the same for all other volumes.

Step 8. Repartition first disk /dev/hda

Now it is getting serious:

Going any further may leave your system in an unusable state!!

I deleted my former partitions on the first disk /dev/hda and recreated them, but this time I changed the partition type to fd (Linux raid autodetect):

fdisk -ul now shows:

Disk /dev/hda: 255 heads, 63 sectors, 7476 cylinders
Units = sectors of 1 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 63 112454 56196 83 Linux
/dev/hda2 112455 120101939 59994742+ f Win95 Ext'd (LBA)
/dev/hda5 112518 10602899 5245191 fd Linux raid autodetect
/dev/hda6 10602963 12707414 1052226 82 Linux swap
/dev/hda7 12707478 120101939 53697231 fd Linux raid autodetect

Disk /dev/hdb: 255 heads, 63 sectors, 7476 cylinders
Units = sectors of 1 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hdb1 63 112454 56196 83 Linux
/dev/hdb2 112455 120101939 59994742+ f Win95 Ext'd (LBA)
/dev/hdb5 112518 10602899 5245191 fd Linux raid autodetect
/dev/hdb6 10602963 12707414 1052226 82 Linux swap
/dev/hdb7 12707478 120101939 53697231 fd Linux raid autodetect

Step 9. Add the first disk to the RAID

Now edit the /etc/raidtab and change all "failed-disk" entries to "raid-disk".

Now unmount all filesystems and type:

raidhotadd /dev/md0 /dev/hda5
raidhotadd /dev/md1 /dev/hda7

Now "cat /proc/mdstat" and you should see your RAID rebuilding.

Step 10. Adopt your lilo.conf

After your RAID-Rebuild completed successfully mount the root RAID volume and copy the raidtab control file to disk:

mount /dev/md0 /mnt

cp /etc/raidtab /mnt/etc

cd /mnt/etc

edit and DO NOT EXECUTE lilo the lilo-conf appropriate to your needs. Here is the example:

linux:~ # cat /mnt/etc/lilo.conf

install=/boot/boot-bmp.b
boot = /dev/md0
vga = 771
read-only
menu-scheme = Wg:kw:Wg:Wg
lba32
prompt
timeout = 80
raid-extra-boot = "/dev/hda,/dev/hdb"
#message = /boot/message

image = /boot/vmlinuz
initrd = /boot/initrd # automatically added by patch_lilo_conf
label = linux
root = /dev/md0

image = /boot/vmlinuz.suse
initrd = /boot/initrd.suse # automatically added by patch_lilo_conf
label = SuSE
root = /dev/md0
optional

This will put the mbr record on both disks as well as on your root RAID-volume:

Step 11. Create your initial ram disk

To create our initial ram disk properly your have to edit your rc.config file. Find the section INITRD_MODULES and

MAKE SURE IT CONTAINS ALL MODULES YOU NEED DURING STARTUP!

If your root RAID volume is on reiserfs make sure reiserfs module is in the configuration.

Here is my example:

INITRD_MODULES="usbcore raid0 raid1 raid5 reiserfs"

Now you can create your initial ram disk:
Where your initial-ram disk is located can be seen in your lilo.conf. If you like to backup them this is your last chance:

cd /mnt/sbin

./mk_initrd -d /dev/md0 /mnt

Step 12. Lilo and the RAID-Volume

Now we have to start lilo. CAUTION: On the rescue system there is still our old lilo installed. You must use our newly installed version:
cd /mnt/sbin
./lilo -v -C /mnt/etc/lilo.conf -r /mnt

Step 13. Don't forget your fstab

Now edit your fstab to make sure every file system is mounted properly after reboot

cd /mnt/etc
vi fstab

Here is the example:

/dev/md0 / ext2 defaults 1 1
devpts /dev/pts devpts defaults 0 0
/dev/cdrom /media/cdrom auto ro,noauto,user,exec 0 0
/dev/dvd /media/dvd auto ro,noauto,user,exec 0 0
/dev/fd0 /media/floppy auto noauto,user,sync 0 0
/dev/md1 /opt reiserfs defaults 1 2
proc /proc proc defaults 0 0
/dev/hda6 swap swap pri=42 0 0
/dev/hdb6 swap swap pri=42 0 0

Step 14. Prepare to reboot

Now unmount your RAID partition and reboot:

umount /mnt
raidstop /dev/md0
raidstop /dev/md1
reboot

Congrats! Youre finished.

Step XX. Some errormessage occured. What can I do?

I experienced many oddities during my experiments with raid:
If you get the message similar to this:
raidstart /dev/md0
-unknown option-
Your RAID is somehow broken. Recreate it with mkraid (see -force option).

I highly recommend during all work in your rescue system that you treat your filesystems with mucho care.

If you have to reboot

• make sure no disk partition is mounted any more
• that the RAID volumes are properly stopped (raidstop /dev/mdX)
• type "sync; sync" before type reboot to make sure all buffers are flushed properly.

If something is going wrong here is a little hint how to still get access to your data.

If for example rebuilding the RAID fails for some reason, or the command raidstart fails, you can still mount the disk partitions containing your original data by typing:

mount -o ro /dev/hda5

Though this partition is part of a RAID volume you can get access to the underlying data.
The reason for this is probably that the difference between a type fd (raid) and 83 (standard linux) volume is just the RAID superblock at the end of the partition. Therefore all data on those partitions can be mounted without having the RAID started.

I highly recommend mounting it read-only otherwise the RAID Information may not be consistent any more and you have to recreate that volume.

I experienced most problems during the RAID-Rebuild in the rescue-system. Many times the rebuild came to 99.9 percent with just 2 Bytes left and then synced forever. Especially on my large reiserfs volume this turned out to be very troublesome. I could not find the reason for this. If you have a hint please let me know.