August 29, 2008

Format ext3 for Many Large Files on RAID

We've already covered how to format a filesystem for many small files. Now we will look at how to format a filesystem for many LARGE files.

Say we have a very big device, as the Linux kernel puts it. For this example, we will assume that it is a 7TB RAID enclosure of direct-attach storage on /dev/sdb. We have already done critical firmware updates. And we have initialized the array for RAID 6 with hot spares, through the controller bios, as one big virtual disk.

First initialize the device as an LVM physical volume, accounting for 128k stripe alignment of our RAID array:
# pvcreate --metadatasize 250k /dev/sdb
# pvs -o pe_start


Create an LVM volume group for the RAID array, and then create the LVM logical volume, using 100% of the volume group:
# vgcreate RaidVolGroup00 /dev/sdb
# lvcreate --extents 100%VG --name \
RaidLogVol00 RaidVolGroup00


Now format the logical volume ext3:
# mkfs -t ext3 -E stride=32 -m 1 \
-O dir_index,filetype,has_journal,sparse_super \
/dev/RaidVolGroup00/RaidLogVol00


I've been told that uping the read_ahead queue can increase performance. YMMV
# echo 1024 >/sys/block/sdb/queue/read_ahead_kb

2 comments:

swiftg said...

Could you please explain how you calculated the metadatasize of 250k to accommodate the 128k striping of the RAID array?

Gavin said...

To quote Ted Tso, "Why 250k and not 256k? I can’t tell you — sometimes the LVM tools aren’t terribly intuitive." You can verify with "# pvs -o pe_start"