Partitioning the New Disk

Introduction

Every few years, I have to add a new disk to a Debian system. Either an old hard drive has failed, or I've run out of space on the old disk and need a bigger one. Either way, the routine involves choosing partition sizes and filesystem parameters; and of course I've forgotten how to do all these things, because it's been too long since the last time. So here's how to do them.

First, a couple of excellent references. Wayne Pollock has written a fine discussion of the actual considerations for disk partitioning. Unlike most of the treatments you find on the Web, his avoids the “one size fits nobody” problem by explaining why different systems need different choices, instead of assuming there is a single magic answer to that problem.

Once you have chosen the sizes and mount points for your partitions, you have to make filesystems on them. But filesystems have a lot of adjustable parameters. The obvious tools like df and du ought to show what you're currently using; but they each report a different subset of the necessary information. I've found another good discussion of the tools required, which points out that you also need tune2fs -l — not the most obvious choice, but a very useful one, if you are using the standard ext[234] filesystems. But be careful: what df calls a “block” is 1 kiB, or (usually) 2 disk sectors; but what tune2fs calls a “block” is one of those 4 kiB filesystem blocks, or 8 of the 512-byte sectors.

General considerations

Usually, the new disk is considerably larger than the old one. Western Digital seems to be improving the technology faster than I can add files to my filesystems. Furthermore, the file sizes keep growing larger with time: either I add bells and whistles to my own programs, or new ones appear in old file types (like PDF). The kernels keep growing, as more hardware types have to be managed. So the average file size steadily creeps up, which changes the balance between block sizes on disk, and number of inodes per partition, and so on.

That means that the set of parameters I chose the last time around is far from optimal. Years ago, I never kept hordes of PDF files, because they chewed up disk space too fast; these days, with disk sizes of many GB, it's not the problem it was in the days of 20 MB disks. So even if the new disk is the same size as the old one, some adjustments are needed for optimal use of the space.

These filesystem parameters need to be a bit different for different parts of the whole file tree. My average file size is about 93kB in /boot, but 64kB in /home, and only 16kB in /var. The default partitioning parameters would create a lot of useless inodes in /boot, if I didn't adjust things.

Frequently-used partitions should be grouped together, so the heads don't have to do big seeks between tracks. My single-user desktop machine rarely uses swap space, so that can go near the end of the disk. I usually distribute my partitions like this:

Partition     Mount
number       point
----------     ---------
      1            /boot
      2            /root
      3            /tmp
      4         [extended]
      5            /home
      6            /var
      7            /usr/local
      8            [swap]
      9            /spare

Finally, although disks have gotten faster, the first few tracks (at the edge of the disk) transfer data appreciably faster than the last ones (near the center of the disk). I never used to take this into account when planning the arrangement of partitions; but it's worth thinking about.

Strategy

We can divide the whole project into a planning stage and an execution stage:

Planning

1) Partition layout

Use df and df -i to find typical file size and inode usage for existing filesystems.
Draw up the planned partition sizes and ordering (see step 7).

Remark: although df displays a partition-size column headed “1K-blocks”, it is slightly smaller than the actual size of the disk partition. The excluded material includes the partition's boot record, and other housekeeping items, such as the journal of an ext4 partition. The difference is less than 1%, and can be neglected in planning the size of a new partition.

Note that the “Used” columns of df and df -i output for the existing disk's partitions can be used to make good estimates of the Bytes-to-Inodes ratio, which is useful in making the filesystems on the new disk.

Special considerations for /var
If you put /var in a separate partition, you can run into problems when it's time to upgrade the operating system. This filesystem is indeed quite variable; it contains both many small files and some very large ones. In particular, the directory /var/cache/apt/archives holds all the downloaded packages — of which there might be thousands in a major upgrade. A typical *.deb package file these days is about 2 MB. That means that a full system upgrade might need several GB more space in /var than you need for routine use. What seems like a reasonable amount of space during normal operations can be painfully inadequate at upgrade time.

These wide variations present a problem in choosing the logical block size for a /var partition. If you choose 1024 bytes, the big files will be stored inefficiently; but if you choose 4096 B, each file will waste half a block, or 2048 B, on the average.

Furthermore, lots of little blocks mean slow file loading when lots of /var files are in use — as happens when you bring up a browser. If you're not using the X Window system and Firefox, you only rarely need files from /var. If you start up X and Firefox, you'll suddenly be opening thousands of their files that live in /var.

If you have plenty of disk space, and are willing to sacrifice a few GB of it for faster operation, allocate 12 to 15 GB of space for /var, and choose a 4096 B block size. If you're pressed for space, use smaller blocks.

Finally, the wide disparity in file sizes here means that you can skimp a bit on the number of inodes in this filesystem. I tried 16384 bytes/inode and found I had wasted lots of inode space. Probably 32768 is a reasonable choice; you may even find that 65536 will provide enough inodes.

Partition names

One more thing requires planning: filesystem labels. Some people like to use UUIDs as unique names for partitions; but if you ever have to re-partition a disk, to make room for some unexpectedly growing filesystem, you'll find that the UUIDs get changed when you re-make partitions. That can propagate into errors in vital files like /etc/fstab, and boot-configuration files.

The standard argument against using filesystem labels is that they are not unique enough: if you have several different root filesystems for different operating systems, you can't call them all “root”. But you can generate persistent names for filesystems if you name all the ones on a given disk with a distinctive prefix: all the filesystems on Disk A get labels like Aboot and Aroot and Ahome, and all those on Disk B can be Bboot and Broot and so on. Such names are more user-friendly than the gibberish of UUIDs, and are much less likely to be garbled by typos in re-creating partitions.

Another set of unique names is found in /dev/disk/by-id/, in which the udev daemon puts soft links to the traditional device files like /dev/sda and so on. These traditional designations may not persist from one booting to another. For example, when I added a third disk to my Linux box, the new disk appeared as /dev/sdb, and the one that used to be sdb was changed to /dev/sdc. The names in /dev/disk/by-id/ are very handy, as they usually contain both the model and serial number of each disk — information that's visible on the disk drive itself. But these useful names are too big to fit in the 16-byte space available for labels in filesystem headers. Although they are too long for filesystem labels, they work nicely in the /boot/grub/device.map file.

2) Print  man  pages

Make sure you have paper copies of essential man pages, in case you accidentally clobber a partition needed for normal use.

Use   man -t xxx | lpr -Pduplex   to print copies of man pages for:

cfdisk  and/or  parted     NOTE: other partitioners are less capable.
mkfs.ext4
mkswap

If the disk you are replacing is the one you normally boot from, you should also print out documents related to booting:

man -t boot | lpr -Pduplex

The Grub manual is too big (and too incomprehensible!) to print; but there are some useful Web pages at the Ubuntu Community Help Wiki that are worth printing:

https://help.ubuntu.com/community/Grub2   and
https://help.ubuntu.com/community/Grub2/Troubleshooting

An especially useful guide to recovering from Grub disasters is at https://www.linux.com/learn/tutorials/776643-how-to-rescue-a-non-booting-grub-2-on-linux/, which explains not only how to use the normal Grub menu and shell, but also how to use the peculiar “grub rescue” shell, with its very limited set of commands. Print out the first 6 pages of this to have in case of problems.

Paper copies of these documents will be necessary if you do something to make the system unbootable during the transition. Be prepared. (I use my duplex printer to save paper.)

NOTE: If replacing both a working and a backup disk to gain space, replace the backup disk first.

Execution

3) Back up files

Back up ALL old partitions.

4) Edit /etc/fstab

Edit /etc/fstab to comment out all references to swap partitions on the disk to be replaced.

5) Install new disk

Physically install the new disk:

a)  Shutdown and turn off AT power supply (or unplug box).
b)  Remove connectors from old disk.
c)  Remove screws from both sides.
d)  Swap disk drives.
e)  Replace screws and tighten.
f)  Replace connectors.

6) Partition new disk

Manually partition the new disk as planned in step 1.

Note 1: Use the    m     command in   cfdisk, or
          the   -align min   option in  parted,
        to maximize space and avoid lost sectors after the boot sector.

Note 2: When selecting partition number 5 (after 3, in cfdisk), remember to make it as "Logical" and not "Primary".

Note 3: Don't forget to mark the swap partitions!

7) Create filesystems

One reason to make separate partitions for different parts of the whole system of files is to optimize the storage space for size and speed. In the examples below, I use /dev/sdX as the name of the new disk, and /dev/sdXY as the name of its Yth partition; the values of Y agree with the partition ordering described above.

Create filesystems on the new disk:

a)   mkfs.ext3  -N 448  -b 4096  -c -c  -L Xboot  /dev/sdX1         # in grub-legacy
                or
      mkfs.ext4  -N 448  -I 128  -b 4096  -c -c  -L Xboot  /dev/sdX1     # in grub2

Note: 448 inodes suffice for the /boot partition. (grub-legacy can only read ext3 ; use small inodes for ext4 in grub2.)

b)   mkfs.ext4  -i 8192  -b 4096  -c -m 2  -L Xroot  /dev/sdX2

Note: 8192 bytes/inode and 2% reserved, for root partition

c)   mkfs.ext4  -i 32768  -b 4096  -m 1  -L Xhome /dev/sdX5

Note: 32758 bytes/inode for /var; 65536 B/inode for /home

32768 bytes/inode and 1% reserved, for all others

Possible special values for /var: 65536 bytes/inode, and 2% or 3% reserved for root, because of log files.

8) Make swap partition on new disk

Prepare swap partition:   mkswap -c -L swap /dev/sdX8

9) Copy files to new disk

If the new partitioning scheme differs from the old one, use manual copying instead of a backup script to copy files:

a)   e2fsck -f /dev/sdXY     # check new fs
b)   mount /dev/sdXY /mnt
c)   cd oldmountpoint       # necessary to get correct dirnames
d)   cp -a -f -x --sparse=always . /mnt         # actual copy
e)   find . -xdev -type f -exec cmp \{\} /mnt/\{\} \;     # verify
f)   cd -        # get off old mount point
g)   umount /dev/sdXY     # unmount for fsck
h)   e2fsck -f /dev/sdXY     # final check

Repeat this operation for each new filesystem.

10) Edit some of the copied files

Back in step 4, we made some changes to /etc/fstab on the old disk. Now we need to reverse those changes in the copy on the new disk.

We also need to change the names of partitions in the new  fstab  to those on the new disk. And, if you upgraded your new boot partition from ext3 to ext4, don't forget to say so in the new disk's /etc/fstab .

Before you can edit the new  fstab, you need to mount the new root partition:

a)  mount /dev/sdX2 /mnt
b)  vi /mnt/etc/fstab

Some other editing is needed to prepare the new disk for use. Remember to umount the new root partition before you mount the new boot partition.

11) Make new disk bootable

          Edit device.map

The first step is to make Grub aware of the new disk. That means adding it to the /boot/grub/device.map file on both the old and new disks. (See the paragraph on naming disks by ID above.)

To edit the device.map file on the new disk, mount its boot partition on /mnt and edit /mnt/grub/device.map — remembering to umount /mnt  when you are done.

          Fix Grub's configuration

Next, tell Grub to add a line for the new disk to the boot-time menu. In grub-legacy, the file to edit was  menu.lst; in grub2, it's files in the  /etc/grub.d  directory. Again, these changes should be made on both old and new disks.

Append a new  menuentry  stanza for the new disk to the 40_custom  file in /etc/grub.d . You can just copy the whole  menuentry  stanza from the default section of /boot/grub/grub.cfg that is generated by the 10_linux  script, and modify a few lines. Look in /boot/grub/grub.cfg for the first occurrence of “menuentry”; it should immediately follow the line

### BEGIN /etc/grub.d/10_linux ### .

The stanza ends with a line containing a single right brace. Copy the whole stanza to the end of the 40_custom  file, and change its references to partitions on the old disk to the corresponding partitions of the new disk. (Be sure the new root partition is mounted on /mnt so you can edit its 40_custom  file.)

If the new disk's root partition is mounted on /mnt, you can just do

vi /boot/grub/grub.cfg /etc/grub.d/40_custom /mnt/etc/grub.d/40_custom

to edit these files. Copy the working default stanza from grub.cfg to the end of each 40_custom file, and change the 3 lines that refer to partitions on the old disk to refer to the corresponding partitions on the new disk. In my case, these lines are:

set root='(hd0,msdos1)'

(which refers to the boot [not root!] partition on the old disk, using Grub's strange notation — here you only need to change hd0 to the proper designation for the new disk, as it appears in device.map );

search —no-floppy —fs-uuid —set=root d074378b

(where the gibberish at the end of the line is in fact the UUID of that same old boot partition, which you must change to the UUID of the boot partition on the new disk); and

linux   /vmlinuz-3.2.0-4-686-pae root=UUID=0899983c

(where the UUID is that of the root partition on the old disk, which should be changed to that of the root partition on the new disk.)

As I explained earlier, I find UUIDs so inconvenient that I actually replace the --fs-uuid in the “search” line with --label, and the bare UUID of the new boot partition with its filesystem label. Likewise, I replace the root=UUID=… in the linux line with root=LABEL= and the label of the new root partition.

Then you can

umount /mnt

to free the mount point.

Now, when you run  update-grub, it will create a boot-menu stanza for the new disk in Grub's configuration file,  /boot/grub/grub.cfg . By adding all references to the new disk to the 40_custom file in /etc/grub.d/, you have left your normal booting procedure undisturbed.

Unfortunately, this may not do what you want, because it may run the kernel stored on the new disk without the correct command-line arguments. If you copied the working menuentry for the old disk from its old /boot/grub/grub.cfg file to the 40_custom file on the old (working) disk, and changed only the partition references to refer to the new disk, you should be OK. But if you only changed the 40_custom file on the new disk before running  update-grub, you won't get the new menu-item when you run update-grub. Or, if you accidentally lost the linux command-line arguments in the editing, the Grub menu item for the new disk will omit the argument video=640x480; when you reboot in this condition, the text screen that shows the progress of the booting is all in microscopic print that's nearly impossible to read.

          Check the results

So, before you reboot, check the new /boot/grub/grub.cfg carefully to make sure it's correct. An easy way to do this is to write the new  grub.cfg file to /tmp, instead of over-writing the working file at /boot/grub/grub.cfg:

grub-mkconfig > /tmp/grub.cfg

Then you can inspect this dry-run copy at leisure, without worrying. As a double check, you can do

cmp /boot/grub/grub.cfg /tmp/grub.cfg

and see where the old and new versions differ: if it's after line 90, the first (default) menuentry stanza is undisturbed, and you can safely reboot into your current version on Linux, after updating the real /boot/grub/grub.cfg .

Now, Grub should be able to boot the kernel copied to the new disk. Do:

a)   run   update-grub   to make a new   /boot/grub/grub.cfg
            and
      run   grub-install /dev/sdX   to make it bootable

b)   new /boot/grub/grub.cfg to make the *other* disk bootable

c)   add "ext4" module to /etc/initramfs-tools/modules when upgrading from ext3

d)   new /etc/fstab to mount new-disk partitions & swap space (Don't forget to change ext3 to ext4 when upgrading from ext3.)

e)   fix ./backup and ./fstab_fix as needed

f)   chmod 1777 /tmp # on the new disk, /tmp MOUNTED on new /

(umount the new root partition when done)

12) Reboot

Shut down and try to boot from the new disk.

a) Hit DEL to get into SETUP, go to the Boot menu, and move the new disk into position to boot.

b) If you changed the number of the root partition, you may need to invoke "root=/dev/sdxx" on the kernel cmd. line at the boot: prompt.

13) Check for success

Verify that all file systems mounted properly. Check browser; networking & IP address; printing; access to user files; etc.

14) Repeat for second new disk

Repeat as needed for the other disk.

Useful checks: hdparm -I /dev/hdx     # to show all disk info

hdparm -t -T       # to show transfer rates

Special considerations

Some parts of this process require special attention. For example:

Filesystem parameters

It can be useful to pick particular values for the parameters used to build a filesystem. For the usual ext4 filesystem, the defaults in /etc/mke2fs.conf are reasonable; but sometimes we can do better.

Block size

What mkfs.ext4 calls a “block” is not the 512-byte sector of a hardware disk, nor the 4096-byte “block” used by the software in reading from and writing to the disk, nor even the 1024-byte buffer size used by the kernel. Instead, it's the size of the chunks that files are logically divided into in a filesystem. The only sizes permitted are 1024, 2048, and 4096 bytes.

Even a 1-line file uses a whole block of this kind. Every file wastes about half a block, on the average. So storage space is usually conserved by using 1024-byte blocks.

On the other hand, the kernel has to send out twice as many orders for 1024-byte blocks as for 2048-byte blocks; so the overhead diminishes with the block size. Furthermore, if the filesystem contains some very large files, they will require more indirect blocks in their inodes to refer to all the blocks in the file. This increases the overhead still more, and also decreases the conservation of disk space achieved by using small blocks.

In choosing a block size, remember that empty files contain no disk blocks at all; so they are unaffected by the block size. So, in computing an average or a median file size, empty files should be excluded.

See the special discussion of block size for /var above.

Reserved blocks

When mkfs.ext4 creates a filesystem on a partition, it sets aside 5% of the blocks for root's use, in case some user fills up all the other blocks (usually, with some runaway process). This was necessary in the days of small disks and still smaller partitions. The superuser could go in and restore any vital files that had been clobbered, and remove some excessive garbage, and the system could run normally again.

These days, partitions are several (or even many!) GB in size, and 5% is a lot of blocks. There's no need to set aside more than a few hundred MB at most. Generally, leaving space for a few of the largest files on the filesystem is enough for root to restore normal operations.

Another function of the reserved space is to leave room for the kernel to assign contiguous blocks to new files. The ext4 file system is very good at this, but it needs some empty space to play with; the reserved space, which is not in any fixed location, provides the needed slack. Here, too, space a few times larger than the largest files expected in the partition should be enough. Notice that the reserved space is not counted by df when it reports the number of free blocks; so users and administrators will have adequate warning when a partition is getting too full. Fragmentation usually becomes a problem when the partition is 90% or 95% full; so the 5% nominal value is a safe rule of thumb.

However, reserved space isn't really needed in some cases. A small partition mounted on /boot is rarely written to; and when it is, root is the one writing to it anyway. So why care about reserved blocks? And /home could fill up completely without causing serious problems, especially in a desktop machine with a single user. Besides, it could be a huge partition — 100 GB or more these days — and 5% of that would be 5GB, which is vastly more than needs to be reserved.

Even 1% of 100 GB is 1 GB, which is a lot of space that could be useful. So even -m 1 would reserve too much space. Fortunately, the argument to the -m option need not be an integer: we can tell mkfs.ext4 to save only 0.1% in this situation.

 

Copyright © 2015 – 2017 Andrew T. Young


Back to the . . .
main LaTeX page

or the alphabetic index page

or the GF home page

or the website overview page