Every few years, I have to add a new disk to a Debian system. Either an old hard drive has failed, or I've run out of space on the old disk and need a bigger one. Either way, the routine involves choosing partition sizes and filesystem parameters; and of course I've forgotten how to do all these things, because it's been too long since the last time. So here's how to do them.
First, a couple of excellent references. Wayne Pollock has written a fine discussion of the actual considerations for disk partitioning. Unlike most of the treatments you find on the Web, his avoids the “one size fits nobody” problem by explaining why different systems need different choices, instead of assuming there is a single magic answer to that problem.
Once you have chosen the sizes and mount points for your partitions, you have to make filesystems on them. But filesystems have a lot of adjustable parameters. The obvious tools like df and du ought to show what you're currently using; but they each report a different subset of the necessary information. I've found another good discussion of the tools required, which points out that you also need tune2fs -l — not the most obvious choice, but a very useful one, if you are using the standard ext[234] filesystems. But be careful: what df calls a “block” is 1 kiB, or (usually) 2 disk sectors; but what tune2fs calls a “block” is one of those 4 kiB filesystem blocks, or 8 of the 512-byte sectors.
Usually, the new disk is considerably larger than the old one. Western Digital seems to be improving the technology faster than I can add files to my filesystems. Furthermore, the file sizes keep growing larger with time: either I add bells and whistles to my own programs, or new ones appear in old file types (like PDF). The kernels, initrd.img, and System.map files keep growing, as more hardware types have to be managed. So the average file size steadily creeps up, which changes the balance between block sizes on disk, and number of inodes per partition, and so on.
That means that the set of parameters I chose the last time around is far from optimal. Years ago, I never kept hordes of PDF files, because they chewed up disk space too fast; these days, with disk sizes of many GB, it's not the problem it was in the days of 20 MB disks. So even if the new disk is the same size as the old one, some adjustments are needed for optimal use of the space.
These filesystem parameters need to be a bit different for different parts of the whole file tree. My average file size is about 80kB in /boot, but 47kB in /home, and only 38kB in /var. The default partitioning parameters would create a lot of useless inodes in /boot, if I didn't adjust things.
Frequently-used partitions should be grouped together, so the heads don't have to do big seeks between tracks. My single-user desktop machine rarely uses swap space, so swap can go near the end of the disk. I usually distribute my partitions like this:
Partition Mount
number point
---------- ---------
1 /boot
2 /root
3 /tmp
4 [extended]
5 /home
6 /var
7 /usr/local
8 [swap]
9 /spare
Finally, although disks have gotten faster, the first few tracks (at the edge of the disk) transfer data appreciably faster than the last ones (near the center of the disk). I never used to take this into account when planning the arrangement of partitions; but it's worth thinking about.
Use df and df -i to find typical file size
and inode usage for existing filesystems.
Draw up the planned
partition sizes and ordering (see step 7).
Remark: although df displays a partition-size column headed “1K-blocks”, it is slightly smaller than the actual size of the disk partition. The excluded material includes the partition's boot record, and other housekeeping items, such as the journal of an ext4 partition. The difference is less than 1%, and can be neglected in planning the size of a new partition.
Note that the “Used” columns of df and df -i output for the existing disk's partitions can be used to make good estimates of the Bytes-to-Inodes ratio, which is useful in making the filesystems on the new disk.
These wide variations present a problem in choosing the logical block size for a /var partition. If you choose 1024 bytes, the big files will be stored inefficiently; but if you choose 4096 B, each file will waste half a block, or 2048 B, on the average.
Furthermore, lots of little blocks mean slow file loading when lots of /var files are in use — as happens when you bring up a browser. If you're not using the X Window system and Firefox, you only rarely need files from /var. If you start up X and Firefox, you'll suddenly be opening thousands of their files that live in /var.
If you have plenty of disk space, and are willing to sacrifice a few GB of it for faster operation, allocate 12 to 15 GB of space for /var, and choose a 4096 B block size. If you're pressed for space, use smaller blocks.
Finally, the wide disparity in file sizes here means that you can skimp a bit on the number of inodes in this filesystem. I tried 16384 bytes/inode and found I had wasted lots of inode space. Probably 32768 is a reasonable choice; you might even find that 65536 will provide enough inodes. [Notice that the 256-byte inodes of an ext4 filesystem are already less than 1% of the whole partition, if you set the bytes-per-inode parameter to 32768.]
The standard argument against using filesystem labels is that they are not unique enough: if you have several different root filesystems for different operating systems, you can't call them all “root”. But you can generate persistent names for filesystems if you name all the ones on a given disk with a distinctive prefix: all the filesystems on Disk A get labels like Aboot and Aroot and Ahome, and all those on Disk B can be Bboot and Broot and so on. Such names are more user-friendly than the gibberish of UUIDs, and are much less likely to be garbled by typos in re-creating partitions.
Another set of unique names is found in /dev/disk/by-id/, in which the udev daemon puts soft links to the traditional device files like /dev/sda and so on. These traditional designations may not persist from one booting to another. For example, when I added a third disk to my Linux box, the new disk appeared as /dev/sdb, and the one that used to be sdb was changed to /dev/sdc. The names in /dev/disk/by-id/ are very handy, as they usually contain both the model and serial number of each disk — information that's visible on the disk drive itself. But these useful names are too big to fit in the 16-byte space available for labels in filesystem headers. Although they are too long for filesystem labels, they work nicely in the /boot/grub/device.map file.
Because of all these alternative designations for partitions, you should write out a cross-index of the traditional names (like /dev/sda1), the UUIDs, the 16-byte labels, and the mount points of existing partitions; and keep it updated as you create new partitions.
Make sure you have paper copies of essential man pages, in case you accidentally clobber a partition needed for normal use. The main partition-manipulators are fdisk, cfdisk (a curses-based derivative with a nicer user interface), and parted. Some other partitioners are less capable; but fdisk is more powerful than cfdisk.
Use man -t xxx | lpr -Pduplex
to print copies of man pages for:
fdisk , cfdisk and/or parted
mkfs.ext4
mkswap
and maybe
tune2fs and/or dumpe2fs
More detailed information is available about each of the partitioning programs in its “info” file. To print the “info” for parted, do:
info parted | col -b | lpr -Pduplex
— but be aware that this produces more than an order of magnitude more output than the simple man page does. However, you may need the details of the various kinds of flags, labels, and/or units.
If the disk you are replacing is the one you normally boot from, you should also print out documents related to booting:
man -t boot | lpr -Pduplex
and maybe my Web page on using the grub2 “rescue shell”.
The Grub manual is too big (and too incomprehensible!) to print; but there are some useful Web pages at the Ubuntu Community Help Wiki that are worth printing:
https://help.ubuntu.com/community/Grub2 and
https://help.ubuntu.com/community/Grub2/Troubleshooting
An especially useful guide to recovering from Grub disasters is at https://www.linux.com/learn/tutorials/776643-how-to-rescue-a-non-booting-grub-2-on-linux/, which explains not only how to use the normal Grub menu and shell, but also how to use the peculiar “grub rescue” shell, with its very limited set of commands. Print out the first 6 pages of this to have in case of problems. (You might also like to have a copy of my Web page on using the grub2 “rescue shell”.)
Paper copies of these documents will be necessary if you do something to make the system unbootable during the transition. Be prepared. (I use my duplex printer to save paper.)
NOTE: If replacing both a working and a backup disk to gain space, replace the backup disk first.
Back up ALL old partitions.
Edit /etc/fstab to comment out all references to swap partitions on the disk to be replaced.
Physically install the new disk:
a) Shutdown and turn off AT power supply (or unplug box).
b) Remove connectors from old disk.
c) Remove screws from both sides.
d) Swap disk drives.
e) Replace screws and tighten.
f) Replace connectors.
Manually partition the new disk as planned in step 1.
Note 1: Use the m command
in cfdisk, or
the -align min
option in parted,
to maximize space and avoid lost sectors after the boot sector.
Note 2: When selecting partition number 5 (after 3, in cfdisk), remember to make it as "Logical" and not "Primary".
Note 3: Remember that Primary partitions can follow one another
sequentially; each can begin in the next
sector after the last sector of the previous partition.
But Logical partitions must be preceded by a 1-sector
Extended Boot Record ("EBR"); so each Logical partition
begins at the second sector after the end of the
preceding Logical partition. If you use the minimum-spacing
alignment option of parted, you'll need to add 2
sectors to the last sector in the preceding partition, to
get the first sector of the following logical partition.
Note 4: Don't forget to mark the swap partitions!
One reason to make separate partitions for different parts of the whole system of files is to optimize the storage space for size and speed. In the examples below, I use /dev/sdX as the name of the new disk, and /dev/sdXY as the name of its Yth partition; the values of Y agree with the partition ordering described above.
Remember to change the letters X and Y in the lines below with the appropriate values for your system.
Create filesystems on the new disk:
a) mkfs.ext3 -N 448 -b 4096 -c -c
-L Xboot /dev/sdX1
# in grub-legacy
or
mkfs.ext4 -N 448 -I 128 -b 4096
-c -c -L Xboot /dev/sdX1
# in grub2
Note: 448 inodes suffice for the /boot partition. (grub-legacy can only read ext3 ; use small inodes for ext4 in grub2.)
b) mkfs.ext4 -i 8192 -b 4096 -c -m 2 -L Xroot /dev/sdX2
Note: 8192 bytes/inode and 2% reserved, for root partition
c) mkfs.ext4 -i 32768 -b 4096 -m 1 -L Xhome /dev/sdX5
Note: 32758 bytes/inode for /var; 65536 B/inode for /home
32768 bytes/inode and 1% reserved, for all others
Possible special values for /var: 65536 bytes/inode, and 2% or 3% reserved for root, because of log files.
Prepare swap partition: mkswap -c -L swap /dev/sdX8
If the new partitioning scheme differs from the old one, use manual copying instead of a backup script to copy files:
a) e2fsck -f /dev/sdXY # check new fs
b) mount /dev/sdXY /mnt
c) cd oldmountpoint # necessary
to get correct dirnames
d) cp -a -f -x --sparse=always . /mnt
# actual copy
e) find . -xdev -type f -exec cmp \{\} /mnt/\{\} \;
# verify
f) cd -
# get off old mount point
g) umount /dev/sdXY # unmount for fsck
h) e2fsck -f /dev/sdXY # final check
Repeat this operation for each new filesystem.
Back in step 4, we made some changes to /etc/fstab on the old disk. Now we need to reverse those changes in the copy on the new disk.
We also need to change the names of partitions in the new fstab to those on the new disk. And, if you upgraded your new boot partition from ext3 to ext4, don't forget to say so in the new disk's /etc/fstab .
Before you can edit the new fstab, you need to mount the new root partition:
a) mount /dev/sdX2 /mnt
b) vi /mnt/etc/fstab
c) umount /dev/sdX2
Some other editing is needed to prepare the new disk for use. Remember to umount the new root partition before you mount the new boot partition.
The first step is to make Grub aware of the new disk. That means adding it to the /boot/grub/device.map file on both the old and new disks. (See the paragraph on naming disks by ID above.)
To edit the device.map file on the new disk, mount its boot partition on /mnt and edit /mnt/grub/device.map — remembering to umount /mnt when you are done.
Next, tell Grub to add a line for the new disk to the boot-time menu. In grub-legacy, the file to edit was menu.lst; in grub2, it's files in the /etc/grub.d directory. Again, these changes should be made on both old and new disks.
Append a new menuentry stanza for the new disk to the 40_custom file in /etc/grub.d . You can just copy the whole menuentry stanza from the default section of /boot/grub/grub.cfg that is generated by the 10_linux script, and modify a few lines. Look in /boot/grub/grub.cfg for the first occurrence of “menuentry”; it should immediately follow the line
### BEGIN /etc/grub.d/10_linux ### .
The stanza ends with a line containing a single right brace. Copy the whole stanza to the end of the 40_custom file, and change its references to partitions on the old disk to the corresponding partitions of the new disk. (Be sure the new root partition is mounted on /mnt so you can edit its 40_custom file.)
If the new disk's root partition is mounted on /mnt, you can just do
vi /boot/grub/grub.cfg /etc/grub.d/40_custom /mnt/etc/grub.d/40_custom
to edit these files. Copy the working default stanza from grub.cfg to the end of each 40_custom file, and change the 3 lines that refer to partitions on the old disk to refer to the corresponding partitions on the new disk. In my case, these lines are:
set root='(hd0,msdos1)'
(which refers to the boot [not root!] partition on the old disk, using Grub's strange notation — here you only need to change hd0 to the proper designation for the new disk, as it appears in device.map );
search —no-floppy —fs-uuid —set=root d074378b…
(where the gibberish at the end of the line is in fact the UUID of that same old boot partition, which you must change to the UUID of the boot partition on the new disk); and
linux /vmlinuz-3.2.0-4-686-pae root=UUID=0899983c…
(where the UUID is that of the root partition on the old disk, which should be changed to that of the root partition on the new disk.)
As I explained earlier, I find UUIDs so inconvenient that I actually replace the --fs-uuid in the “search” line with --label, and the bare UUID of the new boot partition with its filesystem label. Likewise, I replace the root=UUID=… in the linux line with root=LABEL= and the label of the new root partition.
Then you can
umount /mnt
to free the mount point.
Now, when you run update-grub, it will create a boot-menu stanza for the new disk in Grub's configuration file, /boot/grub/grub.cfg . By adding all references to the new disk to the 40_custom file in /etc/grub.d/, you have left your normal booting procedure undisturbed.
Unfortunately, this may not do what you want, because it may run the kernel stored on the new disk without the correct command-line arguments. If you copied the working menuentry for the old disk from its old /boot/grub/grub.cfg file to the 40_custom file on the old (working) disk, and changed only the partition references to refer to the new disk, you should be OK. But if you only changed the 40_custom file on the new disk before running update-grub, you won't get the new menu-item when you run update-grub. Or, if you accidentally lost the linux command-line arguments in the editing, the Grub menu item for the new disk will omit the argument video=640x480; when you reboot in this condition, the text screen that shows the progress of the booting is all in microscopic print that's nearly impossible to read.
So, before you reboot, check the new /boot/grub/grub.cfg carefully to make sure it's correct. An easy way to do this is to write the new grub.cfg file to /tmp, instead of over-writing the working file at /boot/grub/grub.cfg:
grub-mkconfig > /tmp/grub.cfg
Then you can inspect this dry-run copy at leisure, without worrying. As a double check, you can do
cmp /boot/grub/grub.cfg /tmp/grub.cfg
and see where the old and new versions differ: if it's after line 90, the first (default) menuentry stanza is undisturbed, and you can safely reboot into your current version on Linux, after updating the real /boot/grub/grub.cfg .
Now, Grub should be able to boot the kernel copied to the new disk. Do:
a) run update-grub to make a new
/boot/grub/grub.cfg
and
run grub-install /dev/sdX to make it bootable
b) new /boot/grub/grub.cfg to make the *other* disk bootable
c) add "ext4" module to /etc/initramfs-tools/modules when upgrading from ext3
d) new /etc/fstab to mount new-disk partitions & swap space (Don't forget to change ext3 to ext4 when upgrading from ext3.)
e) fix ./backup and ./fstab_fix as needed
f) chmod 1777 /tmp # on the new disk, /tmp MOUNTED on new /
(umount the new root partition when done)
Shut down and try to boot from the new disk.
a) Hit DEL to get into SETUP, go to the Boot menu, and move the new disk into position to boot.
b) If you changed the number of the root partition, you may need to invoke "root=/dev/sdxx" on the kernel cmd. line at the boot: prompt.
Verify that all file systems mounted properly. Check browser; networking & IP address; printing; access to user files; etc.
Repeat as needed for the other disk.
Useful checks: hdparm -I /dev/hdx # to show all disk info
hdparm -t -T # to show transfer rates
Even a 1-line file uses a whole block of this kind. Every file wastes about half a block, on the average. So storage space is usually conserved by using 1024-byte blocks.
On the other hand, the kernel has to send out twice as many orders for 1024-byte blocks as for 2048-byte blocks; so the overhead diminishes with the block size. Furthermore, if the filesystem contains some very large files, they will require more indirect blocks in their inodes to refer to all the blocks in the file. This increases the overhead still more, and also decreases the conservation of disk space achieved by using small blocks.
In choosing a block size, remember that empty files contain no disk blocks at all; so they are unaffected by the block size. So, in computing an average or a median file size, empty files should be excluded.
See the special discussion of block size for /var above.
These days, partitions are several (or even many!) GB in size, and 5% is a lot of blocks. There's no need to set aside more than a few hundred MB at most. Generally, leaving space for a few of the largest files on the filesystem is enough for root to restore normal operations.
Another function of the reserved space is to leave room for the kernel to assign contiguous blocks to new files. The ext4 file system is very good at this, but it needs some empty space to play with; the reserved space, which is not in any fixed location, provides the needed slack. Here, too, space a few times larger than the largest files expected in the partition should be enough. Notice that the reserved space is not counted by df when it reports the number of free blocks; so users and administrators will have adequate warning when a partition is getting too full. Fragmentation usually becomes a problem when the partition is 90% or 95% full; so the 5% nominal value is a safe rule of thumb.
However, reserved space isn't really needed in some cases. A small partition mounted on /boot is rarely written to; and when it is, root is the one writing to it anyway. So why care about reserved blocks? And /home could fill up completely without causing serious problems, especially in a desktop machine with a single user. Besides, it could be a huge partition — 100 GB or more these days — and 5% of that would be 5GB, which is vastly more than needs to be reserved.
Even 1% of 100 GB is 1 GB, which is a lot of space that could be useful. So even -m 1 would reserve too much space. Fortunately, the argument to the -m option need not be an integer: we can tell mkfs.ext4 to save only 0.1% in this situation.
Copyright © 2015 – 2018, 2021 Andrew T. Young
or the
alphabetic index page
or the
GF home page
or the website overview page