Occasionally I come across a damaged filesystem due to an unclean shutdown. Most are easy to fix with a live distribution.
First step is to boot the box with a live distro: any with the tools mentioned below will do. Open a terminal, become root (most commands in this article must be done as root) and see if your live distro automatically mounted the hard disk. In one case, I saw:
# df -T Filesystem Type Size Used Avail Use% Mounted on none unionfs 2.9G 266M 2.7G 9% / /dev/sdb1 xfs 7.8G 4.5G 3.4G 57% /media/hd6 /dev/sdb6 xfs 142G 103G 40G 73% /media/hd7
Here the root partition on the hard disk, /dev/sdb1, is mounted as /media/hd6. So we unmount it, using the mount point, not the device name: umount /media/hd6. Notice also that the filesystem is given; in the example above, the root partition is XFS.
Of course, the mount points reported by df -T are what your live distro just set up. If you’re not sure what partition was mounted where in the system you’re trying to fix, look for clues in the latest backup of /etc/fstab.
With this information we can try to repair the damage, using the device name. How this is done depends what the filesystem is:
e2fsck /dev/sdb1, replacing sdb1 with your device name. For journaled filesystems, e2fsck will replay the journal and exit, unless its superblock indicates that further checking is required. So don’t be dismayed if it seems to do its work too quickly to be believed.
If you suspect bad blocks on the device, add the option -c to use badblocks to find and mark any bad blocks.
xfs_repair /dev/sdb1, replacing sdb1 with your device name.
Ideally, run badblocks before xfs_repair. The xfs_repair manpage warns “The filesystem to be checked and repaired must have been unmounted cleanly…, not as a result of a crash or system reset.” Being lazy, I normally try xfs_repair directly and only run badblocks if xfs_repair fails with an error message that might imply bad blocks.
Using older versions of XFS tools on a newer version XFS filesystem can produce spurious errors and/or messages such as “XFS: bad version”. Insure your live distro is at least as recent as the filesystem you are repairing.
Once finished, reboot the machine from the hard disk and all should be well.
man e2fsck, man ntfsprogs, man xfs_repair