How to unmount NFS share mounted with hard option
Few days ago I worked on some customer’s server and there was a problem – their nfs server went down and we were forced to change some settings on their FC4 clients to prevent shares from dieing because of kernel bug. But when we’ve changed settings in /etc/fstab there was one more step before task was completed – we need to remount this share (I mean unmount/mount). But how to perform this operation if there are some processes in D (non-interruptible sleep) waiting for dead share and prevent it from unmounting? They wait because of hard option on the share and lack of intr option and any unmount request would produce a following results:
1 2 3 | streaming01:~# umount /storages/2 umount: /storages/2: device is busy umount: /storages/2: device is busy |
So, here is a list of steps you need to do to be able to remount your share.
First of all, you need to send KILL(9) signal to all you processes waiting for share. I’ve used ps axu and filtered all processes in D state. When all processes ‘killed’ (they can’t be killed actually), you’ll need to send the same signal to rpciod processes in your system. After this all your sleeping processes will die and you’ll be able to unmount your share.
That’s it – simple and really useful tip for people using NFS in their systems.
No related posts.
5 Responses to this entry
cool doc
I recommend using ‘lsof’ to check which processes is using the nfs share before killing one.
i.e. if one has a nfs share mounted on /netfs the syntax would be:
lsof +f — /netfs
(for more examples consult the lsof man pages)
Finally the answer! Thanks a lot man I was looking everywhere for this! The key for me was the ‘D’ state. I had an vsftpd process AND lsof process holding up the NFS umount. What happened was that I lost connection to my NFS server and then my IP changed. I couldn’t even run fuser OR lsof as it would just freeze and never timeout. I tried restarting all the rpc.nfs stuff and the ftp server to no avail. Finally this works, thanks a ton man!
man: umount
—————
-f Force unmount (in case of an unreachable NFS system). (Requires kernel 2.1.116 or later.)
-l Lazy unmount. Detach the filesystem from the filesystem hierarchy now, and cleanup all references to the filesystem as
soon as it is not busy anymore. (Requires kernel 2.4.11 or later.)
——————
umount -f does not work when NFS mount is stuffed due to the NFS server rebooting or something like that. The only thing which worked for me was what was posted here by Scoundrel. Thanks a million it helped me finally sorting this out without rebooting all involved servers. Triple kudos.