In Solaris it is sufficient to hold a reference to a vnode to prevent it from getting reclaimed. By extension that also keeps its znode around.

In FreeBSD only a vnode lock prevents vnode reclaiming. A reference does not help e.g. during forceful unmounting. That seems to be the only practical case today, but in theory VFS allows an active vnode to be reclaimed.

This problem is most visible in zfs_zget where we first get a znode under “object hold” lock. We are guaranteed that the znode and by extension its vnode are valid at that moment. Then we vref the node, drop all locks and return. As described above that does not provide any firm guarantee that either the vnode or the znode would stay valid after zfs_zget returns.

In Solaris ZFS had a dummy vnode lock. Solaris VFS allows for that. ZFS completely relies on a number of private locks.

z_teardown_lock protects against concurrent execution of most of vnode operations with unmounting. This lock is of “rrw” type — recursive read-write. vnode methods take a read lock, while unmount takes a write lock. When filesystem is forcefully unmounted, operations on the remaining vnodes will be aborted because of z_unmounted flag.

z_teardown_inactive_lock protects against concurrent execution of vfs_unmount and vop_inactive. Where vop_inactive is closer to FreeBSD vop_reclaim.

In FreeBSD all vnode operations are invoked with the vnode lock held.

znode had several special state flags:

• z_unlinked — set when a file is unlinked/removed, but its znode is still valid
• z_sa_hdl — this field is only manipulated under the object hold lock if this field is NULL, then the znode is effectively invalidated all vnodes operations on this znode will terminate with EIO it will not be returned by any lookups we are only waiting for its vnode to get reclaimed, so that the znode could be freed

zfsvfs (a filesystem/mountpoint structure) has the following state flags:

• z_unmounted — indicates that a filesystem has been unmounted this flag is primarily needed for forceful unmount

As described above zfs_zget returns a potentially unstable znode. Currently only forceful unmount can damage the znode. Most places that call zfs_zget do that under ZFS_ENTER (z_teardown_lock held and z_unmounted checked). So the znode should be stable. Places that return a vnode or make use of it further acquire a vnode lock, which allows to keep the vnode stable too.

One problematic place is zfs_root, which explicitly opens a hole in the logic by not checking for z_unmounted. So, it may call zfs_zget on a filesystem being actively unmounted (z_teardown_lock write lock is not held around vflush). Although zfs_root tries to obtain the vnode lock, the znode can become destroyed after zfs_zget returns and before the returned znode is accessed.

The hole was opened because zfs_zroot is called from vflush and is expected to succeed.

Some porting rules that I've just made up:

• z_vnode field should be cleared only after z_sa_hdl is cleared (zfs_znode_dmu_fini) we should not allow a znode with invalid z_vnode to be visible