NFS: Share NFS superblocks per-protocol per-server per-FSID

The attached patch makes NFS share superblocks between mounts from the same
server and FSID over the same protocol.

It does this by creating each superblock with a false root and returning the
real root dentry in the vfsmount presented by get_sb(). The root dentry set
starts off as an anonymous dentry if we don't already have the dentry for its
inode, otherwise it simply returns the dentry we already have.

We may thus end up with several trees of dentries in the superblock, and if at
some later point one of anonymous tree roots is discovered by normal filesystem
activity to be located in another tree within the superblock, the anonymous
root is named and materialises attached to the second tree at the appropriate
point.

Why do it this way? Why not pass an extra argument to the mount() syscall to
indicate the subpath and then pathwalk from the server root to the desired
directory? You can't guarantee this will work for two reasons:

 (1) The root and intervening nodes may not be accessible to the client.

     With NFS2 and NFS3, for instance, mountd is called on the server to get
     the filehandle for the tip of a path. mountd won't give us handles for
     anything we don't have permission to access, and so we can't set up NFS
     inodes for such nodes, and so can't easily set up dentries (we'd have to
     have ghost inodes or something).

     With this patch we don't actually create dentries until we get handles
     from the server that we can use to set up their inodes, and we don't
     actually bind them into the tree until we know for sure where they go.

 (2) Inaccessible symbolic links.

     If we're asked to mount two exports from the server, eg:

	mount warthog:/warthog/aaa/xxx /mmm
	mount warthog:/warthog/bbb/yyy /nnn

     We may not be able to access anything nearer the root than xxx and yyy,
     but we may find out later that /mmm/www/yyy, say, is actually the same
     directory as the one mounted on /nnn. What we might then find out, for
     example, is that /warthog/bbb was actually a symbolic link to
     /warthog/aaa/xxx/www, but we can't actually determine that by talking to
     the server until /warthog is made available by NFS.

     This would lead to having constructed an errneous dentry tree which we
     can't easily fix. We can end up with a dentry marked as a directory when
     it should actually be a symlink, or we could end up with an apparently
     hardlinked directory.

     With this patch we need not make assumptions about the type of a dentry
     for which we can't retrieve information, nor need we assume we know its
     place in the grand scheme of things until we actually see that place.

This patch reduces the possibility of aliasing in the inode and page caches for
inodes that may be accessed by more than one NFS export. It also reduces the
number of superblocks required for NFS where there are many NFS exports being
used from a server (home directory server + autofs for example).

This in turn makes it simpler to do local caching of network filesystems, as it
can then be guaranteed that there won't be links from multiple inodes in
separate superblocks to the same cache file.

Obviously, cache aliasing between different levels of NFS protocol could still
be a problem, but at least that gives us another key to use when indexing the
cache.

This patch makes the following changes:

 (1) The server record construction/destruction has been abstracted out into
     its own set of functions to make things easier to get right.  These have
     been moved into fs/nfs/client.c.

     All the code in fs/nfs/client.c has to do with the management of
     connections to servers, and doesn't touch superblocks in any way; the
     remaining code in fs/nfs/super.c has to do with VFS superblock management.

 (2) The sequence of events undertaken by NFS mount is now reordered:

     (a) A volume representation (struct nfs_server) is allocated.

     (b) A server representation (struct nfs_client) is acquired.  This may be
     	 allocated or shared, and is keyed on server address, port and NFS
     	 version.

     (c) If allocated, the client representation is initialised.  The state
     	 member variable of nfs_client is used to prevent a race during
     	 initialisation from two mounts.

     (d) For NFS4 a simple pathwalk is performed, walking from FH to FH to find
     	 the root filehandle for the mount (fs/nfs/getroot.c).  For NFS2/3 we
     	 are given the root FH in advance.

     (e) The volume FSID is probed for on the root FH.

     (f) The volume representation is initialised from the FSINFO record
     	 retrieved on the root FH.

     (g) sget() is called to acquire a superblock.  This may be allocated or
     	 shared, keyed on client pointer and FSID.

     (h) If allocated, the superblock is initialised.

     (i) If the superblock is shared, then the new nfs_server record is
     	 discarded.

     (j) The root dentry for this mount is looked up from the root FH.

     (k) The root dentry for this mount is assigned to the vfsmount.

 (3) nfs_readdir_lookup() creates dentries for each of the entries readdir()
     returns; this function now attaches disconnected trees from alternate
     roots that happen to be discovered attached to a directory being read (in
     the same way nfs_lookup() is made to do for lookup ops).

     The new d_materialise_unique() function is now used to do this, thus
     permitting the whole thing to be done under one set of locks, and thus
     avoiding any race between mount and lookup operations on the same
     directory.

 (4) The client management code uses a new debug facility: NFSDBG_CLIENT which
     is set by echoing 1024 to /proc/net/sunrpc/nfs_debug.

 (5) Clone mounts are now called xdev mounts.

 (6) Use the dentry passed to the statfs() op as the handle for retrieving fs
     statistics rather than the root dentry of the superblock (which is now a
     dummy).

Signed-Off-By: David Howells <dhowells@redhat.com>
Signed-off-by: Trond Myklebust <Trond.Myklebust@netapp.com>

1 files changed, 306 insertions, 0 deletions

diff --git a/fs/nfs/getroot.c b/fs/nfs/getroot.c
new file mode 100644
index 000000000000..977e59088eeb
--- /dev/null
+++ b/fs/nfs/getroot.c
@@ -0,0 +1,306 @@
+/* getroot.c: get the root dentry for an NFS mount
+ *
+ * Copyright (C) 2006 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/init.h>
+
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/mm.h>
+#include <linux/string.h>
+#include <linux/stat.h>
+#include <linux/errno.h>
+#include <linux/unistd.h>
+#include <linux/sunrpc/clnt.h>
+#include <linux/sunrpc/stats.h>
+#include <linux/nfs_fs.h>
+#include <linux/nfs_mount.h>
+#include <linux/nfs4_mount.h>
+#include <linux/lockd/bind.h>
+#include <linux/smp_lock.h>
+#include <linux/seq_file.h>
+#include <linux/mount.h>
+#include <linux/nfs_idmap.h>
+#include <linux/vfs.h>
+#include <linux/namei.h>
+#include <linux/namespace.h>
+
+#include <asm/system.h>
+#include <asm/uaccess.h>
+
+#include "nfs4_fs.h"
+#include "delegation.h"
+#include "internal.h"
+
+#define NFSDBG_FACILITY         NFSDBG_CLIENT
+#define NFS_PARANOIA 1
+
+/*
+ * get an NFS2/NFS3 root dentry from the root filehandle
+ */
+struct dentry *nfs_get_root(struct super_block *sb, struct nfs_fh *mntfh)
+{
+        struct nfs_server *server = NFS_SB(sb);
+        struct nfs_fsinfo fsinfo;
+        struct nfs_fattr fattr;
+        struct dentry *mntroot;
+        struct inode *inode;
+        int error;
+
+        /* create a dummy root dentry with dummy inode for this superblock */
+        if (!sb->s_root) {
+                struct nfs_fh dummyfh;
+                struct dentry *root;
+                struct inode *iroot;
+
+                memset(&dummyfh, 0, sizeof(dummyfh));
+                memset(&fattr, 0, sizeof(fattr));
+                nfs_fattr_init(&fattr);
+                fattr.valid = NFS_ATTR_FATTR;
+                fattr.type = NFDIR;
+                fattr.mode = S_IFDIR | S_IRUSR | S_IWUSR;
+                fattr.nlink = 2;
+
+                iroot = nfs_fhget(sb, &dummyfh, &fattr);
+                if (IS_ERR(iroot))
+                        return ERR_PTR(PTR_ERR(iroot));
+
+                root = d_alloc_root(iroot);
+                if (!root) {
+                        iput(iroot);
+                        return ERR_PTR(-ENOMEM);
+                }
+
+                sb->s_root = root;
+        }
+
+        /* get the actual root for this mount */
+        fsinfo.fattr = &fattr;
+
+        error = server->nfs_client->rpc_ops->getroot(server, mntfh, &fsinfo);
+        if (error < 0) {
+                dprintk("nfs_get_root: getattr error = %d\n", -error);
+                return ERR_PTR(error);
+        }
+
+        inode = nfs_fhget(sb, mntfh, fsinfo.fattr);
+        if (IS_ERR(inode)) {
+                dprintk("nfs_get_root: get root inode failed\n");
+                return ERR_PTR(PTR_ERR(inode));
+        }
+
+        /* root dentries normally start off anonymous and get spliced in later
+         * if the dentry tree reaches them; however if the dentry already
+         * exists, we'll pick it up at this point and use it as the root
+         */
+        mntroot = d_alloc_anon(inode);
+        if (!mntroot) {
+                iput(inode);
+                dprintk("nfs_get_root: get root dentry failed\n");
+                return ERR_PTR(-ENOMEM);
+        }
+
+        if (!mntroot->d_op)
+                mntroot->d_op = server->nfs_client->rpc_ops->dentry_ops;
+
+        return mntroot;
+}
+
+#ifdef CONFIG_NFS_V4
+
+/*
+ * Do a simple pathwalk from the root FH of the server to the nominated target
+ * of the mountpoint
+ * - give error on symlinks
+ * - give error on ".." occurring in the path
+ * - follow traversals
+ */
+int nfs4_path_walk(struct nfs_server *server,
+                   struct nfs_fh *mntfh,
+                   const char *path)
+{
+        struct nfs_fsinfo fsinfo;
+        struct nfs_fattr fattr;
+        struct nfs_fh lastfh;
+        struct qstr name;
+        int ret;
+        //int referral_count = 0;
+
+        dprintk("--> nfs4_path_walk(,,%s)\n", path);
+
+        fsinfo.fattr = &fattr;
+        nfs_fattr_init(&fattr);
+
+        if (*path++ != '/') {
+                dprintk("nfs4_get_root: Path does not begin with a slash\n");
+                return -EINVAL;
+        }
+
+        /* Start by getting the root filehandle from the server */
+        ret = server->nfs_client->rpc_ops->getroot(server, mntfh, &fsinfo);
+        if (ret < 0) {
+                dprintk("nfs4_get_root: getroot error = %d\n", -ret);
+                return ret;
+        }
+
+        if (fattr.type != NFDIR) {
+                printk(KERN_ERR "nfs4_get_root:"
+                       " getroot encountered non-directory\n");
+                return -ENOTDIR;
+        }
+
+        if (fattr.valid & NFS_ATTR_FATTR_V4_REFERRAL) {
+                printk(KERN_ERR "nfs4_get_root:"
+                       " getroot obtained referral\n");
+                return -EREMOTE;
+        }
+
+next_component:
+        dprintk("Next: %s\n", path);
+
+        /* extract the next bit of the path */
+        if (!*path)
+                goto path_walk_complete;
+
+        name.name = path;
+        while (*path && *path != '/')
+                path++;
+        name.len = path - (const char *) name.name;
+
+eat_dot_dir:
+        while (*path == '/')
+                path++;
+
+        if (path[0] == '.' && (path[1] == '/' || !path[1])) {
+                path += 2;
+                goto eat_dot_dir;
+        }
+
+        if (path[0] == '.' && path[1] == '.' && (path[2] == '/' || !path[2])
+            ) {
+                printk(KERN_ERR "nfs4_get_root:"
+                       " Mount path contains reference to \"..\"\n");
+                return -EINVAL;
+        }
+
+        /* lookup the next FH in the sequence */
+        memcpy(&lastfh, mntfh, sizeof(lastfh));
+
+        dprintk("LookupFH: %*.*s [%s]\n", name.len, name.len, name.name, path);
+
+        ret = server->nfs_client->rpc_ops->lookupfh(server, &lastfh, &name,
+                                                    mntfh, &fattr);
+        if (ret < 0) {
+                dprintk("nfs4_get_root: getroot error = %d\n", -ret);
+                return ret;
+        }
+
+        if (fattr.type != NFDIR) {
+                printk(KERN_ERR "nfs4_get_root:"
+                       " lookupfh encountered non-directory\n");
+                return -ENOTDIR;
+        }
+
+        if (fattr.valid & NFS_ATTR_FATTR_V4_REFERRAL) {
+                printk(KERN_ERR "nfs4_get_root:"
+                       " lookupfh obtained referral\n");
+                return -EREMOTE;
+        }
+
+        goto next_component;
+
+path_walk_complete:
+        memcpy(&server->fsid, &fattr.fsid, sizeof(server->fsid));
+        dprintk("<-- nfs4_path_walk() = 0\n");
+        return 0;
+}
+
+/*
+ * get an NFS4 root dentry from the root filehandle
+ */
+struct dentry *nfs4_get_root(struct super_block *sb, struct nfs_fh *mntfh)
+{
+        struct nfs_server *server = NFS_SB(sb);
+        struct nfs_fattr fattr;
+        struct dentry *mntroot;
+        struct inode *inode;
+        int error;
+
+        dprintk("--> nfs4_get_root()\n");
+
+        /* create a dummy root dentry with dummy inode for this superblock */
+        if (!sb->s_root) {
+                struct nfs_fh dummyfh;
+                struct dentry *root;
+                struct inode *iroot;
+
+                memset(&dummyfh, 0, sizeof(dummyfh));
+                memset(&fattr, 0, sizeof(fattr));
+                nfs_fattr_init(&fattr);
+                fattr.valid = NFS_ATTR_FATTR;
+                fattr.type = NFDIR;
+                fattr.mode = S_IFDIR | S_IRUSR | S_IWUSR;
+                fattr.nlink = 2;
+
+                iroot = nfs_fhget(sb, &dummyfh, &fattr);
+                if (IS_ERR(iroot))
+                        return ERR_PTR(PTR_ERR(iroot));
+
+                root = d_alloc_root(iroot);
+                if (!root) {
+                        iput(iroot);
+                        return ERR_PTR(-ENOMEM);
+                }
+
+                sb->s_root = root;
+        }
+
+        /* get the info about the server and filesystem */
+        error = nfs4_server_capabilities(server, mntfh);
+        if (error < 0) {
+                dprintk("nfs_get_root: getcaps error = %d\n",
+                        -error);
+                return ERR_PTR(error);
+        }
+
+        /* get the actual root for this mount */
+        error = server->nfs_client->rpc_ops->getattr(server, mntfh, &fattr);
+        if (error < 0) {
+                dprintk("nfs_get_root: getattr error = %d\n", -error);
+                return ERR_PTR(error);
+        }
+
+        inode = nfs_fhget(sb, mntfh, &fattr);
+        if (IS_ERR(inode)) {
+                dprintk("nfs_get_root: get root inode failed\n");
+                return ERR_PTR(PTR_ERR(inode));
+        }
+
+        /* root dentries normally start off anonymous and get spliced in later
+         * if the dentry tree reaches them; however if the dentry already
+         * exists, we'll pick it up at this point and use it as the root
+         */
+        mntroot = d_alloc_anon(inode);
+        if (!mntroot) {
+                iput(inode);
+                dprintk("nfs_get_root: get root dentry failed\n");
+                return ERR_PTR(-ENOMEM);
+        }
+
+        if (!mntroot->d_op)
+                mntroot->d_op = server->nfs_client->rpc_ops->dentry_ops;
+
+        dprintk("<-- nfs4_get_root()\n");
+        return mntroot;
+}
+
+#endif /* CONFIG_NFS_V4 */