aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--Documentation/filesystems/nfs-rdma.txt75
1 files changed, 44 insertions, 31 deletions
diff --git a/Documentation/filesystems/nfs-rdma.txt b/Documentation/filesystems/nfs-rdma.txt
index d0ec45ae4e7d..9ad453d4891a 100644
--- a/Documentation/filesystems/nfs-rdma.txt
+++ b/Documentation/filesystems/nfs-rdma.txt
@@ -5,7 +5,7 @@
5################################################################################ 5################################################################################
6 6
7 Author: NetApp and Open Grid Computing 7 Author: NetApp and Open Grid Computing
8 Date: April 15, 2008 8 Date: May 29, 2008
9 9
10Table of Contents 10Table of Contents
11~~~~~~~~~~~~~~~~~ 11~~~~~~~~~~~~~~~~~
@@ -60,16 +60,18 @@ Installation
60 The procedures described in this document have been tested with 60 The procedures described in this document have been tested with
61 distributions from Red Hat's Fedora Project (http://fedora.redhat.com/). 61 distributions from Red Hat's Fedora Project (http://fedora.redhat.com/).
62 62
63 - Install nfs-utils-1.1.1 or greater on the client 63 - Install nfs-utils-1.1.2 or greater on the client
64 64
65 An NFS/RDMA mount point can only be obtained by using the mount.nfs 65 An NFS/RDMA mount point can be obtained by using the mount.nfs command in
66 command in nfs-utils-1.1.1 or greater. To see which version of mount.nfs 66 nfs-utils-1.1.2 or greater (nfs-utils-1.1.1 was the first nfs-utils version
67 you are using, type: 67 with support for NFS/RDMA mounts, but for various reasons we recommend using
68 nfs-utils-1.1.2 or greater). To see which version of mount.nfs you are
69 using, type:
68 70
69 > /sbin/mount.nfs -V 71 $ /sbin/mount.nfs -V
70 72
71 If the version is less than 1.1.1 or the command does not exist, 73 If the version is less than 1.1.2 or the command does not exist,
72 then you will need to install the latest version of nfs-utils. 74 you should install the latest version of nfs-utils.
73 75
74 Download the latest package from: 76 Download the latest package from:
75 77
@@ -77,22 +79,32 @@ Installation
77 79
78 Uncompress the package and follow the installation instructions. 80 Uncompress the package and follow the installation instructions.
79 81
80 If you will not be using GSS and NFSv4, the installation process 82 If you will not need the idmapper and gssd executables (you do not need
81 can be simplified by disabling these features when running configure: 83 these to create an NFS/RDMA enabled mount command), the installation
84 process can be simplified by disabling these features when running
85 configure:
82 86
83 > ./configure --disable-gss --disable-nfsv4 87 $ ./configure --disable-gss --disable-nfsv4
84 88
85 For more information on this see the package's README and INSTALL files. 89 To build nfs-utils you will need the tcp_wrappers package installed. For
90 more information on this see the package's README and INSTALL files.
86 91
87 After building the nfs-utils package, there will be a mount.nfs binary in 92 After building the nfs-utils package, there will be a mount.nfs binary in
88 the utils/mount directory. This binary can be used to initiate NFS v2, v3, 93 the utils/mount directory. This binary can be used to initiate NFS v2, v3,
89 or v4 mounts. To initiate a v4 mount, the binary must be called mount.nfs4. 94 or v4 mounts. To initiate a v4 mount, the binary must be called mount.nfs4.
90 The standard technique is to create a symlink called mount.nfs4 to mount.nfs. 95 The standard technique is to create a symlink called mount.nfs4 to mount.nfs.
91 96
92 NOTE: mount.nfs and therefore nfs-utils-1.1.1 or greater is only needed 97 This mount.nfs binary should be installed at /sbin/mount.nfs as follows:
98
99 $ sudo cp utils/mount/mount.nfs /sbin/mount.nfs
100
101 In this location, mount.nfs will be invoked automatically for NFS mounts
102 by the system mount commmand.
103
104 NOTE: mount.nfs and therefore nfs-utils-1.1.2 or greater is only needed
93 on the NFS client machine. You do not need this specific version of 105 on the NFS client machine. You do not need this specific version of
94 nfs-utils on the server. Furthermore, only the mount.nfs command from 106 nfs-utils on the server. Furthermore, only the mount.nfs command from
95 nfs-utils-1.1.1 is needed on the client. 107 nfs-utils-1.1.2 is needed on the client.
96 108
97 - Install a Linux kernel with NFS/RDMA 109 - Install a Linux kernel with NFS/RDMA
98 110
@@ -156,8 +168,8 @@ Check RDMA and NFS Setup
156 this time. For example, if you are using a Mellanox Tavor/Sinai/Arbel 168 this time. For example, if you are using a Mellanox Tavor/Sinai/Arbel
157 card: 169 card:
158 170
159 > modprobe ib_mthca 171 $ modprobe ib_mthca
160 > modprobe ib_ipoib 172 $ modprobe ib_ipoib
161 173
162 If you are using InfiniBand, make sure there is a Subnet Manager (SM) 174 If you are using InfiniBand, make sure there is a Subnet Manager (SM)
163 running on the network. If your IB switch has an embedded SM, you can 175 running on the network. If your IB switch has an embedded SM, you can
@@ -166,7 +178,7 @@ Check RDMA and NFS Setup
166 178
167 If an SM is running on your network, you should see the following: 179 If an SM is running on your network, you should see the following:
168 180
169 > cat /sys/class/infiniband/driverX/ports/1/state 181 $ cat /sys/class/infiniband/driverX/ports/1/state
170 4: ACTIVE 182 4: ACTIVE
171 183
172 where driverX is mthca0, ipath5, ehca3, etc. 184 where driverX is mthca0, ipath5, ehca3, etc.
@@ -174,10 +186,10 @@ Check RDMA and NFS Setup
174 To further test the InfiniBand software stack, use IPoIB (this 186 To further test the InfiniBand software stack, use IPoIB (this
175 assumes you have two IB hosts named host1 and host2): 187 assumes you have two IB hosts named host1 and host2):
176 188
177 host1> ifconfig ib0 a.b.c.x 189 host1$ ifconfig ib0 a.b.c.x
178 host2> ifconfig ib0 a.b.c.y 190 host2$ ifconfig ib0 a.b.c.y
179 host1> ping a.b.c.y 191 host1$ ping a.b.c.y
180 host2> ping a.b.c.x 192 host2$ ping a.b.c.x
181 193
182 For other device types, follow the appropriate procedures. 194 For other device types, follow the appropriate procedures.
183 195
@@ -214,9 +226,9 @@ NFS/RDMA Setup
214 226
215 For InfiniBand using a Mellanox adapter: 227 For InfiniBand using a Mellanox adapter:
216 228
217 > modprobe ib_mthca 229 $ modprobe ib_mthca
218 > modprobe ib_ipoib 230 $ modprobe ib_ipoib
219 > ifconfig ib0 a.b.c.d 231 $ ifconfig ib0 a.b.c.d
220 232
221 NOTE: use unique addresses for the client and server 233 NOTE: use unique addresses for the client and server
222 234
@@ -225,30 +237,31 @@ NFS/RDMA Setup
225 If the NFS/RDMA server was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in kernel config), 237 If the NFS/RDMA server was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in kernel config),
226 load the RDMA transport module: 238 load the RDMA transport module:
227 239
228 > modprobe svcrdma 240 $ modprobe svcrdma
229 241
230 Regardless of how the server was built (module or built-in), start the server: 242 Regardless of how the server was built (module or built-in), start the server:
231 243
232 > /etc/init.d/nfs start 244 $ /etc/init.d/nfs start
233 245
234 or 246 or
235 247
236 > service nfs start 248 $ service nfs start
237 249
238 Instruct the server to listen on the RDMA transport: 250 Instruct the server to listen on the RDMA transport:
239 251
240 > echo rdma 2050 > /proc/fs/nfsd/portlist 252 $ echo rdma 2050 > /proc/fs/nfsd/portlist
241 253
242 - On the client system 254 - On the client system
243 255
244 If the NFS/RDMA client was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in kernel config), 256 If the NFS/RDMA client was built as a module (CONFIG_SUNRPC_XPRT_RDMA=m in kernel config),
245 load the RDMA client module: 257 load the RDMA client module:
246 258
247 > modprobe xprtrdma.ko 259 $ modprobe xprtrdma.ko
248 260
249 Regardless of how the client was built (module or built-in), issue the mount.nfs command: 261 Regardless of how the client was built (module or built-in), use this command to
262 mount the NFS/RDMA server:
250 263
251 > /path/to/your/mount.nfs <IPoIB-server-name-or-address>:/<export> /mnt -i -o rdma,port=2050 264 $ mount -o rdma,port=2050 <IPoIB-server-name-or-address>:/<export> /mnt
252 265
253 To verify that the mount is using RDMA, run "cat /proc/mounts" and check the 266 To verify that the mount is using RDMA, run "cat /proc/mounts" and check the
254 "proto" field for the given mount. 267 "proto" field for the given mount.