[PATCH] knfsd: move tempsock aging to a timer

Following are 11 patches from Greg Banks which combine to make knfsd more
Numa-aware.  They reduce hitting on 'global' data structures, and create some
data-structures that can be node-local.

knfsd threads are bound to a particular node, and the thread to handle a new
request is chosen from the threads that are attach to the node that received
the interrupt.

The distribution of threads across nodes can be controlled by a new file in
the 'nfsd' filesystem, though the default approach of an even spread is
probably fine for most sites.

Some (old) numbers that show the efficacy of these patches: N == number of
NICs == number of CPUs == nmber of clients.  Number of NUMA nodes == N/2

N	Throughput, MiB/s	CPU usage, % (max=N*100)
	Before	After		Before	After
	---	------	----		-----	-----
	4	312	435		350	228
	6	500	656		501	418
	8	562	804		690	589

This patch:

Move the aging of RPC/TCP connection sockets from the main svc_recv() loop to
a timer which uses a mark-and-sweep algorithm every 6 minutes.  This reduces
the amount of work that needs to be done in the main RPC loop and the length
of time we need to hold the (effectively global) svc_serv->sv_lock.

[akpm@osdl.org: cleanup]
Signed-off-by: Greg Banks <gnb@melbourne.sgi.com>
Signed-off-by: Neil Brown <neilb@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

1 files changed, 3 insertions, 0 deletions

diff --git a/net/sunrpc/svc.c b/net/sunrpc/svc.c
index eee45a58f3ee..0c2c52276285 100644
--- a/net/sunrpc/svc.c
+++ b/net/sunrpc/svc.c
@@ -59,6 +59,7 @@ svc_create(struct svc_program *prog, unsigned int bufsize,
         INIT_LIST_HEAD(&serv->sv_sockets);
         INIT_LIST_HEAD(&serv->sv_tempsocks);
         INIT_LIST_HEAD(&serv->sv_permsocks);
+        init_timer(&serv->sv_temptimer);
         spin_lock_init(&serv->sv_lock);
 
         /* Remove any stale portmap registrations */
@@ -87,6 +88,8 @@ svc_destroy(struct svc_serv *serv)
         } else
                 printk("svc_destroy: no threads for serv=%p!\n", serv);
 
+        del_timer_sync(&serv->sv_temptimer);
+
         while (!list_empty(&serv->sv_tempsocks)) {
                 svsk = list_entry(serv->sv_tempsocks.next,
                                   struct svc_sock,