uprobes/core: Optimize probe hits with the help of a counter

Maintain a per-mm counter: number of uprobes that are inserted
on this process address space.

This counter can be used at probe hit time to determine if we
need a lookup in the uprobes rbtree. Everytime a probe gets
inserted successfully, the probe count is incremented and
everytime a probe gets removed, the probe count is decremented.

The new uprobe_munmap hook ensures the count is correct on a
unmap or remap of a region. We expect that once a
uprobe_munmap() is called, the vma goes away.  So
uprobe_unregister() finding a probe to unregister would either
mean unmap event hasnt occurred yet or a mmap event on the same
executable file occured after a unmap event.

Additionally, uprobe_mmap hook now also gets called:

 a. on every executable vma that is COWed at fork.
 b. a vma of interest is newly mapped; breakpoint insertion also
    happens at the required address.

On process creation, make sure the probes count in the child is
set correctly.

Special cases that are taken care include:

 a. mremap
 b. VM_DONTCOPY vmas on fork()
 c. insertion/removal races in the parent during fork().

Signed-off-by: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Ananth N Mavinakayanahalli <ananth@in.ibm.com>
Cc: Jim Keniston <jkenisto@linux.vnet.ibm.com>
Cc: Linux-mm <linux-mm@kvack.org>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Arnaldo Carvalho de Melo <acme@infradead.org>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
Cc: Anton Arapov <anton@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20120330182646.10018.85805.sendpatchset@srdronam.in.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

1 files changed, 9 insertions, 1 deletions

diff --git a/mm/mmap.c b/mm/mmap.c
index 5a863d328a44..7c112fbca405 100644
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -218,6 +218,7 @@ void unlink_file_vma(struct vm_area_struct *vma)
                 mutex_lock(&mapping->i_mmap_mutex);
                 __remove_shared_vm_struct(vma, file, mapping);
                 mutex_unlock(&mapping->i_mmap_mutex);
+                uprobe_munmap(vma);
         }
 }
 
@@ -546,8 +547,14 @@ again:			remove_next = 1 + (end > next->vm_end);
 
         if (file) {
                 mapping = file->f_mapping;
-                if (!(vma->vm_flags & VM_NONLINEAR))
+                if (!(vma->vm_flags & VM_NONLINEAR)) {
                         root = &mapping->i_mmap;
+                        uprobe_munmap(vma);
+
+                        if (adjust_next)
+                                uprobe_munmap(next);
+                }
+
                 mutex_lock(&mapping->i_mmap_mutex);
                 if (insert) {
                         /*
@@ -626,6 +633,7 @@ again:			remove_next = 1 + (end > next->vm_end);
 
         if (remove_next) {
                 if (file) {
+                        uprobe_munmap(next);
                         fput(file);
                         if (next->vm_flags & VM_EXECUTABLE)
                                 removed_exe_file_vma(mm);