arch, mm: Remove tlb_fast_mode()

Since the introduction of preemptible mmu_gather TLB fast mode has been
broken. TLB fast mode relies on there being absolutely no concurrency;
it frees pages first and invalidates TLBs later.

However now we can get concurrency and stuff goes *bang*.

This patch removes all tlb_fast_mode() code; it was found the better
option vs trying to patch the hole by entangling tlb invalidation with
the scheduler.

Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Reported-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 0 insertions, 9 deletions

diff --git a/mm/memory.c b/mm/memory.c
index 6dc1882fbd72..61a262b08e53 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -220,7 +220,6 @@ void tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, bool fullmm)
         tlb->start      = -1UL;
         tlb->end        = 0;
         tlb->need_flush = 0;
-        tlb->fast_mode  = (num_possible_cpus() == 1);
         tlb->local.next = NULL;
         tlb->local.nr   = 0;
         tlb->local.max  = ARRAY_SIZE(tlb->__pages);
@@ -244,9 +243,6 @@ void tlb_flush_mmu(struct mmu_gather *tlb)
         tlb_table_flush(tlb);
 #endif
 
-        if (tlb_fast_mode(tlb))
-                return;
-
         for (batch = &tlb->local; batch; batch = batch->next) {
                 free_pages_and_swap_cache(batch->pages, batch->nr);
                 batch->nr = 0;
@@ -288,11 +284,6 @@ int __tlb_remove_page(struct mmu_gather *tlb, struct page *page)
 
         VM_BUG_ON(!tlb->need_flush);
 
-        if (tlb_fast_mode(tlb)) {
-                free_page_and_swap_cache(page);
-                return 1; /* avoid calling tlb_flush_mmu() */
-        }
-
         batch = tlb->active;
         batch->pages[batch->nr++] = page;
         if (batch->nr == batch->max) {