diff options
author | Dave Hansen <dave@sr71.net> | 2013-04-12 19:23:54 -0400 |
---|---|---|
committer | Linus Torvalds <torvalds@linux-foundation.org> | 2013-04-12 19:56:47 -0400 |
commit | 1de14c3c5cbc9bb17e9dcc648cda51c0c85d54b9 (patch) | |
tree | ab6010ad78195651c39a8ca3829ba1b95451762f /include/asm-generic/tlb.h | |
parent | bf81710c4b6e2df2cc047f7c8e1f342511904b74 (diff) |
x86-32: Fix possible incomplete TLB invalidate with PAE pagetables
This patch attempts to fix:
https://bugzilla.kernel.org/show_bug.cgi?id=56461
The symptom is a crash and messages like this:
chrome: Corrupted page table at address 34a03000
*pdpt = 0000000000000000 *pde = 0000000000000000
Bad pagetable: 000f [#1] PREEMPT SMP
Ingo guesses this got introduced by commit 611ae8e3f520 ("x86/tlb:
enable tlb flush range support for x86") since that code started to free
unused pagetables.
On x86-32 PAE kernels, that new code has the potential to free an entire
PMD page and will clear one of the four page-directory-pointer-table
(aka pgd_t entries).
The hardware aggressively "caches" these top-level entries and invlpg
does not actually affect the CPU's copy. If we clear one we *HAVE* to
do a full TLB flush, otherwise we might continue using a freed pmd page.
(note, we do this properly on the population side in pud_populate()).
This patch tracks whenever we clear one of these entries in the 'struct
mmu_gather', and ensures that we follow up with a full tlb flush.
BTW, I disassembled and checked that:
if (tlb->fullmm == 0)
and
if (!tlb->fullmm && !tlb->need_flush_all)
generate essentially the same code, so there should be zero impact there
to the !PAE case.
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Peter Anvin <hpa@zytor.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Artem S Tashkinov <t.artem@mailcity.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Diffstat (limited to 'include/asm-generic/tlb.h')
-rw-r--r-- | include/asm-generic/tlb.h | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index 25f01d0bc149..b1b1fa6ffffe 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h | |||
@@ -99,7 +99,12 @@ struct mmu_gather { | |||
99 | unsigned int need_flush : 1, /* Did free PTEs */ | 99 | unsigned int need_flush : 1, /* Did free PTEs */ |
100 | fast_mode : 1; /* No batching */ | 100 | fast_mode : 1; /* No batching */ |
101 | 101 | ||
102 | unsigned int fullmm; | 102 | /* we are in the middle of an operation to clear |
103 | * a full mm and can make some optimizations */ | ||
104 | unsigned int fullmm : 1, | ||
105 | /* we have performed an operation which | ||
106 | * requires a complete flush of the tlb */ | ||
107 | need_flush_all : 1; | ||
103 | 108 | ||
104 | struct mmu_gather_batch *active; | 109 | struct mmu_gather_batch *active; |
105 | struct mmu_gather_batch local; | 110 | struct mmu_gather_batch local; |