powerpc/mm: Rework I$/D$ coherency (v3)

This patch reworks the way we do I and D cache coherency on PowerPC.

The "old" way was split in 3 different parts depending on the processor type:

   - Hash with per-page exec support (64-bit and >= POWER4 only) does it
at hashing time, by preventing exec on unclean pages and cleaning pages
on exec faults.

   - Everything without per-page exec support (32-bit hash, 8xx, and
64-bit < POWER4) does it for all page going to user space in update_mmu_cache().

   - Embedded with per-page exec support does it from do_page_fault() on
exec faults, in a way similar to what the hash code does.

That leads to confusion, and bugs. For example, the method using update_mmu_cache()
is racy on SMP where another processor can see the new PTE and hash it in before
we have cleaned the cache, and then blow trying to execute. This is hard to hit but
I think it has bitten us in the past.

Also, it's inefficient for embedded where we always end up having to do at least
one more page fault.

This reworks the whole thing by moving the cache sync into two main call sites,
though we keep different behaviours depending on the HW capability. The call
sites are set_pte_at() which is now made out of line, and ptep_set_access_flags()
which joins the former in pgtable.c

The base idea for Embedded with per-page exec support, is that we now do the
flush at set_pte_at() time when coming from an exec fault, which allows us
to avoid the double fault problem completely (we can even improve the situation
more by implementing TLB preload in update_mmu_cache() but that's for later).

If for some reason we didn't do it there and we try to execute, we'll hit
the page fault, which will do a minor fault, which will hit ptep_set_access_flags()
to do things like update _PAGE_ACCESSED or _PAGE_DIRTY if needed, we just make
this guys also perform the I/D cache sync for exec faults now. This second path
is the catch all for things that weren't cleaned at set_pte_at() time.

For cpus without per-pag exec support, we always do the sync at set_pte_at(),
thus guaranteeing that when the PTE is visible to other processors, the cache
is clean.

For the 64-bit hash with per-page exec support case, we keep the old mechanism
for now. I'll look into changing it later, once I've reworked a bit how we
use _PAGE_EXEC.

This is also a first step for adding _PAGE_EXEC support for embedded platforms

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

1 files changed, 17 insertions, 29 deletions

diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 91c7b8636b8a..76993941cac9 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -253,45 +253,33 @@ good_area:
 #endif /* CONFIG_8xx */
 
         if (is_exec) {
-#if !(defined(CONFIG_4xx) || defined(CONFIG_BOOKE))
-                /* protection fault */
+#ifdef CONFIG_PPC_STD_MMU
+                /* Protection fault on exec go straight to failure on
+                 * Hash based MMUs as they either don't support per-page
+                 * execute permission, or if they do, it's handled already
+                 * at the hash level. This test would probably have to
+                 * be removed if we change the way this works to make hash
+                 * processors use the same I/D cache coherency mechanism
+                 * as embedded.
+                 */
                 if (error_code & DSISR_PROTFAULT)
                         goto bad_area;
+#endif /* CONFIG_PPC_STD_MMU */
+
                 /*
                  * Allow execution from readable areas if the MMU does not
                  * provide separate controls over reading and executing.
+                 *
+                 * Note: That code used to not be enabled for 4xx/BookE.
+                 * It is now as I/D cache coherency for these is done at
+                 * set_pte_at() time and I see no reason why the test
+                 * below wouldn't be valid on those processors. This -may-
+                 * break programs compiled with a really old ABI though.
                  */
                 if (!(vma->vm_flags & VM_EXEC) &&
                     (cpu_has_feature(CPU_FTR_NOEXECUTE) ||
                      !(vma->vm_flags & (VM_READ | VM_WRITE))))
                         goto bad_area;
-#else
-                pte_t *ptep;
-                pmd_t *pmdp;
-
-                /* Since 4xx/Book-E supports per-page execute permission,
-                 * we lazily flush dcache to icache. */
-                ptep = NULL;
-                if (get_pteptr(mm, address, &ptep, &pmdp)) {
-                        spinlock_t *ptl = pte_lockptr(mm, pmdp);
-                        spin_lock(ptl);
-                        if (pte_present(*ptep)) {
-                                struct page *page = pte_page(*ptep);
-
-                                if (!test_bit(PG_arch_1, &page->flags)) {
-                                        flush_dcache_icache_page(page);
-                                        set_bit(PG_arch_1, &page->flags);
-                                }
-                                pte_update(ptep, 0, _PAGE_HWEXEC |
-                                           _PAGE_ACCESSED);
-                                local_flush_tlb_page(vma, address);
-                                pte_unmap_unlock(ptep, ptl);
-                                up_read(&mm->mmap_sem);
-                                return 0;
-                        }
-                        pte_unmap_unlock(ptep, ptl);
-                }
-#endif
         /* a write */
         } else if (is_write) {
                 if (!(vma->vm_flags & VM_WRITE))