KVM: PPC: Book 3S HV: Do ptesync in radix guest exit path

A radix guest can execute tlbie instructions to invalidate TLB entries.
After a tlbie or a group of tlbies, it must then do the architected
sequence eieio; tlbsync; ptesync to ensure that the TLB invalidation
has been processed by all CPUs in the system before it can rely on
no CPU using any translation that it just invalidated.

In fact it is the ptesync which does the actual synchronization in
this sequence, and hardware has a requirement that the ptesync must
be executed on the same CPU thread as the tlbies which it is expected
to order.  Thus, if a vCPU gets moved from one physical CPU to
another after it has done some tlbies but before it can get to do the
ptesync, the ptesync will not have the desired effect when it is
executed on the second physical CPU.

To fix this, we do a ptesync in the exit path for radix guests.  If
there are any pending tlbies, this will wait for them to complete.
If there aren't, then ptesync will just do the same as sync.

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>

1 files changed, 8 insertions, 0 deletions

diff --git a/arch/powerpc/kvm/book3s_hv_rmhandlers.S b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
index 25c32e421b57..07ca1b2a7966 100644
--- a/arch/powerpc/kvm/book3s_hv_rmhandlers.S
+++ b/arch/powerpc/kvm/book3s_hv_rmhandlers.S
@@ -1908,6 +1908,14 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300)
         cmpwi   cr2, r0, 0
         beq     cr2, 4f
 
+        /*
+         * Radix: do eieio; tlbsync; ptesync sequence in case we
+         * interrupted the guest between a tlbie and a ptesync.
+         */
+        eieio
+        tlbsync
+        ptesync
+
         /* Radix: Handle the case where the guest used an illegal PID */
         LOAD_REG_ADDR(r4, mmu_base_pid)
         lwz     r3, VCPU_GUEST_PID(r9)