[PATCH] x86_64 irq: Allocate a vector across all cpus for genapic_flat.

The problem we can't take advantage of lowest priority delivery mode if
the vectors are allocated for only one cpu at a time.  Nor can we work
around hardware that assumes lowest priority delivery mode is always
used with several cpus.

So this patch introduces the concept of a vector_allocation_domain.  A
set of cpus that will receive an irq on the same vector.  Currently the
code for implementing this is placed in the genapic structure so we can
vary this depending on how we are using the io_apics.

This allows us to restore the previous behaviour of genapic_flat without
removing the benefits of having separate vector allocation for large
machines.

This should also fix the problem report where a hyperthreaded cpu was
receving the irq on the wrong hyperthread when in logical delivery mode
because the previous behaviour is restored.

This patch properly records our allocation of the first 16 irqs to the
first 16 available vectors on all cpus.  This should be fine but it may
run into problems with multiple interrupts at the same interrupt level.
Except for some badly maintained comments in the code and the behaviour
of the interrupt allocator I have no real understanding of that problem.

Signed-off-by: Eric W. Biederman <ebiederm@xmission.com>
Acked-by: Muli Ben-Yehuda <muli@il.ibm.com>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

1 files changed, 24 insertions, 0 deletions

diff --git a/arch/x86_64/kernel/genapic_flat.c b/arch/x86_64/kernel/genapic_flat.c
index 50ad153eaac..0dfc223c183 100644
--- a/arch/x86_64/kernel/genapic_flat.c
+++ b/arch/x86_64/kernel/genapic_flat.c
@@ -22,6 +22,20 @@ static cpumask_t flat_target_cpus(void)
         return cpu_online_map;
 }
 
+static cpumask_t flat_vector_allocation_domain(int cpu)
+{
+        /* Careful. Some cpus do not strictly honor the set of cpus
+         * specified in the interrupt destination when using lowest
+         * priority interrupt delivery mode.
+         *
+         * In particular there was a hyperthreading cpu observed to
+         * deliver interrupts to the wrong hyperthread when only one
+         * hyperthread was specified in the interrupt desitination.
+         */
+        cpumask_t domain = { { [0] = APIC_ALL_CPUS, } };
+        return domain;
+}
+
 /*
  * Set up the logical destination ID.
  *
@@ -121,6 +135,7 @@ struct genapic apic_flat =  {
         .int_delivery_mode = dest_LowestPrio,
         .int_dest_mode = (APIC_DEST_LOGICAL != 0),
         .target_cpus = flat_target_cpus,
+        .vector_allocation_domain = flat_vector_allocation_domain,
         .apic_id_registered = flat_apic_id_registered,
         .init_apic_ldr = flat_init_apic_ldr,
         .send_IPI_all = flat_send_IPI_all,
@@ -141,6 +156,14 @@ static cpumask_t physflat_target_cpus(void)
         return cpumask_of_cpu(0);
 }
 
+static cpumask_t physflat_vector_allocation_domain(int cpu)
+{
+        cpumask_t domain = CPU_MASK_NONE;
+        cpu_set(cpu, domain);
+        return domain;
+}
+
+
 static void physflat_send_IPI_mask(cpumask_t cpumask, int vector)
 {
         send_IPI_mask_sequence(cpumask, vector);
@@ -179,6 +202,7 @@ struct genapic apic_physflat =  {
         .int_delivery_mode = dest_Fixed,
         .int_dest_mode = (APIC_DEST_PHYSICAL != 0),
         .target_cpus = physflat_target_cpus,
+        .vector_allocation_domain = physflat_vector_allocation_domain,
         .apic_id_registered = flat_apic_id_registered,
         .init_apic_ldr = flat_init_apic_ldr,/*not needed, but shouldn't hurt*/
         .send_IPI_all = physflat_send_IPI_all,