[PATCH] x86-64: Calgary IOMMU: Fix off by one when calculating register space location

The purpose of the code being modified is to determine the location
of the calgary chip address space.  This is done by a magical formula
of FE0MB-8MB*OneBasedChassisNumber+1MB*(RioNodeId-ChassisBase) to
find the offset where BIOS puts it.  In this formula,
OneBasedChassisNumber corresponds to the NUMA node, and rionodeid is
always 2 or 3 depending on which chip in the system it is.  The
problem was that we had an off by one error that caused us to account
some busses to the wrong chip and thus give them the wrong address
space.

Fixes RH bugzilla #203971.

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-bu: Muli Ben-Yehuda <muli@il.ibm.com>
Signed-off-by: Andi Kleen <ak@suse.de>

1 files changed, 11 insertions, 2 deletions

diff --git a/arch/x86_64/kernel/pci-calgary.c b/arch/x86_64/kernel/pci-calgary.c
index 1100031528dc..3c0c7fde09d8 100644
--- a/arch/x86_64/kernel/pci-calgary.c
+++ b/arch/x86_64/kernel/pci-calgary.c
@@ -790,7 +790,16 @@ static inline unsigned int __init locate_register_space(struct pci_dev *dev)
         int rionodeid;
         u32 address;
 
-        rionodeid = (dev->bus->number % 15 > 4) ? 3 : 2;
+        /*
+         * Each Calgary has four busses. The first four busses (first Calgary)
+         * have RIO node ID 2, then the next four (second Calgary) have RIO
+         * node ID 3, the next four (third Calgary) have node ID 2 again, etc.
+         * We use a gross hack - relying on the dev->bus->number ordering,
+         * modulo 14 - to decide which Calgary a given bus is on. Busses 0, 1,
+         * 2 and 4 are on the first Calgary (id 2), 6, 8, a and c are on the
+         * second (id 3), and then it repeats modulo 14.
+         */
+        rionodeid = (dev->bus->number % 14 > 4) ? 3 : 2;
         /*
          * register space address calculation as follows:
          * FE0MB-8MB*OneBasedChassisNumber+1MB*(RioNodeId-ChassisBase)
@@ -798,7 +807,7 @@ static inline unsigned int __init locate_register_space(struct pci_dev *dev)
          * RioNodeId is 2 for first Calgary, 3 for second Calgary
          */
         address = START_ADDRESS -
-                (0x800000 * (ONE_BASED_CHASSIS_NUM + dev->bus->number / 15)) +
+                (0x800000 * (ONE_BASED_CHASSIS_NUM + dev->bus->number / 14)) +
                 (0x100000) * (rionodeid - CHASSIS_BASE);
         return address;
 }