[PATCH] IB/mthca: add fast memory region implementation

Implement fast memory regions (FMRs), where the driver writes directly into
the HCA's translation tables rather than requiring a firmware command.  For
Tavor, MTTs for FMR are separate from regular MTTs, and are reserved at driver
initialization.  This is done to limit the amount of virtual memory needed to
map the MTTs.  For Arbel, there's no such limitation, and all MTTs and MPTs
may be used for FMR or for regular MR.

Signed-off-by: Michael S. Tsirkin <mst@mellanox.co.il>
Signed-off-by: Roland Dreier <roland@topspin.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

1 files changed, 16 insertions, 3 deletions

diff --git a/drivers/infiniband/hw/mthca/mthca_profile.c b/drivers/infiniband/hw/mthca/mthca_profile.c
index cfd6f70c8df3..fd3f167e6460 100644
--- a/drivers/infiniband/hw/mthca/mthca_profile.c
+++ b/drivers/infiniband/hw/mthca/mthca_profile.c
@@ -223,9 +223,10 @@ u64 mthca_make_profile(struct mthca_dev *dev,
                         init_hca->mc_hash_sz      = 1 << (profile[i].log_num - 1);
                         break;
                 case MTHCA_RES_MPT:
-                        dev->limits.num_mpts = profile[i].num;
-                        init_hca->mpt_base   = profile[i].start;
-                        init_hca->log_mpt_sz = profile[i].log_num;
+                        dev->limits.num_mpts   = profile[i].num;
+                        dev->mr_table.mpt_base = profile[i].start;
+                        init_hca->mpt_base     = profile[i].start;
+                        init_hca->log_mpt_sz   = profile[i].log_num;
                         break;
                 case MTHCA_RES_MTT:
                         dev->limits.num_mtt_segs = profile[i].num;
@@ -259,6 +260,18 @@ u64 mthca_make_profile(struct mthca_dev *dev,
          */
         dev->limits.num_pds = MTHCA_NUM_PDS;
 
+        /*
+         * For Tavor, FMRs use ioremapped PCI memory. For 32 bit
+         * systems it may use too much vmalloc space to map all MTT
+         * memory, so we reserve some MTTs for FMR access, taking them
+         * out of the MR pool. They don't use additional memory, but
+         * we assign them as part of the HCA profile anyway.
+         */
+        if (dev->hca_type == ARBEL_NATIVE)
+                dev->limits.fmr_reserved_mtts = 0;
+        else
+                dev->limits.fmr_reserved_mtts = request->fmr_reserved_mtts;
+
         kfree(profile);
         return total_size;
 }