RDMA/verbs: Cleanup XRC TGT QPs when destroying XRCD

XRC TGT QPs are intended to be shared among multiple users and
processes.  Allow the destruction of an XRC TGT QP to be done explicitly
through ib_destroy_qp() or when the XRCD is destroyed.

To support destroying an XRC TGT QP, we need to track TGT QPs with the
XRCD.  When the XRCD is destroyed, all tracked XRC TGT QPs are also
cleaned up.

To avoid stale reference issues, if a user is holding a reference on a
TGT QP, we increment a reference count on the QP.  The user releases the
reference by calling ib_release_qp.  This releases any access to the QP
from a user above verbs, but allows the QP to continue to exist until
destroyed by the XRCD.

Signed-off-by: Sean Hefty <sean.hefty@intel.com>
Signed-off-by: Roland Dreier <roland@purestorage.com>

1 files changed, 16 insertions, 1 deletions

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index c3d7602b5e90..ac46dcf04358 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -880,7 +880,10 @@ struct ib_pd {
 
 struct ib_xrcd {
         struct ib_device       *device;
-        atomic_t                usecnt; /* count all resources */
+        atomic_t                usecnt; /* count all exposed resources */
+
+        struct mutex            tgt_qp_mutex;
+        struct list_head        tgt_qp_list;
 };
 
 struct ib_ah {
@@ -926,6 +929,7 @@ struct ib_qp {
         struct ib_cq           *recv_cq;
         struct ib_srq          *srq;
         struct ib_xrcd         *xrcd; /* XRC TGT QPs only */
+        struct list_head        xrcd_list;
         struct ib_uobject      *uobject;
         void                  (*event_handler)(struct ib_event *, void *);
         void                   *qp_context;
@@ -1482,6 +1486,17 @@ int ib_query_qp(struct ib_qp *qp,
 int ib_destroy_qp(struct ib_qp *qp);
 
 /**
+ * ib_release_qp - Release an external reference to a QP.
+ * @qp: The QP handle to release
+ *
+ * The specified QP handle is released by the caller.  If the QP is
+ * referenced internally, it is not destroyed until all internal
+ * references are released.  After releasing the qp, the caller
+ * can no longer access it and all events on the QP are discarded.
+ */
+int ib_release_qp(struct ib_qp *qp);
+
+/**
  * ib_post_send - Posts a list of work requests to the send queue of
  *   the specified QP.
  * @qp: The QP to post the work request on.