drm/i915: Create bind/unbind abstraction for VMAs

To sum up what goes on here, we abstract the vma binding, similarly to
the previous object binding. This helps for distinguishing legacy
binding, versus modern binding. To keep the code churn as minimal as
possible, I am leaving in insert_entries(). It serves as the per
platform pte writing basically. bind_vma and insert_entries do share a
lot of similarities, and I did have designs to combine the two, but as
mentioned already... too much churn in an already massive patchset.

What follows are the 3 commits which existed discretely in the original
submissions. Upon rebasing on Broadwell support, it became clear that
separation was not good, and only made for more error prone code. Below
are the 3 commit messages with all their history.

drm/i915: Add bind/unbind object functions to VMA
drm/i915: Use the new vm [un]bind functions
drm/i915: reduce vm->insert_entries() usage

drm/i915: Add bind/unbind object functions to VMA

As we plumb the code with more VM information, it has become more
obvious that the easiest way to deal with bind and unbind is to simply
put the function pointers in the vm, and let those choose the correct
way to handle the page table updates. This change allows many places in
the code to simply be vm->bind, and not have to worry about
distinguishing PPGTT vs GGTT.

Notice that this patch has no impact on functionality. I've decided to
save the actual change until the next patch because I think it's easier
to review that way. I'm happy to squash the two, or let Daniel do it on
merge.

v2:
Make ggtt handle the quirky aliasing ppgtt
Add flags to bind object to support above
Don't ever call bind/unbind directly for PPGTT until we have real, full
PPGTT (use NULLs to assert this)
Make sure we rebind the ggtt if there already is a ggtt binding.  This
happens on set cache levels.
Use VMA for bind/unbind (Daniel, Ben)

v3: Reorganize ggtt_vma_bind to be more concise and easier to read
(Ville). Change logic in unbind to only unbind ggtt when there is a
global mapping, and to remove a redundant check if the aliasing ppgtt
exists.

v4: Make the bind function a bit smarter about the cache levels to avoid
unnecessary multiple remaps. "I accept it is a wart, I think unifying
the pin_vma / bind_vma could be unified later" (Chris)
Removed the git notes, and put version info here. (Daniel)

v5: Update the comment to not suck (Chris)

v6:
Move bind/unbind to the VMA. It makes more sense in the VMA structure
(always has, but I was previously lazy). With this change, it will allow
us to keep a distinct insert_entries.

Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Ben Widawsky <ben@bwidawsk.net>

drm/i915: Use the new vm [un]bind functions

Building on the last patch which created the new function pointers in
the VM for bind/unbind, here we actually put those new function pointers
to use.

Split out as a separate patch to aid in review. I'm fine with squashing
into the previous patch if people request it.

v2: Updated to address the smart ggtt which can do aliasing as needed
Make sure we bind to global gtt when mappable and fenceable. I thought
we could get away without this initialy, but we cannot.

v3: Make the global GTT binding explicitly use the ggtt VM for
bind_vma(). While at it, use the new ggtt_vma helper (Chris)

At this point the original mailing list thread diverges. ie.

v4^:
use target_obj instead of obj for gen6 relocate_entry
vma->bind_vma() can be called safely during pin. So simply do that
instead of the complicated conditionals.
Don't restore PPGTT bound objects on resume path
Bug fix in resume path for globally bound Bos
Properly handle secure dispatch
Rebased on vma bind/unbind conversion

Signed-off-by: Ben Widawsky <ben@bwidawsk.net>

drm/i915: reduce vm->insert_entries() usage

FKA: drm/i915: eliminate vm->insert_entries()

With bind/unbind function pointers in place, we no longer need
insert_entries. We could, and want, to remove clear_range, however it's
not totally easy at this point. Since it's used in a couple of place
still that don't only deal in objects: setup, ppgtt init, and restore
gtt mappings.

v2: Don't actually remove insert_entries, just limit its usage. It will
be useful when we introduce gen8. It will always be called from the vma
bind/unbind.

Reviewed-by: Chris Wilson <chris@chris-wilson.co.uk> (v1)
Signed-off-by: Ben Widawsky <ben@bwidawsk.net>
Signed-off-by: Daniel Vetter <daniel.vetter@ffwll.ch>

1 files changed, 155 insertions, 37 deletions

diff --git a/drivers/gpu/drm/i915/i915_gem_gtt.c b/drivers/gpu/drm/i915/i915_gem_gtt.c
index 056033791d24..73117ec333cc 100644
--- a/drivers/gpu/drm/i915/i915_gem_gtt.c
+++ b/drivers/gpu/drm/i915/i915_gem_gtt.c
@@ -68,6 +68,11 @@ typedef gen8_gtt_pte_t gen8_ppgtt_pde_t;
 #define PPAT_CACHED_INDEX               _PAGE_PAT /* WB LLCeLLC */
 #define PPAT_DISPLAY_ELLC_INDEX         _PAGE_PCD /* WT eLLC */
 
+static void ppgtt_bind_vma(struct i915_vma *vma,
+                           enum i915_cache_level cache_level,
+                           u32 flags);
+static void ppgtt_unbind_vma(struct i915_vma *vma);
+
 static inline gen8_gtt_pte_t gen8_pte_encode(dma_addr_t addr,
                                              enum i915_cache_level level,
                                              bool valid)
@@ -746,22 +751,26 @@ void i915_gem_cleanup_aliasing_ppgtt(struct drm_device *dev)
         dev_priv->mm.aliasing_ppgtt = NULL;
 }
 
-void i915_ppgtt_bind_object(struct i915_hw_ppgtt *ppgtt,
-                            struct drm_i915_gem_object *obj,
-                            enum i915_cache_level cache_level)
+static void __always_unused
+ppgtt_bind_vma(struct i915_vma *vma,
+               enum i915_cache_level cache_level,
+               u32 flags)
 {
-        ppgtt->base.insert_entries(&ppgtt->base, obj->pages,
-                                   i915_gem_obj_ggtt_offset(obj) >> PAGE_SHIFT,
-                                   cache_level);
+        const unsigned long entry = vma->node.start >> PAGE_SHIFT;
+
+        WARN_ON(flags);
+
+        vma->vm->insert_entries(vma->vm, vma->obj->pages, entry, cache_level);
 }
 
-void i915_ppgtt_unbind_object(struct i915_hw_ppgtt *ppgtt,
-                              struct drm_i915_gem_object *obj)
+static void __always_unused ppgtt_unbind_vma(struct i915_vma *vma)
 {
-        ppgtt->base.clear_range(&ppgtt->base,
-                                i915_gem_obj_ggtt_offset(obj) >> PAGE_SHIFT,
-                                obj->base.size >> PAGE_SHIFT,
-                                true);
+        const unsigned long entry = vma->node.start >> PAGE_SHIFT;
+
+        vma->vm->clear_range(vma->vm,
+                             entry,
+                             vma->obj->base.size >> PAGE_SHIFT,
+                             true);
 }
 
 extern int intel_iommu_gfx_mapped;
@@ -863,8 +872,18 @@ void i915_gem_restore_gtt_mappings(struct drm_device *dev)
                                        true);
 
         list_for_each_entry(obj, &dev_priv->mm.bound_list, global_list) {
+                struct i915_vma *vma = i915_gem_obj_to_vma(obj,
+                                                           &dev_priv->gtt.base);
+                if (!vma)
+                        continue;
+
                 i915_gem_clflush_object(obj, obj->pin_display);
-                i915_gem_gtt_bind_object(obj, obj->cache_level);
+                /* The bind_vma code tries to be smart about tracking mappings.
+                 * Unfortunately above, we've just wiped out the mappings
+                 * without telling our object about it. So we need to fake it.
+                 */
+                obj->has_global_gtt_mapping = 0;
+                vma->bind_vma(vma, obj->cache_level, GLOBAL_BIND);
         }
 
         i915_gem_chipset_flush(dev);
@@ -1023,16 +1042,18 @@ static void gen6_ggtt_clear_range(struct i915_address_space *vm,
         readl(gtt_base);
 }
 
-static void i915_ggtt_insert_entries(struct i915_address_space *vm,
-                                     struct sg_table *st,
-                                     unsigned int pg_start,
-                                     enum i915_cache_level cache_level)
+
+static void i915_ggtt_bind_vma(struct i915_vma *vma,
+                               enum i915_cache_level cache_level,
+                               u32 unused)
 {
+        const unsigned long entry = vma->node.start >> PAGE_SHIFT;
         unsigned int flags = (cache_level == I915_CACHE_NONE) ?
                 AGP_USER_MEMORY : AGP_USER_CACHED_MEMORY;
 
-        intel_gtt_insert_sg_entries(st, pg_start, flags);
-
+        BUG_ON(!i915_is_ggtt(vma->vm));
+        intel_gtt_insert_sg_entries(vma->obj->pages, entry, flags);
+        vma->obj->has_global_gtt_mapping = 1;
 }
 
 static void i915_ggtt_clear_range(struct i915_address_space *vm,
@@ -1043,33 +1064,77 @@ static void i915_ggtt_clear_range(struct i915_address_space *vm,
         intel_gtt_clear_range(first_entry, num_entries);
 }
 
+static void i915_ggtt_unbind_vma(struct i915_vma *vma)
+{
+        const unsigned int first = vma->node.start >> PAGE_SHIFT;
+        const unsigned int size = vma->obj->base.size >> PAGE_SHIFT;
 
-void i915_gem_gtt_bind_object(struct drm_i915_gem_object *obj,
-                              enum i915_cache_level cache_level)
+        BUG_ON(!i915_is_ggtt(vma->vm));
+        vma->obj->has_global_gtt_mapping = 0;
+        intel_gtt_clear_range(first, size);
+}
+
+static void ggtt_bind_vma(struct i915_vma *vma,
+                          enum i915_cache_level cache_level,
+                          u32 flags)
 {
-        struct drm_device *dev = obj->base.dev;
+        struct drm_device *dev = vma->vm->dev;
         struct drm_i915_private *dev_priv = dev->dev_private;
-        const unsigned long entry = i915_gem_obj_ggtt_offset(obj) >> PAGE_SHIFT;
+        struct drm_i915_gem_object *obj = vma->obj;
+        const unsigned long entry = vma->node.start >> PAGE_SHIFT;
 
-        dev_priv->gtt.base.insert_entries(&dev_priv->gtt.base, obj->pages,
-                                          entry,
-                                          cache_level);
+        /* If there is no aliasing PPGTT, or the caller needs a global mapping,
+         * or we have a global mapping already but the cacheability flags have
+         * changed, set the global PTEs.
+         *
+         * If there is an aliasing PPGTT it is anecdotally faster, so use that
+         * instead if none of the above hold true.
+         *
+         * NB: A global mapping should only be needed for special regions like
+         * "gtt mappable", SNB errata, or if specified via special execbuf
+         * flags. At all other times, the GPU will use the aliasing PPGTT.
+         */
+        if (!dev_priv->mm.aliasing_ppgtt || flags & GLOBAL_BIND) {
+                if (!obj->has_global_gtt_mapping ||
+                    (cache_level != obj->cache_level)) {
+                        vma->vm->insert_entries(vma->vm, obj->pages, entry,
+                                                cache_level);
+                        obj->has_global_gtt_mapping = 1;
+                }
+        }
 
-        obj->has_global_gtt_mapping = 1;
+        if (dev_priv->mm.aliasing_ppgtt &&
+            (!obj->has_aliasing_ppgtt_mapping ||
+             (cache_level != obj->cache_level))) {
+                struct i915_hw_ppgtt *appgtt = dev_priv->mm.aliasing_ppgtt;
+                appgtt->base.insert_entries(&appgtt->base,
+                                            vma->obj->pages, entry, cache_level);
+                vma->obj->has_aliasing_ppgtt_mapping = 1;
+        }
 }
 
-void i915_gem_gtt_unbind_object(struct drm_i915_gem_object *obj)
+static void ggtt_unbind_vma(struct i915_vma *vma)
 {
-        struct drm_device *dev = obj->base.dev;
+        struct drm_device *dev = vma->vm->dev;
         struct drm_i915_private *dev_priv = dev->dev_private;
-        const unsigned long entry = i915_gem_obj_ggtt_offset(obj) >> PAGE_SHIFT;
-
-        dev_priv->gtt.base.clear_range(&dev_priv->gtt.base,
-                                       entry,
-                                       obj->base.size >> PAGE_SHIFT,
-                                       true);
+        struct drm_i915_gem_object *obj = vma->obj;
+        const unsigned long entry = vma->node.start >> PAGE_SHIFT;
+
+        if (obj->has_global_gtt_mapping) {
+                vma->vm->clear_range(vma->vm, entry,
+                                     vma->obj->base.size >> PAGE_SHIFT,
+                                     true);
+                obj->has_global_gtt_mapping = 0;
+        }
 
-        obj->has_global_gtt_mapping = 0;
+        if (obj->has_aliasing_ppgtt_mapping) {
+                struct i915_hw_ppgtt *appgtt = dev_priv->mm.aliasing_ppgtt;
+                appgtt->base.clear_range(&appgtt->base,
+                                         entry,
+                                         obj->base.size >> PAGE_SHIFT,
+                                         true);
+                obj->has_aliasing_ppgtt_mapping = 0;
+        }
 }
 
 void i915_gem_gtt_finish_object(struct drm_i915_gem_object *obj)
@@ -1444,7 +1509,6 @@ static int i915_gmch_probe(struct drm_device *dev,
 
         dev_priv->gtt.do_idle_maps = needs_idle_maps(dev_priv->dev);
         dev_priv->gtt.base.clear_range = i915_ggtt_clear_range;
-        dev_priv->gtt.base.insert_entries = i915_ggtt_insert_entries;
 
         return 0;
 }
@@ -1496,3 +1560,57 @@ int i915_gem_gtt_init(struct drm_device *dev)
 
         return 0;
 }
+
+static struct i915_vma *__i915_gem_vma_create(struct drm_i915_gem_object *obj,
+                                              struct i915_address_space *vm)
+{
+        struct i915_vma *vma = kzalloc(sizeof(*vma), GFP_KERNEL);
+        if (vma == NULL)
+                return ERR_PTR(-ENOMEM);
+
+        INIT_LIST_HEAD(&vma->vma_link);
+        INIT_LIST_HEAD(&vma->mm_list);
+        INIT_LIST_HEAD(&vma->exec_list);
+        vma->vm = vm;
+        vma->obj = obj;
+
+        switch (INTEL_INFO(vm->dev)->gen) {
+        case 8:
+        case 7:
+        case 6:
+                vma->unbind_vma = ggtt_unbind_vma;
+                vma->bind_vma = ggtt_bind_vma;
+                break;
+        case 5:
+        case 4:
+        case 3:
+        case 2:
+                BUG_ON(!i915_is_ggtt(vm));
+                vma->unbind_vma = i915_ggtt_unbind_vma;
+                vma->bind_vma = i915_ggtt_bind_vma;
+                break;
+        default:
+                BUG();
+        }
+
+        /* Keep GGTT vmas first to make debug easier */
+        if (i915_is_ggtt(vm))
+                list_add(&vma->vma_link, &obj->vma_list);
+        else
+                list_add_tail(&vma->vma_link, &obj->vma_list);
+
+        return vma;
+}
+
+struct i915_vma *
+i915_gem_obj_lookup_or_create_vma(struct drm_i915_gem_object *obj,
+                                  struct i915_address_space *vm)
+{
+        struct i915_vma *vma;
+
+        vma = i915_gem_obj_to_vma(obj, vm);
+        if (!vma)
+                vma = __i915_gem_vma_create(obj, vm);
+
+        return vma;
+}