drm: add support for tiled/compressed/etc modifier in addfb2

In DRM/KMS we are lacking a good way to deal with tiled/compressed
formats.  Especially in the case of dmabuf/prime buffer sharing, where
we cannot always rely on under-the-hood flags passed to driver specific
gem-create ioctl to pass around these extra flags.

The proposal is to add a per-plane format modifier.  This allows to, if
necessary, use different tiling patters for sub-sampled planes, etc.
The format modifiers are added at the end of the ioctl struct, so for
legacy userspace it will be zero padded.

v1: original
v1.5: increase modifier to 64b

v2: Incorporate review comments from the big thread, plus a few more.

- Add a getcap so that userspace doesn't have to jump through hoops.
- Allow modifiers only when a flag is set. That way drivers know when
  they're dealing with old userspace and need to fish out e.g. tiling
  from other information.
- After rolling out checks for ->modifier to all drivers I've decided
  that this is way too fragile and needs an explicit opt-in flag. So
  do that instead.
- Add a define (just for documentation really) for the "NONE"
  modifier. Imo we don't need to add mask #defines since drivers
  really should only do exact matches against values defined with
  fourcc_mod_code.
- Drop the Samsung tiling modifier on Rob's request since he's not yet
  sure whether that one is accurate.

v3:
- Also add a new ->modifier[] array to struct drm_framebuffer and fill
  it in drm_helper_mode_fill_fb_struct. Requested by Tvrkto Uruslin.
- Remove TODO in comment and add code comment that modifiers should be
  properly documented, requested by Rob.

Cc: Rob Clark <robdclark@gmail.com>
Cc: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Daniel Stone <daniel@fooishbar.org>
Cc: Ville Syrjälä <ville.syrjala@linux.intel.com>
Cc: Michel Dänzer <michel@daenzer.net>
Signed-off-by: Rob Clark <robdclark@gmail.com> (v1.5)
Reviewed-by: Rob Clark <robdclark@gmail.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Acked-by: Dave Airlie <airlied@redhat.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>

3 files changed, 17 insertions, 1 deletions

diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
index ad2934ba0bd2..b15d720eda4c 100644
--- a/drivers/gpu/drm/drm_crtc.c
+++ b/drivers/gpu/drm/drm_crtc.c
@@ -3314,6 +3314,12 @@ static int framebuffer_check(const struct drm_mode_fb_cmd2 *r)
                         DRM_DEBUG_KMS("bad pitch %u for plane %d\n", r->pitches[i], i);
                         return -EINVAL;
                 }
+
+                if (r->modifier[i] && !(r->flags & DRM_MODE_FB_MODIFIERS)) {
+                        DRM_DEBUG_KMS("bad fb modifier %llu for plane %d\n",
+                                      r->modifier[i], i);
+                        return -EINVAL;
+                }
         }
 
         return 0;
@@ -3327,7 +3333,7 @@ static struct drm_framebuffer *add_framebuffer_internal(struct drm_device *dev,
         struct drm_framebuffer *fb;
         int ret;
 
-        if (r->flags & ~DRM_MODE_FB_INTERLACED) {
+        if (r->flags & ~(DRM_MODE_FB_INTERLACED | DRM_MODE_FB_MODIFIERS)) {
                 DRM_DEBUG_KMS("bad framebuffer flags 0x%08x\n", r->flags);
                 return ERR_PTR(-EINVAL);
         }
@@ -3343,6 +3349,12 @@ static struct drm_framebuffer *add_framebuffer_internal(struct drm_device *dev,
                 return ERR_PTR(-EINVAL);
         }
 
+        if (r->flags & DRM_MODE_FB_MODIFIERS &&
+            !dev->mode_config.allow_fb_modifiers) {
+                DRM_DEBUG_KMS("driver does not support fb modifiers\n");
+                return ERR_PTR(-EINVAL);
+        }
+
         ret = framebuffer_check(r);
         if (ret)
                 return ERR_PTR(ret);
diff --git a/drivers/gpu/drm/drm_crtc_helper.c b/drivers/gpu/drm/drm_crtc_helper.c
index b1979e7bdc88..3053aab968f9 100644
--- a/drivers/gpu/drm/drm_crtc_helper.c
+++ b/drivers/gpu/drm/drm_crtc_helper.c
@@ -837,6 +837,7 @@ void drm_helper_mode_fill_fb_struct(struct drm_framebuffer *fb,
         for (i = 0; i < 4; i++) {
                 fb->pitches[i] = mode_cmd->pitches[i];
                 fb->offsets[i] = mode_cmd->offsets[i];
+                fb->modifier[i] = mode_cmd->modifier[i];
         }
         drm_fb_get_bpp_depth(mode_cmd->pixel_format, &fb->depth,
                                     &fb->bits_per_pixel);
diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c
index 3785d66721f2..a6d773a61c2d 100644
--- a/drivers/gpu/drm/drm_ioctl.c
+++ b/drivers/gpu/drm/drm_ioctl.c
@@ -321,6 +321,9 @@ static int drm_getcap(struct drm_device *dev, void *data, struct drm_file *file_
                 else
                         req->value = 64;
                 break;
+        case DRM_CAP_ADDFB2_MODIFIERS:
+                req->value = dev->mode_config.allow_fb_modifiers;
+                break;
         default:
                 return -EINVAL;
         }