gpu: nvgpu: handle ioctl l2_fb_ops better

Background: There is a race that occurs when l2_fb_ops ioctl is
invoked. The race occurs as part of the flush() call while a
gk20_idle() is in progress.

This patch handles the race by making changes in the l2_fb_ops
ioctl itself. For cases where pm_runtime is disabled or railgate is
disabled, we allow this ioctl call to always go ahead as power is
assumed to be always on.

For the other case, we first check the status of g->power_on. In the
driver, g->power_on is set to true, once unrailgate is completed and is
set to false just before calling railgate.

For linux, the driver invokes gk20a_idle() but there is a delay after
which the call to the rpm_suspend()'s callback gets triggered. This
leads to a scenario where we cannot efficiently rely on the
runtime_pm's APIs to allow us to block an imminent suspend or exit if
the suspend is currently in progress. Previous attempts at solving this
has lead to ineffective solutions and make it much complicated to
maintain the code.

With regards to the above, this patch attempts to simplify the way this
can be solved. The patch calls gk20a_busy() when g->power_on = true.
This prevents the race with gk20a_idle(). Based on the rpm_resume and
rpm_suspend's upstream code, resume is prioritized over a suspend
unless a suspend is already in progress i.e. the delay period has been
served and the suspend invokes the callback. There is a very small
window for this to happen and the ioctl can then power_up the device as
evident from the gk20a_busy's calls.

A new function gk20a_check_poweron() is added. This function protects
the access to g->power_on via a mutex. By preventing a read from
happening simulatenously as a write on g->power_on, the likelihood of
an runtime_suspend triggering before a runtime_resume is further
reduced.

Bug 200507468

Change-Id: I5c02dfa8ea855732e59b759d167152cf45a1131f
Signed-off-by:Debarshi Dutta <ddutta@nvidia.com>
Reviewed-on: https://git-master.nvidia.com/r/c/linux-nvgpu/+/2299545
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>

1 files changed, 30 insertions, 2 deletions

diff --git a/drivers/gpu/nvgpu/os/linux/ioctl_ctrl.c b/drivers/gpu/nvgpu/os/linux/ioctl_ctrl.c
index 6fa15421..271c5d92 100644
--- a/drivers/gpu/nvgpu/os/linux/ioctl_ctrl.c
+++ b/drivers/gpu/nvgpu/os/linux/ioctl_ctrl.c
@@ -19,6 +19,7 @@
 #include <linux/file.h>
 #include <linux/anon_inodes.h>
 #include <linux/fs.h>
+#include <linux/pm_runtime.h>
 #include <uapi/linux/nvgpu.h>
 
 #include <nvgpu/bitops.h>
@@ -596,19 +597,46 @@ static int gk20a_ctrl_get_fbp_l2_masks(
 static int nvgpu_gpu_ioctl_l2_fb_ops(struct gk20a *g,
                 struct nvgpu_gpu_l2_fb_args *args)
 {
-        int err = 0;
+        int ret;
+        bool always_poweron;
 
         if ((!args->l2_flush && !args->fb_flush) ||
             (!args->l2_flush && args->l2_invalidate))
                 return -EINVAL;
 
+        /* Handle this case for joint rails or DGPU */
+        always_poweron = (!nvgpu_is_enabled(g, NVGPU_CAN_RAILGATE) ||
+                                !pm_runtime_enabled(dev_from_gk20a(g)));
+
+        /* In case of not always power_on, exit if g->power_on is false */
+        if (!always_poweron && !gk20a_check_poweron(g)) {
+                return 0;
+        }
+
+        /* There is a small window between a call to gk20a_idle() has occured
+         * and railgate being actually triggered(setting g->power_on = false),
+         * when l2_flush can race with railgate. Its better to take a busy_lock
+         * to prevent the gk20a_idle() from proceeding. There is a very small
+         * chance that gk20a_idle() might begin before gk20a_busy(). Having
+         * a locked access to g->power_on further reduces the probability of
+         * gk20a_idle() being triggered before gk20a_busy()
+         */
+        ret = gk20a_busy(g);
+
+        if (ret != 0) {
+                nvgpu_err(g, "failed to take power ref");
+                return ret;
+        }
+
         if (args->l2_flush)
                 g->ops.mm.l2_flush(g, args->l2_invalidate ? true : false);
 
         if (args->fb_flush)
                 g->ops.mm.fb_flush(g);
 
-        return err;
+        gk20a_idle(g);
+
+        return 0;
 }
 
 static int nvgpu_gpu_ioctl_set_mmu_debug_mode(