gpu: nvgpu: add worker for watchdog and job cleanup

Implement a worker thread to replace the delayed works in channel
watchdog and job cleanups. Watchdog runs by polling the channel states
periodically, and job cleanup is performed on channels that are appended
on a work queue consumed by the worker thread. Handling both of these
two in the same thread makes it impossible for them to cause a deadlock,
as has previously happened.

The watchdog takes references to channels during checking and possibly
recovering channels. Jobs in the cleanup queue have an additional
reference taken which is released after the channel is processed. The
worker is woken up from periodic sleep when channels are added to the
queue.

Currently, the queue is only used for job cleanups, but it is extendable
for other per-channel works too. The worker can also process other
periodic actions dependent on channels.

Neither the semantics of timeout handling or of job cleanups are yet
significantly changed - this patch only serializes them into one
background thread.

Each job that needs cleanup is tracked and holds a reference to its
channel and a power reference, and timeouts can only be processed on
channels that are tracked, so the thread will always be idle if the
system is going to be suspended, so there is currently no need to
explicitly suspend or stop it.

Bug 1848834
Bug 1851689
Bug 1814773
Bug 200270332
Jira NVGPU-21

Change-Id: I355101802f50841ea9bd8042a017f91c931d2dc7
Signed-off-by: Konsta Holtta <kholtta@nvidia.com>
Reviewed-on: http://git-master/r/1297183
Reviewed-by: mobile promotions <svcmobile_promotions@nvidia.com>
Tested-by: mobile promotions <svcmobile_promotions@nvidia.com>

1 files changed, 4 insertions, 0 deletions

diff --git a/drivers/gpu/nvgpu/gk20a/fifo_gk20a.c b/drivers/gpu/nvgpu/gk20a/fifo_gk20a.c
index 95351a43..e9eab551 100644
--- a/drivers/gpu/nvgpu/gk20a/fifo_gk20a.c
+++ b/drivers/gpu/nvgpu/gk20a/fifo_gk20a.c
@@ -502,6 +502,7 @@ static void gk20a_remove_fifo_support(struct fifo_gk20a *f)
 
         gk20a_dbg_fn("");
 
+        nvgpu_channel_worker_deinit(g);
         /*
          * Make sure all channels are closed before deleting them.
          */
@@ -900,6 +901,9 @@ static int gk20a_init_fifo_setup_sw(struct gk20a *g)
         }
         nvgpu_mutex_init(&f->tsg_inuse_mutex);
 
+        err = nvgpu_channel_worker_init(g);
+        if (err)
+                goto clean_up;
         f->remove_support = gk20a_remove_fifo_support;
 
         f->deferred_reset_pending = false;