Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/suspend-2.6

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/suspend-2.6:
  PM: Runtime PM documentation update
  PM / Runtime: Use device type and device class callbacks
  PM: Use pm_runtime_put_sync in system resume
  PM: Measure device suspend and resume times
  PM: Make the initcall_debug style timing for suspend/resume complete

3 files changed, 288 insertions, 108 deletions

diff --git a/Documentation/power/runtime_pm.txt b/Documentation/power/runtime_pm.txt
index 4a3109b28847..356fd86f4ea8 100644
--- a/Documentation/power/runtime_pm.txt
+++ b/Documentation/power/runtime_pm.txt
@@ -42,80 +42,81 @@ struct dev_pm_ops {
         ...
 };
 
-The ->runtime_suspend() callback is executed by the PM core for the bus type of
-the device being suspended.  The bus type's callback is then _entirely_
-_responsible_ for handling the device as appropriate, which may, but need not
-include executing the device driver's own ->runtime_suspend() callback (from the
+The ->runtime_suspend(), ->runtime_resume() and ->runtime_idle() callbacks are
+executed by the PM core for either the bus type, or device type (if the bus
+type's callback is not defined), or device class (if the bus type's and device
+type's callbacks are not defined) of given device.  The bus type, device type
+and device class callbacks are referred to as subsystem-level callbacks in what
+follows.
+
+The subsystem-level suspend callback is _entirely_ _responsible_ for handling
+the suspend of the device as appropriate, which may, but need not include
+executing the device driver's own ->runtime_suspend() callback (from the
 PM core's point of view it is not necessary to implement a ->runtime_suspend()
-callback in a device driver as long as the bus type's ->runtime_suspend() knows
-what to do to handle the device).
+callback in a device driver as long as the subsystem-level suspend callback
+knows what to do to handle the device).
 
-  * Once the bus type's ->runtime_suspend() callback has completed successfully
+  * Once the subsystem-level suspend callback has completed successfully
     for given device, the PM core regards the device as suspended, which need
     not mean that the device has been put into a low power state.  It is
     supposed to mean, however, that the device will not process data and will
-    not communicate with the CPU(s) and RAM until its bus type's
-    ->runtime_resume() callback is executed for it.  The run-time PM status of
-    a device after successful execution of its bus type's ->runtime_suspend()
-    callback is 'suspended'.
-
-  * If the bus type's ->runtime_suspend() callback returns -EBUSY or -EAGAIN,
-    the device's run-time PM status is supposed to be 'active', which means that
-    the device _must_ be fully operational afterwards.
-
-  * If the bus type's ->runtime_suspend() callback returns an error code
-    different from -EBUSY or -EAGAIN, the PM core regards this as a fatal
-    error and will refuse to run the helper functions described in Section 4
-    for the device, until the status of it is directly set either to 'active'
-    or to 'suspended' (the PM core provides special helper functions for this
-    purpose).
-
-In particular, if the driver requires remote wakeup capability for proper
-functioning and device_run_wake() returns 'false' for the device, then
-->runtime_suspend() should return -EBUSY.  On the other hand, if
-device_run_wake() returns 'true' for the device and the device is put
-into a low power state during the execution of its bus type's
-->runtime_suspend(), it is expected that remote wake-up (i.e. hardware mechanism
-allowing the device to request a change of its power state, such as PCI PME)
-will be enabled for the device.  Generally, remote wake-up should be enabled
-for all input devices put into a low power state at run time.
-
-The ->runtime_resume() callback is executed by the PM core for the bus type of
-the device being woken up.  The bus type's callback is then _entirely_
-_responsible_ for handling the device as appropriate, which may, but need not
-include executing the device driver's own ->runtime_resume() callback (from the
-PM core's point of view it is not necessary to implement a ->runtime_resume()
-callback in a device driver as long as the bus type's ->runtime_resume() knows
-what to do to handle the device).
-
-  * Once the bus type's ->runtime_resume() callback has completed successfully,
-    the PM core regards the device as fully operational, which means that the
-    device _must_ be able to complete I/O operations as needed.  The run-time
-    PM status of the device is then 'active'.
-
-  * If the bus type's ->runtime_resume() callback returns an error code, the PM
-    core regards this as a fatal error and will refuse to run the helper
-    functions described in Section 4 for the device, until its status is
-    directly set either to 'active' or to 'suspended' (the PM core provides
-    special helper functions for this purpose).
-
-The ->runtime_idle() callback is executed by the PM core for the bus type of
-given device whenever the device appears to be idle, which is indicated to the
-PM core by two counters, the device's usage counter and the counter of 'active'
-children of the device.
+    not communicate with the CPU(s) and RAM until the subsystem-level resume
+    callback is executed for it.  The run-time PM status of a device after
+    successful execution of the subsystem-level suspend callback is 'suspended'.
+
+  * If the subsystem-level suspend callback returns -EBUSY or -EAGAIN,
+    the device's run-time PM status is 'active', which means that the device
+    _must_ be fully operational afterwards.
+
+  * If the subsystem-level suspend callback returns an error code different
+    from -EBUSY or -EAGAIN, the PM core regards this as a fatal error and will
+    refuse to run the helper functions described in Section 4 for the device,
+    until the status of it is directly set either to 'active', or to 'suspended'
+    (the PM core provides special helper functions for this purpose).
+
+In particular, if the driver requires remote wake-up capability (i.e. hardware
+mechanism allowing the device to request a change of its power state, such as
+PCI PME) for proper functioning and device_run_wake() returns 'false' for the
+device, then ->runtime_suspend() should return -EBUSY.  On the other hand, if
+device_run_wake() returns 'true' for the device and the device is put into a low
+power state during the execution of the subsystem-level suspend callback, it is
+expected that remote wake-up will be enabled for the device.  Generally, remote
+wake-up should be enabled for all input devices put into a low power state at
+run time.
+
+The subsystem-level resume callback is _entirely_ _responsible_ for handling the
+resume of the device as appropriate, which may, but need not include executing
+the device driver's own ->runtime_resume() callback (from the PM core's point of
+view it is not necessary to implement a ->runtime_resume() callback in a device
+driver as long as the subsystem-level resume callback knows what to do to handle
+the device).
+
+  * Once the subsystem-level resume callback has completed successfully, the PM
+    core regards the device as fully operational, which means that the device
+    _must_ be able to complete I/O operations as needed.  The run-time PM status
+    of the device is then 'active'.
+
+  * If the subsystem-level resume callback returns an error code, the PM core
+    regards this as a fatal error and will refuse to run the helper functions
+    described in Section 4 for the device, until its status is directly set
+    either to 'active' or to 'suspended' (the PM core provides special helper
+    functions for this purpose).
+
+The subsystem-level idle callback is executed by the PM core whenever the device
+appears to be idle, which is indicated to the PM core by two counters, the
+device's usage counter and the counter of 'active' children of the device.
 
   * If any of these counters is decreased using a helper function provided by
     the PM core and it turns out to be equal to zero, the other counter is
     checked.  If that counter also is equal to zero, the PM core executes the
-    device bus type's ->runtime_idle() callback (with the device as an
-    argument).
+    subsystem-level idle callback with the device as an argument.
 
-The action performed by a bus type's ->runtime_idle() callback is totally
-dependent on the bus type in question, but the expected and recommended action
-is to check if the device can be suspended (i.e. if all of the conditions
-necessary for suspending the device are satisfied) and to queue up a suspend
-request for the device in that case.  The value returned by this callback is
-ignored by the PM core.
+The action performed by a subsystem-level idle callback is totally dependent on
+the subsystem in question, but the expected and recommended action is to check
+if the device can be suspended (i.e. if all of the conditions necessary for
+suspending the device are satisfied) and to queue up a suspend request for the
+device in that case.  The value returned by this callback is ignored by the PM
+core.
 
 The helper functions provided by the PM core, described in Section 4, guarantee
 that the following constraints are met with respect to the bus type's run-time
@@ -238,41 +239,41 @@ drivers/base/power/runtime.c and include/linux/pm_runtime.h:
       removing the device from device hierarchy
 
   int pm_runtime_idle(struct device *dev);
-    - execute ->runtime_idle() for the device's bus type; returns 0 on success
-      or error code on failure, where -EINPROGRESS means that ->runtime_idle()
-      is already being executed
+    - execute the subsystem-level idle callback for the device; returns 0 on
+      success or error code on failure, where -EINPROGRESS means that
+      ->runtime_idle() is already being executed
 
   int pm_runtime_suspend(struct device *dev);
-    - execute ->runtime_suspend() for the device's bus type; returns 0 on
+    - execute the subsystem-level suspend callback for the device; returns 0 on
       success, 1 if the device's run-time PM status was already 'suspended', or
       error code on failure, where -EAGAIN or -EBUSY means it is safe to attempt
       to suspend the device again in future
 
   int pm_runtime_resume(struct device *dev);
-    - execute ->runtime_resume() for the device's bus type; returns 0 on
+    - execute the subsystem-leve resume callback for the device; returns 0 on
       success, 1 if the device's run-time PM status was already 'active' or
       error code on failure, where -EAGAIN means it may be safe to attempt to
       resume the device again in future, but 'power.runtime_error' should be
       checked additionally
 
   int pm_request_idle(struct device *dev);
-    - submit a request to execute ->runtime_idle() for the device's bus type
-      (the request is represented by a work item in pm_wq); returns 0 on success
-      or error code if the request has not been queued up
+    - submit a request to execute the subsystem-level idle callback for the
+      device (the request is represented by a work item in pm_wq); returns 0 on
+      success or error code if the request has not been queued up
 
   int pm_schedule_suspend(struct device *dev, unsigned int delay);
-    - schedule the execution of ->runtime_suspend() for the device's bus type
-      in future, where 'delay' is the time to wait before queuing up a suspend
-      work item in pm_wq, in milliseconds (if 'delay' is zero, the work item is
-      queued up immediately); returns 0 on success, 1 if the device's PM
+    - schedule the execution of the subsystem-level suspend callback for the
+      device in future, where 'delay' is the time to wait before queuing up a
+      suspend work item in pm_wq, in milliseconds (if 'delay' is zero, the work
+      item is queued up immediately); returns 0 on success, 1 if the device's PM
       run-time status was already 'suspended', or error code if the request
       hasn't been scheduled (or queued up if 'delay' is 0); if the execution of
       ->runtime_suspend() is already scheduled and not yet expired, the new
       value of 'delay' will be used as the time to wait
 
   int pm_request_resume(struct device *dev);
-    - submit a request to execute ->runtime_resume() for the device's bus type
-      (the request is represented by a work item in pm_wq); returns 0 on
+    - submit a request to execute the subsystem-level resume callback for the
+      device (the request is represented by a work item in pm_wq); returns 0 on
       success, 1 if the device's run-time PM status was already 'active', or
       error code if the request hasn't been queued up
 
@@ -303,12 +304,12 @@ drivers/base/power/runtime.c and include/linux/pm_runtime.h:
       run-time PM callbacks described in Section 2
 
   int pm_runtime_disable(struct device *dev);
-    - prevent the run-time PM helper functions from running the device bus
-      type's run-time PM callbacks, make sure that all of the pending run-time
-      PM operations on the device are either completed or canceled; returns
-      1 if there was a resume request pending and it was necessary to execute
-      ->runtime_resume() for the device's bus type to satisfy that request,
-      otherwise 0 is returned
+    - prevent the run-time PM helper functions from running subsystem-level
+      run-time PM callbacks for the device, make sure that all of the pending
+      run-time PM operations on the device are either completed or canceled;
+      returns 1 if there was a resume request pending and it was necessary to
+      execute the subsystem-level resume callback for the device to satisfy that
+      request, otherwise 0 is returned
 
   void pm_suspend_ignore_children(struct device *dev, bool enable);
     - set/unset the power.ignore_children flag of the device
@@ -378,5 +379,55 @@ pm_runtime_suspend() or pm_runtime_idle() or their asynchronous counterparts,
 they will fail returning -EAGAIN, because the device's usage counter is
 incremented by the core before executing ->probe() and ->remove().  Still, it
 may be desirable to suspend the device as soon as ->probe() or ->remove() has
-finished, so the PM core uses pm_runtime_idle_sync() to invoke the device bus
-type's ->runtime_idle() callback at that time.
+finished, so the PM core uses pm_runtime_idle_sync() to invoke the
+subsystem-level idle callback for the device at that time.
+
+6. Run-time PM and System Sleep
+
+Run-time PM and system sleep (i.e., system suspend and hibernation, also known
+as suspend-to-RAM and suspend-to-disk) interact with each other in a couple of
+ways.  If a device is active when a system sleep starts, everything is
+straightforward.  But what should happen if the device is already suspended?
+
+The device may have different wake-up settings for run-time PM and system sleep.
+For example, remote wake-up may be enabled for run-time suspend but disallowed
+for system sleep (device_may_wakeup(dev) returns 'false').  When this happens,
+the subsystem-level system suspend callback is responsible for changing the
+device's wake-up setting (it may leave that to the device driver's system
+suspend routine).  It may be necessary to resume the device and suspend it again
+in order to do so.  The same is true if the driver uses different power levels
+or other settings for run-time suspend and system sleep.
+
+During system resume, devices generally should be brought back to full power,
+even if they were suspended before the system sleep began.  There are several
+reasons for this, including:
+
+  * The device might need to switch power levels, wake-up settings, etc.
+
+  * Remote wake-up events might have been lost by the firmware.
+
+  * The device's children may need the device to be at full power in order
+    to resume themselves.
+
+  * The driver's idea of the device state may not agree with the device's
+    physical state.  This can happen during resume from hibernation.
+
+  * The device might need to be reset.
+
+  * Even though the device was suspended, if its usage counter was > 0 then most
+    likely it would need a run-time resume in the near future anyway.
+
+  * Always going back to full power is simplest.
+
+If the device was suspended before the sleep began, then its run-time PM status
+will have to be updated to reflect the actual post-system sleep status.  The way
+to do this is:
+
+        pm_runtime_disable(dev);
+        pm_runtime_set_active(dev);
+        pm_runtime_enable(dev);
+
+The PM core always increments the run-time usage counter before calling the
+->prepare() callback and decrements it after calling the ->complete() callback.
+Hence disabling run-time PM temporarily like this will not cause any run-time
+suspend callbacks to be lost.
diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c
index 1a216c114a0f..48adf80926a0 100644
--- a/drivers/base/power/main.c
+++ b/drivers/base/power/main.c
@@ -161,6 +161,32 @@ void device_pm_move_last(struct device *dev)
         list_move_tail(&dev->power.entry, &dpm_list);
 }
 
+static ktime_t initcall_debug_start(struct device *dev)
+{
+        ktime_t calltime = ktime_set(0, 0);
+
+        if (initcall_debug) {
+                pr_info("calling  %s+ @ %i\n",
+                                dev_name(dev), task_pid_nr(current));
+                calltime = ktime_get();
+        }
+
+        return calltime;
+}
+
+static void initcall_debug_report(struct device *dev, ktime_t calltime,
+                                  int error)
+{
+        ktime_t delta, rettime;
+
+        if (initcall_debug) {
+                rettime = ktime_get();
+                delta = ktime_sub(rettime, calltime);
+                pr_info("call %s+ returned %d after %Ld usecs\n", dev_name(dev),
+                        error, (unsigned long long)ktime_to_ns(delta) >> 10);
+        }
+}
+
 /**
  * pm_op - Execute the PM operation appropriate for given PM event.
  * @dev: Device to handle.
@@ -172,13 +198,9 @@ static int pm_op(struct device *dev,
                  pm_message_t state)
 {
         int error = 0;
-        ktime_t calltime, delta, rettime;
+        ktime_t calltime;
 
-        if (initcall_debug) {
-                pr_info("calling  %s+ @ %i\n",
-                                dev_name(dev), task_pid_nr(current));
-                calltime = ktime_get();
-        }
+        calltime = initcall_debug_start(dev);
 
         switch (state.event) {
 #ifdef CONFIG_SUSPEND
@@ -227,12 +249,7 @@ static int pm_op(struct device *dev,
                 error = -EINVAL;
         }
 
-        if (initcall_debug) {
-                rettime = ktime_get();
-                delta = ktime_sub(rettime, calltime);
-                pr_info("call %s+ returned %d after %Ld usecs\n", dev_name(dev),
-                        error, (unsigned long long)ktime_to_ns(delta) >> 10);
-        }
+        initcall_debug_report(dev, calltime, error);
 
         return error;
 }
@@ -309,8 +326,9 @@ static int pm_noirq_op(struct device *dev,
         if (initcall_debug) {
                 rettime = ktime_get();
                 delta = ktime_sub(rettime, calltime);
-                printk("initcall %s_i+ returned %d after %Ld usecs\n", dev_name(dev),
-                        error, (unsigned long long)ktime_to_ns(delta) >> 10);
+                printk("initcall %s_i+ returned %d after %Ld usecs\n",
+                        dev_name(dev), error,
+                        (unsigned long long)ktime_to_ns(delta) >> 10);
         }
 
         return error;
@@ -354,6 +372,23 @@ static void pm_dev_err(struct device *dev, pm_message_t state, char *info,
                 kobject_name(&dev->kobj), pm_verb(state.event), info, error);
 }
 
+static void dpm_show_time(ktime_t starttime, pm_message_t state, char *info)
+{
+        ktime_t calltime;
+        s64 usecs64;
+        int usecs;
+
+        calltime = ktime_get();
+        usecs64 = ktime_to_ns(ktime_sub(calltime, starttime));
+        do_div(usecs64, NSEC_PER_USEC);
+        usecs = usecs64;
+        if (usecs == 0)
+                usecs = 1;
+        pr_info("PM: %s%s%s of devices complete after %ld.%03ld msecs\n",
+                info ?: "", info ? " " : "", pm_verb(state.event),
+                usecs / USEC_PER_MSEC, usecs % USEC_PER_MSEC);
+}
+
 /*------------------------- Resume routines -------------------------*/
 
 /**
@@ -390,6 +425,7 @@ static int device_resume_noirq(struct device *dev, pm_message_t state)
 void dpm_resume_noirq(pm_message_t state)
 {
         struct device *dev;
+        ktime_t starttime = ktime_get();
 
         mutex_lock(&dpm_list_mtx);
         transition_started = false;
@@ -403,11 +439,32 @@ void dpm_resume_noirq(pm_message_t state)
                                 pm_dev_err(dev, state, " early", error);
                 }
         mutex_unlock(&dpm_list_mtx);
+        dpm_show_time(starttime, state, "early");
         resume_device_irqs();
 }
 EXPORT_SYMBOL_GPL(dpm_resume_noirq);
 
 /**
+ * legacy_resume - Execute a legacy (bus or class) resume callback for device.
+ * dev: Device to resume.
+ * cb: Resume callback to execute.
+ */
+static int legacy_resume(struct device *dev, int (*cb)(struct device *dev))
+{
+        int error;
+        ktime_t calltime;
+
+        calltime = initcall_debug_start(dev);
+
+        error = cb(dev);
+        suspend_report_result(cb, error);
+
+        initcall_debug_report(dev, calltime, error);
+
+        return error;
+}
+
+/**
  * device_resume - Execute "resume" callbacks for given device.
  * @dev: Device to handle.
  * @state: PM transition of the system being carried out.
@@ -427,7 +484,7 @@ static int device_resume(struct device *dev, pm_message_t state)
                         error = pm_op(dev, dev->bus->pm, state);
                 } else if (dev->bus->resume) {
                         pm_dev_dbg(dev, state, "legacy ");
-                        error = dev->bus->resume(dev);
+                        error = legacy_resume(dev, dev->bus->resume);
                 }
                 if (error)
                         goto End;
@@ -448,7 +505,7 @@ static int device_resume(struct device *dev, pm_message_t state)
                         error = pm_op(dev, dev->class->pm, state);
                 } else if (dev->class->resume) {
                         pm_dev_dbg(dev, state, "legacy class ");
-                        error = dev->class->resume(dev);
+                        error = legacy_resume(dev, dev->class->resume);
                 }
         }
  End:
@@ -468,6 +525,7 @@ static int device_resume(struct device *dev, pm_message_t state)
 static void dpm_resume(pm_message_t state)
 {
         struct list_head list;
+        ktime_t starttime = ktime_get();
 
         INIT_LIST_HEAD(&list);
         mutex_lock(&dpm_list_mtx);
@@ -496,6 +554,7 @@ static void dpm_resume(pm_message_t state)
         }
         list_splice(&list, &dpm_list);
         mutex_unlock(&dpm_list_mtx);
+        dpm_show_time(starttime, state, NULL);
 }
 
 /**
@@ -548,7 +607,7 @@ static void dpm_complete(pm_message_t state)
                         mutex_unlock(&dpm_list_mtx);
 
                         device_complete(dev, state);
-                        pm_runtime_put_noidle(dev);
+                        pm_runtime_put_sync(dev);
 
                         mutex_lock(&dpm_list_mtx);
                 }
@@ -628,6 +687,7 @@ static int device_suspend_noirq(struct device *dev, pm_message_t state)
 int dpm_suspend_noirq(pm_message_t state)
 {
         struct device *dev;
+        ktime_t starttime = ktime_get();
         int error = 0;
 
         suspend_device_irqs();
@@ -643,11 +703,34 @@ int dpm_suspend_noirq(pm_message_t state)
         mutex_unlock(&dpm_list_mtx);
         if (error)
                 dpm_resume_noirq(resume_event(state));
+        else
+                dpm_show_time(starttime, state, "late");
         return error;
 }
 EXPORT_SYMBOL_GPL(dpm_suspend_noirq);
 
 /**
+ * legacy_suspend - Execute a legacy (bus or class) suspend callback for device.
+ * dev: Device to suspend.
+ * cb: Suspend callback to execute.
+ */
+static int legacy_suspend(struct device *dev, pm_message_t state,
+                          int (*cb)(struct device *dev, pm_message_t state))
+{
+        int error;
+        ktime_t calltime;
+
+        calltime = initcall_debug_start(dev);
+
+        error = cb(dev, state);
+        suspend_report_result(cb, error);
+
+        initcall_debug_report(dev, calltime, error);
+
+        return error;
+}
+
+/**
  * device_suspend - Execute "suspend" callbacks for given device.
  * @dev: Device to handle.
  * @state: PM transition of the system being carried out.
@@ -664,8 +747,7 @@ static int device_suspend(struct device *dev, pm_message_t state)
                         error = pm_op(dev, dev->class->pm, state);
                 } else if (dev->class->suspend) {
                         pm_dev_dbg(dev, state, "legacy class ");
-                        error = dev->class->suspend(dev, state);
-                        suspend_report_result(dev->class->suspend, error);
+                        error = legacy_suspend(dev, state, dev->class->suspend);
                 }
                 if (error)
                         goto End;
@@ -686,8 +768,7 @@ static int device_suspend(struct device *dev, pm_message_t state)
                         error = pm_op(dev, dev->bus->pm, state);
                 } else if (dev->bus->suspend) {
                         pm_dev_dbg(dev, state, "legacy ");
-                        error = dev->bus->suspend(dev, state);
-                        suspend_report_result(dev->bus->suspend, error);
+                        error = legacy_suspend(dev, state, dev->bus->suspend);
                 }
         }
  End:
@@ -703,6 +784,7 @@ static int device_suspend(struct device *dev, pm_message_t state)
 static int dpm_suspend(pm_message_t state)
 {
         struct list_head list;
+        ktime_t starttime = ktime_get();
         int error = 0;
 
         INIT_LIST_HEAD(&list);
@@ -728,6 +810,8 @@ static int dpm_suspend(pm_message_t state)
         }
         list_splice(&list, dpm_list.prev);
         mutex_unlock(&dpm_list_mtx);
+        if (!error)
+                dpm_show_time(starttime, state, NULL);
         return error;
 }
 
@@ -796,7 +880,7 @@ static int dpm_prepare(pm_message_t state)
                 pm_runtime_get_noresume(dev);
                 if (pm_runtime_barrier(dev) && device_may_wakeup(dev)) {
                         /* Wake-up requested during system sleep transition. */
-                        pm_runtime_put_noidle(dev);
+                        pm_runtime_put_sync(dev);
                         error = -EBUSY;
                 } else {
                         error = device_prepare(dev, state);
diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c
index 40d7720a4b21..f8b044e8aef7 100644
--- a/drivers/base/power/runtime.c
+++ b/drivers/base/power/runtime.c
@@ -85,6 +85,19 @@ static int __pm_runtime_idle(struct device *dev)
                 dev->bus->pm->runtime_idle(dev);
 
                 spin_lock_irq(&dev->power.lock);
+        } else if (dev->type && dev->type->pm && dev->type->pm->runtime_idle) {
+                spin_unlock_irq(&dev->power.lock);
+
+                dev->type->pm->runtime_idle(dev);
+
+                spin_lock_irq(&dev->power.lock);
+        } else if (dev->class && dev->class->pm
+            && dev->class->pm->runtime_idle) {
+                spin_unlock_irq(&dev->power.lock);
+
+                dev->class->pm->runtime_idle(dev);
+
+                spin_lock_irq(&dev->power.lock);
         }
 
         dev->power.idle_notification = false;
@@ -194,6 +207,22 @@ int __pm_runtime_suspend(struct device *dev, bool from_wq)
 
                 spin_lock_irq(&dev->power.lock);
                 dev->power.runtime_error = retval;
+        } else if (dev->type && dev->type->pm
+            && dev->type->pm->runtime_suspend) {
+                spin_unlock_irq(&dev->power.lock);
+
+                retval = dev->type->pm->runtime_suspend(dev);
+
+                spin_lock_irq(&dev->power.lock);
+                dev->power.runtime_error = retval;
+        } else if (dev->class && dev->class->pm
+            && dev->class->pm->runtime_suspend) {
+                spin_unlock_irq(&dev->power.lock);
+
+                retval = dev->class->pm->runtime_suspend(dev);
+
+                spin_lock_irq(&dev->power.lock);
+                dev->power.runtime_error = retval;
         } else {
                 retval = -ENOSYS;
         }
@@ -359,6 +388,22 @@ int __pm_runtime_resume(struct device *dev, bool from_wq)
 
                 spin_lock_irq(&dev->power.lock);
                 dev->power.runtime_error = retval;
+        } else if (dev->type && dev->type->pm
+            && dev->type->pm->runtime_resume) {
+                spin_unlock_irq(&dev->power.lock);
+
+                retval = dev->type->pm->runtime_resume(dev);
+
+                spin_lock_irq(&dev->power.lock);
+                dev->power.runtime_error = retval;
+        } else if (dev->class && dev->class->pm
+            && dev->class->pm->runtime_resume) {
+                spin_unlock_irq(&dev->power.lock);
+
+                retval = dev->class->pm->runtime_resume(dev);
+
+                spin_lock_irq(&dev->power.lock);
+                dev->power.runtime_error = retval;
         } else {
                 retval = -ENOSYS;
         }