aboutsummaryrefslogtreecommitdiffstats
path: root/init
diff options
context:
space:
mode:
authorLinus Torvalds <torvalds@linux-foundation.org>2013-05-05 16:23:27 -0400
committerLinus Torvalds <torvalds@linux-foundation.org>2013-05-05 16:23:27 -0400
commit534c97b0950b1967bca1c753aeaed32f5db40264 (patch)
tree9421d26e4f6d479d1bc32b036a731b065daab0fa /init
parent64049d1973c1735f543eb7a55653e291e108b0cb (diff)
parent265f22a975c1e4cc3a4d1f94a3ec53ffbb6f5b9f (diff)
Merge branch 'timers-nohz-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull 'full dynticks' support from Ingo Molnar: "This tree from Frederic Weisbecker adds a new, (exciting! :-) core kernel feature to the timer and scheduler subsystems: 'full dynticks', or CONFIG_NO_HZ_FULL=y. This feature extends the nohz variable-size timer tick feature from idle to busy CPUs (running at most one task) as well, potentially reducing the number of timer interrupts significantly. This feature got motivated by real-time folks and the -rt tree, but the general utility and motivation of full-dynticks runs wider than that: - HPC workloads get faster: CPUs running a single task should be able to utilize a maximum amount of CPU power. A periodic timer tick at HZ=1000 can cause a constant overhead of up to 1.0%. This feature removes that overhead - and speeds up the system by 0.5%-1.0% on typical distro configs even on modern systems. - Real-time workload latency reduction: CPUs running critical tasks should experience as little jitter as possible. The last remaining source of kernel-related jitter was the periodic timer tick. - A single task executing on a CPU is a pretty common situation, especially with an increasing number of cores/CPUs, so this feature helps desktop and mobile workloads as well. The cost of the feature is mainly related to increased timer reprogramming overhead when a CPU switches its tick period, and thus slightly longer to-idle and from-idle latency. Configuration-wise a third mode of operation is added to the existing two NOHZ kconfig modes: - CONFIG_HZ_PERIODIC: [formerly !CONFIG_NO_HZ], now explicitly named as a config option. This is the traditional Linux periodic tick design: there's a HZ tick going on all the time, regardless of whether a CPU is idle or not. - CONFIG_NO_HZ_IDLE: [formerly CONFIG_NO_HZ=y], this turns off the periodic tick when a CPU enters idle mode. - CONFIG_NO_HZ_FULL: this new mode, in addition to turning off the tick when a CPU is idle, also slows the tick down to 1 Hz (one timer interrupt per second) when only a single task is running on a CPU. The .config behavior is compatible: existing !CONFIG_NO_HZ and CONFIG_NO_HZ=y settings get translated to the new values, without the user having to configure anything. CONFIG_NO_HZ_FULL is turned off by default. This feature is based on a lot of infrastructure work that has been steadily going upstream in the last 2-3 cycles: related RCU support and non-periodic cputime support in particular is upstream already. This tree adds the final pieces and activates the feature. The pull request is marked RFC because: - it's marked 64-bit only at the moment - the 32-bit support patch is small but did not get ready in time. - it has a number of fresh commits that came in after the merge window. The overwhelming majority of commits are from before the merge window, but still some aspects of the tree are fresh and so I marked it RFC. - it's a pretty wide-reaching feature with lots of effects - and while the components have been in testing for some time, the full combination is still not very widely used. That it's default-off should reduce its regression abilities and obviously there are no known regressions with CONFIG_NO_HZ_FULL=y enabled either. - the feature is not completely idempotent: there is no 100% equivalent replacement for a periodic scheduler/timer tick. In particular there's ongoing work to map out and reduce its effects on scheduler load-balancing and statistics. This should not impact correctness though, there are no known regressions related to this feature at this point. - it's a pretty ambitious feature that with time will likely be enabled by most Linux distros, and we'd like you to make input on its design/implementation, if you dislike some aspect we missed. Without flaming us to crisp! :-) Future plans: - there's ongoing work to reduce 1Hz to 0Hz, to essentially shut off the periodic tick altogether when there's a single busy task on a CPU. We'd first like 1 Hz to be exposed more widely before we go for the 0 Hz target though. - once we reach 0 Hz we can remove the periodic tick assumption from nr_running>=2 as well, by essentially interrupting busy tasks only as frequently as the sched_latency constraints require us to do - once every 4-40 msecs, depending on nr_running. I am personally leaning towards biting the bullet and doing this in v3.10, like the -rt tree this effort has been going on for too long - but the final word is up to you as usual. More technical details can be found in Documentation/timers/NO_HZ.txt" * 'timers-nohz-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (39 commits) sched: Keep at least 1 tick per second for active dynticks tasks rcu: Fix full dynticks' dependency on wide RCU nocb mode nohz: Protect smp_processor_id() in tick_nohz_task_switch() nohz_full: Add documentation. cputime_nsecs: use math64.h for nsec resolution conversion helpers nohz: Select VIRT_CPU_ACCOUNTING_GEN from full dynticks config nohz: Reduce overhead under high-freq idling patterns nohz: Remove full dynticks' superfluous dependency on RCU tree nohz: Fix unavailable tick_stop tracepoint in dynticks idle nohz: Add basic tracing nohz: Select wide RCU nocb for full dynticks nohz: Disable the tick when irq resume in full dynticks CPU nohz: Re-evaluate the tick for the new task after a context switch nohz: Prepare to stop the tick on irq exit nohz: Implement full dynticks kick nohz: Re-evaluate the tick from the scheduler IPI sched: New helper to prevent from stopping the tick in full dynticks sched: Kick full dynticks CPU that have more than one task enqueued. perf: New helper to prevent full dynticks CPUs from stopping tick perf: Kick full dynticks CPU if events rotation is needed ...
Diffstat (limited to 'init')
-rw-r--r--init/Kconfig12
-rw-r--r--init/main.c1
2 files changed, 7 insertions, 6 deletions
diff --git a/init/Kconfig b/init/Kconfig
index a76d13189e47..9d3a7887a6d3 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -302,7 +302,7 @@ choice
302# Kind of a stub config for the pure tick based cputime accounting 302# Kind of a stub config for the pure tick based cputime accounting
303config TICK_CPU_ACCOUNTING 303config TICK_CPU_ACCOUNTING
304 bool "Simple tick based cputime accounting" 304 bool "Simple tick based cputime accounting"
305 depends on !S390 305 depends on !S390 && !NO_HZ_FULL
306 help 306 help
307 This is the basic tick based cputime accounting that maintains 307 This is the basic tick based cputime accounting that maintains
308 statistics about user, system and idle time spent on per jiffies 308 statistics about user, system and idle time spent on per jiffies
@@ -312,7 +312,7 @@ config TICK_CPU_ACCOUNTING
312 312
313config VIRT_CPU_ACCOUNTING_NATIVE 313config VIRT_CPU_ACCOUNTING_NATIVE
314 bool "Deterministic task and CPU time accounting" 314 bool "Deterministic task and CPU time accounting"
315 depends on HAVE_VIRT_CPU_ACCOUNTING 315 depends on HAVE_VIRT_CPU_ACCOUNTING && !NO_HZ_FULL
316 select VIRT_CPU_ACCOUNTING 316 select VIRT_CPU_ACCOUNTING
317 help 317 help
318 Select this option to enable more accurate task and CPU time 318 Select this option to enable more accurate task and CPU time
@@ -342,7 +342,7 @@ config VIRT_CPU_ACCOUNTING_GEN
342 342
343config IRQ_TIME_ACCOUNTING 343config IRQ_TIME_ACCOUNTING
344 bool "Fine granularity task level IRQ time accounting" 344 bool "Fine granularity task level IRQ time accounting"
345 depends on HAVE_IRQ_TIME_ACCOUNTING 345 depends on HAVE_IRQ_TIME_ACCOUNTING && !NO_HZ_FULL
346 help 346 help
347 Select this option to enable fine granularity task irq time 347 Select this option to enable fine granularity task irq time
348 accounting. This is done by reading a timestamp on each 348 accounting. This is done by reading a timestamp on each
@@ -576,7 +576,7 @@ config RCU_FANOUT_EXACT
576 576
577config RCU_FAST_NO_HZ 577config RCU_FAST_NO_HZ
578 bool "Accelerate last non-dyntick-idle CPU's grace periods" 578 bool "Accelerate last non-dyntick-idle CPU's grace periods"
579 depends on NO_HZ && SMP 579 depends on NO_HZ_COMMON && SMP
580 default n 580 default n
581 help 581 help
582 This option permits CPUs to enter dynticks-idle state even if 582 This option permits CPUs to enter dynticks-idle state even if
@@ -687,7 +687,7 @@ choice
687 687
688config RCU_NOCB_CPU_NONE 688config RCU_NOCB_CPU_NONE
689 bool "No build_forced no-CBs CPUs" 689 bool "No build_forced no-CBs CPUs"
690 depends on RCU_NOCB_CPU 690 depends on RCU_NOCB_CPU && !NO_HZ_FULL
691 help 691 help
692 This option does not force any of the CPUs to be no-CBs CPUs. 692 This option does not force any of the CPUs to be no-CBs CPUs.
693 Only CPUs designated by the rcu_nocbs= boot parameter will be 693 Only CPUs designated by the rcu_nocbs= boot parameter will be
@@ -695,7 +695,7 @@ config RCU_NOCB_CPU_NONE
695 695
696config RCU_NOCB_CPU_ZERO 696config RCU_NOCB_CPU_ZERO
697 bool "CPU 0 is a build_forced no-CBs CPU" 697 bool "CPU 0 is a build_forced no-CBs CPU"
698 depends on RCU_NOCB_CPU 698 depends on RCU_NOCB_CPU && !NO_HZ_FULL
699 help 699 help
700 This option forces CPU 0 to be a no-CBs CPU. Additional CPUs 700 This option forces CPU 0 to be a no-CBs CPU. Additional CPUs
701 may be designated as no-CBs CPUs using the rcu_nocbs= boot 701 may be designated as no-CBs CPUs using the rcu_nocbs= boot
diff --git a/init/main.c b/init/main.c
index ceed17aaedfd..9484f4ba88d0 100644
--- a/init/main.c
+++ b/init/main.c
@@ -544,6 +544,7 @@ asmlinkage void __init start_kernel(void)
544 idr_init_cache(); 544 idr_init_cache();
545 perf_event_init(); 545 perf_event_init();
546 rcu_init(); 546 rcu_init();
547 tick_nohz_init();
547 radix_tree_init(); 548 radix_tree_init();
548 /* init some links before init_ISA_irqs() */ 549 /* init some links before init_ISA_irqs() */
549 early_irq_init(); 550 early_irq_init();