Paul E. McKenney 25502a6c13
rcu: refactor RCU's context-switch handling ...
The addition of preemptible RCU to treercu resulted in a bit of
confusion and inefficiency surrounding the handling of context switches
for RCU-sched and for RCU-preempt. For RCU-sched, a context switch
is a quiescent state, pure and simple, just like it always has been.
For RCU-preempt, a context switch is in no way a quiescent state, but
special handling is required when a task blocks in an RCU read-side
critical section.
However, the callout from the scheduler and the outer loop in ksoftirqd
still calls something named rcu_sched_qs(), whose name is no longer
accurate. Furthermore, when rcu_check_callbacks() notes an RCU-sched
quiescent state, it ends up unnecessarily (though harmlessly, aside
from the performance hit) enqueuing the current task if it happens to
be running in an RCU-preempt read-side critical section. This not only
increases the maximum latency of scheduler_tick(), it also needlessly
increases the overhead of the next outermost rcu_read_unlock() invocation.
This patch addresses this situation by separating the notion of RCU's
context-switch handling from that of RCU-sched's quiescent states.
The context-switch handling is covered by rcu_note_context_switch() in
general and by rcu_preempt_note_context_switch() for preemptible RCU.
This permits rcu_sched_qs() to handle quiescent states and only quiescent
states. It also reduces the maximum latency of scheduler_tick(), though
probably by much less than a microsecond. Finally, it means that tasks
within preemptible-RCU read-side critical sections avoid incurring the
overhead of queuing unless there really is a context switch.
Suggested-by: Lai Jiangshan <laijs@cn.fujitsu.com >
Acked-by: Lai Jiangshan <laijs@cn.fujitsu.com >
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com >
Cc: Ingo Molnar <mingo@elte.hu >
Cc: Peter Zijlstra <peterz@infradead.org >
2010-05-10 11:08:33 -07:00
2010-03-29 17:33:33 +01:00
2010-02-26 05:22:03 -08:00
2010-03-12 15:52:44 -08:00
2010-03-07 22:17:32 +01:00
2010-04-05 23:02:00 -07:00
2010-02-16 16:01:22 -08:00
2010-03-07 22:17:39 +01:00
2010-03-24 16:31:21 -07:00
2010-02-02 13:02:16 +09:00
2010-03-20 14:29:03 -07:00
2010-02-24 18:32:59 +01:00
2010-01-18 08:25:47 +01:00
2010-02-24 18:34:48 +01:00
2010-02-24 18:34:48 +01:00
2009-12-15 15:01:46 -05:00
2010-04-19 13:17:10 +01:00
2010-03-30 22:02:32 +09:00
2010-03-22 05:32:44 -04:00
2010-03-30 22:02:32 +09:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:53:10 -08:00
2010-01-06 16:18:15 -05:00
2009-12-16 07:20:13 -08:00
2010-04-08 12:53:57 -04:00
2009-12-15 08:53:35 -08:00
2010-02-08 14:38:36 -05:00
2010-04-25 08:54:42 +02:00
2010-03-16 19:47:54 +00:00
2010-03-06 11:26:46 -08:00
2009-12-16 07:20:18 -08:00
2010-04-07 08:38:01 -07:00
2010-03-19 08:58:16 +01:00
2010-02-17 11:17:38 +09:00
2010-02-12 09:41:59 -08:00
2010-02-17 17:27:40 -08:00
2010-02-22 13:44:45 +01:00
2010-02-22 13:44:45 +01:00
2010-05-04 09:25:02 -07:00
2010-03-24 16:31:22 -07:00
2010-03-12 19:10:29 +01:00
2010-02-05 14:54:10 +01:00
2010-04-28 09:20:33 +02:00
2010-03-12 15:52:32 -08:00
2010-02-05 07:35:05 -08:00
2010-02-02 15:58:48 -08:00
2010-03-12 15:52:32 -08:00
2010-03-06 11:26:29 -08:00
2010-03-04 11:46:14 +01:00
2009-12-15 08:53:28 -08:00
2009-12-15 08:53:32 -08:00
2010-02-04 13:29:53 -08:00
2010-03-30 22:02:32 +09:00
2010-03-06 02:32:27 +00:00
2010-03-07 17:04:52 -08:00
2010-02-05 12:22:34 +01:00
2010-03-06 02:32:33 +00:00
2010-03-06 02:32:31 +00:00
2010-03-12 15:52:42 -08:00
2010-03-04 08:20:14 -08:00
2010-03-11 16:32:14 +01:00
2010-03-11 15:51:23 +01:00
2010-02-26 08:25:35 +01:00
2009-12-15 08:53:33 -08:00
2010-03-06 11:26:46 -08:00
2010-03-06 11:26:46 -08:00
2009-12-15 08:53:27 -08:00
2010-03-05 14:00:17 -08:00
2010-01-25 12:26:38 -02:00
2010-02-20 13:13:44 -08:00
2010-03-29 14:30:19 -07:00
2009-12-23 13:44:12 +01:00
2010-03-29 14:30:19 -07:00
2009-12-23 13:33:54 +01:00
2010-02-26 19:19:39 +02:00
2010-03-01 16:20:37 +10:00
2010-02-25 10:34:49 +01:00
2009-12-17 20:55:57 -08:00
2009-12-16 12:16:42 -05:00
2010-04-15 22:18:36 +02:00
2010-04-15 22:18:36 +02:00
2009-12-29 19:58:17 +01:00
2010-03-06 11:26:25 -08:00
2010-03-26 23:51:44 +01:00
2009-12-17 10:58:17 -05:00
2010-04-23 10:39:48 -07:00
2010-03-29 09:14:47 -07:00
2010-03-30 22:02:32 +09:00
2010-03-10 14:47:18 +01:00
2010-02-28 10:20:25 -08:00
2010-03-30 22:02:32 +09:00
2009-12-16 12:16:49 -05:00
2010-03-16 08:55:32 +01:00
2010-03-06 11:26:27 -08:00
2010-03-01 14:08:10 +00:00
2010-02-04 00:50:44 -08:00
2009-12-16 07:20:01 -08:00
2010-02-25 17:39:16 +01:00
2010-02-26 17:05:10 -08:00
2010-02-09 11:13:56 +01:00
2010-03-07 22:17:09 +01:00
2010-02-27 16:23:39 +01:00
2010-03-13 20:56:56 +01:00
2010-02-12 17:32:41 +00:00
2010-03-02 12:23:42 +01:00
2010-03-14 11:14:58 -07:00
2010-05-04 11:09:28 +02:00
2010-03-24 08:20:03 +01:00
2010-02-18 14:30:17 -08:00
2010-04-01 01:31:13 -07:00
2010-01-22 16:11:27 -05:00
2010-02-12 16:56:07 -08:00
2010-02-18 14:08:38 -08:00
2010-02-04 20:24:10 -08:00
2010-02-17 16:35:17 -08:00
2010-03-21 21:19:02 -07:00
2010-02-02 07:32:29 -08:00
2010-02-07 03:06:22 -05:00
2010-01-11 16:28:01 -08:00
2010-02-16 15:55:17 -08:00
2010-03-12 15:52:40 -08:00
2009-12-15 20:42:06 +01:00
2010-03-08 23:19:15 -08:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:52:36 -08:00
2009-12-16 07:20:18 -08:00
2010-03-23 13:33:50 -07:00
2010-03-12 15:52:40 -08:00
2010-03-12 15:52:40 -08:00
2010-03-03 08:15:37 -08:00
2010-04-01 16:12:48 -07:00
2010-03-30 22:02:32 +09:00
2010-03-30 22:02:32 +09:00
2009-12-15 08:53:26 -08:00
2010-01-30 01:47:49 -08:00
2010-03-02 10:41:26 -08:00
2009-12-16 12:19:59 +01:00
2010-04-07 08:38:04 -07:00
2010-02-16 16:01:21 -08:00
2010-02-17 11:17:38 +09:00
2010-04-07 08:38:02 -07:00
2010-01-07 11:58:37 -06:00
2010-01-11 09:34:04 -08:00
2010-03-12 15:53:10 -08:00
2009-12-31 19:45:04 +00:00
2010-03-07 17:04:49 -08:00
2010-02-25 17:49:25 +01:00
2009-12-16 06:56:12 -08:00
2010-04-20 13:08:30 +03:00
2010-03-01 12:36:14 -03:00
2010-03-15 12:47:59 +01:00
2009-12-17 11:41:51 +00:00
2009-12-17 11:33:33 +00:00
2009-12-17 11:27:09 +00:00
2010-04-06 10:55:33 -04:00
2009-12-15 08:53:36 -08:00
2010-01-12 21:02:00 -08:00
2010-03-06 11:26:35 -08:00
2009-12-26 20:40:34 -08:00
2010-02-03 17:39:50 +11:00
2010-03-12 15:53:10 -08:00
2010-02-09 11:13:56 +01:00
2010-02-19 03:35:12 -05:00
2010-03-12 15:52:38 -08:00
2010-03-17 18:43:47 -07:00
2010-01-15 01:43:29 -08:00
2010-03-12 15:52:28 -08:00
2010-04-07 08:38:04 -07:00
2010-03-08 16:55:37 +01:00
2010-03-03 14:07:59 -05:00
2010-04-05 19:50:02 -07:00
2010-03-04 08:15:33 -08:00
2010-02-10 23:49:08 +09:00
2009-12-22 12:27:34 -05:00
2010-04-22 12:31:11 +02:00
2010-02-03 20:32:28 -08:00
2010-03-18 21:16:45 -07:00
2010-03-25 11:17:26 +01:00
2010-02-19 08:03:28 +01:00
2010-03-20 14:29:03 -07:00
2010-01-13 20:38:26 -08:00
2010-04-12 07:55:15 -04:00
2010-04-22 15:35:57 -04:00
2009-12-16 10:43:34 -08:00
2010-02-13 12:26:02 +09:00
2010-02-19 15:52:40 -05:00
2010-03-12 15:52:38 -08:00
2010-02-14 07:13:47 -07:00
2010-03-18 07:30:31 -06:00
2010-02-15 15:14:34 +02:00
2009-12-16 07:19:57 -08:00
2010-01-06 19:47:10 +11:00
2010-04-07 08:38:05 -07:00
2009-12-17 07:23:42 -08:00
2010-01-27 09:20:03 -08:00
2010-02-22 16:15:17 -08:00
2010-03-03 08:47:22 -08:00
2010-02-23 23:14:47 -08:00
2010-02-22 16:21:02 -08:00
2010-03-12 15:52:42 -08:00
2010-03-12 15:52:42 -08:00
2010-03-03 07:34:18 -08:00
2010-04-05 09:39:11 -07:00
2010-04-01 08:26:31 +02:00
2010-01-17 21:52:11 +11:00
2010-02-04 10:23:02 -08:00
2010-02-24 08:30:08 +01:00
2010-03-07 17:04:49 -08:00
2010-01-13 10:51:39 +01:00
2010-03-06 21:28:37 +01:00
2010-03-06 21:28:37 +01:00
2009-12-15 17:35:26 -05:00
2010-04-24 11:31:25 -07:00
2010-03-12 15:53:11 -08:00
2010-01-15 16:59:21 -05:00
2010-03-12 15:52:38 -08:00
2009-12-17 11:46:01 +00:00
2010-03-05 00:20:31 +01:00
2010-03-05 00:20:30 +01:00
2010-04-09 10:12:03 -07:00
2010-02-08 18:19:41 -06:00
2010-02-10 17:47:17 -08:00
2010-02-25 11:45:20 +01:00
2010-02-25 09:41:02 +01:00
2010-02-28 19:23:06 -08:00
2010-05-10 11:08:32 -07:00
2010-05-10 11:08:33 -07:00
2010-05-10 11:08:33 -07:00
2010-03-12 15:52:44 -08:00
2010-03-05 13:25:52 -05:00
2010-03-24 16:31:21 -07:00
2010-01-04 11:33:58 +01:00
2010-02-26 20:39:09 +01:00
2010-03-02 14:28:49 -05:00
2010-03-06 11:26:27 -08:00
2010-03-12 15:52:28 -08:00
2010-03-12 15:53:10 -08:00
2010-02-28 19:23:06 -08:00
2010-03-13 01:21:21 +01:00
2009-12-15 08:53:26 -08:00
2010-03-13 14:43:01 -08:00
2010-03-30 22:02:32 +09:00
2009-12-16 07:20:10 -08:00
2010-02-22 15:45:54 -08:00
2010-03-12 15:52:43 -08:00
2010-03-10 11:55:05 +09:00
2010-01-21 23:55:25 -08:00
2010-02-16 13:38:56 +09:00
2009-12-16 12:16:49 -05:00
2010-03-12 15:52:44 -08:00
2010-03-24 11:13:35 -07:00
2010-04-09 10:09:50 -07:00
2009-12-20 10:17:59 +02:00
2009-12-16 07:20:04 -08:00
2010-04-22 12:37:07 +02:00
2010-03-06 11:26:32 -08:00
2010-03-08 10:45:27 -08:00
2010-03-27 08:29:01 -07:00
2009-12-16 22:32:29 -05:00
2010-03-13 01:21:21 +01:00
2010-05-10 11:08:32 -07:00
2010-01-07 17:06:06 -08:00
2010-01-14 22:38:09 -05:00
2010-03-12 15:52:36 -08:00
2010-03-22 13:12:33 -07:00
2010-02-16 15:55:18 -08:00
2010-03-07 17:04:47 -08:00
2010-03-07 17:04:52 -08:00
2010-02-04 14:20:41 +11:00
2010-03-30 22:02:32 +09:00
2010-02-18 15:43:09 -08:00
2009-12-16 07:20:00 -08:00
2010-01-29 10:15:19 +01:00
2010-01-21 13:39:03 +01:00
2009-12-16 07:20:08 -08:00
2010-03-16 10:10:40 +01:00
2010-03-02 14:55:11 -08:00
2010-03-12 15:52:43 -08:00
2010-03-19 07:17:57 -07:00
2010-01-07 11:58:36 -06:00
2010-04-30 09:25:12 -07:00
2010-03-01 22:21:58 +11:00
2010-01-15 01:43:29 -08:00
2010-02-26 15:10:40 -03:00
2010-03-13 08:57:28 -06:00
2010-02-24 14:22:08 +10:30
2010-02-24 14:22:26 +10:30
2010-04-08 09:46:19 +09:30
2010-03-13 08:57:29 -06:00
2010-01-05 09:17:33 +09:00
2010-03-19 07:17:52 -07:00
2010-03-12 10:03:42 +01:00
2009-12-16 12:16:49 -05:00
2010-02-22 16:19:45 -08:00
25502a6c13