Nick Piggin
e286781d5f
mm: speculative page references
...
If we can be sure that elevating the page_count on a pagecache page will
pin it, we can speculatively run this operation, and subsequently check to
see if we hit the right page rather than relying on holding a lock or
otherwise pinning a reference to the page.
This can be done if get_page/put_page behaves consistently throughout the
whole tree (ie. if we "get" the page after it has been used for something
else, we must be able to free it with a put_page).
Actually, there is a period where the count behaves differently: when the
page is free or if it is a constituent page of a compound page. We need
an atomic_inc_not_zero operation to ensure we don't try to grab the page
in either case.
This patch introduces the core locking protocol to the pagecache (ie.
adds page_cache_get_speculative, and tweaks some update-side code to make
it work).
Thanks to Hugh for pointing out an improvement to the algorithm setting
page_count to zero when we have control of all references, in order to
hold off speculative getters.
[kamezawa.hiroyu@jp.fujitsu.com: fix migration_entry_wait()]
[hugh@veritas.com: fix add_to_page_cache]
[akpm@linux-foundation.org: repair a comment]
Signed-off-by: Nick Piggin <npiggin@suse.de>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Hugh Dickins <hugh@veritas.com>
Cc: "Paul E. McKenney" <paulmck@us.ibm.com>
Reviewed-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Signed-off-by: Hugh Dickins <hugh@veritas.com>
Acked-by: Nick Piggin <npiggin@suse.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-07-26 12:00:06 -07:00
..
2008-07-25 09:28:09 -07:00
2008-07-25 10:53:30 -07:00
2008-07-15 16:11:29 -04:00
2008-07-23 18:24:08 -07:00
2008-07-23 08:12:26 -07:00
2008-07-16 15:17:52 -07:00
2008-07-24 13:32:56 +03:00
2008-07-21 10:11:00 -07:00
2008-07-25 10:53:34 -07:00
2008-07-21 17:05:25 +10:00
2008-07-25 10:53:30 -07:00
2008-07-26 12:00:03 -07:00
2008-07-20 21:21:46 -07:00
2008-07-25 10:53:29 -07:00
2008-07-25 10:53:47 -07:00
2008-07-24 10:47:24 -07:00
2008-07-01 11:28:06 +10:00
2008-07-26 12:00:04 -07:00
2008-07-24 10:47:27 -07:00
2008-07-17 17:59:55 -07:00
2008-07-24 10:47:41 -07:00
2008-07-24 10:47:24 -07:00
2008-07-24 10:47:33 -07:00
2008-07-25 15:44:39 +10:00
2008-07-24 10:47:33 -07:00
2008-07-24 10:47:22 -07:00
2008-07-03 13:21:14 +02:00
2008-07-16 14:51:41 -07:00
2008-07-03 13:21:12 +02:00
2008-07-25 11:36:44 -07:00
2008-07-04 10:40:08 -07:00
2008-07-04 10:40:05 -07:00
2008-07-25 10:53:37 -07:00
2008-07-25 10:53:33 -07:00
2008-07-17 15:21:29 -07:00
2008-07-25 10:53:40 -07:00
2008-07-23 19:36:53 -07:00
2008-07-18 13:22:25 +02:00
2008-07-25 10:53:41 -07:00
2008-07-12 08:22:32 -05:00
2008-07-10 20:35:18 +08:00
2008-07-20 17:12:38 -07:00
2008-07-22 17:30:57 -07:00
2008-07-11 10:46:50 +02:00
2008-07-13 11:51:40 +01:00
2008-07-05 13:10:50 -07:00
2008-07-21 21:54:59 -07:00
2008-07-25 10:53:47 -07:00
2008-07-21 12:00:37 +01:00
2008-07-21 21:54:53 -07:00
2008-07-25 10:53:34 -07:00
2008-07-21 12:00:37 +01:00
2008-07-22 10:39:36 +10:00
2008-07-17 17:59:56 -07:00
2008-07-08 11:59:42 -07:00
2008-07-21 21:54:53 -07:00
2008-07-01 14:47:09 +10:00
2008-07-02 03:47:41 -07:00
2008-07-24 10:47:29 -07:00
2008-07-24 10:47:29 -07:00
2008-07-25 10:53:31 -07:00
2008-07-25 10:53:32 -07:00
2008-07-24 10:47:41 -07:00
2008-07-08 17:55:41 +02:00
2008-07-10 14:30:13 +01:00
2008-07-16 23:27:03 +02:00
2008-07-22 19:09:51 -07:00
2008-07-25 10:53:47 -07:00
2008-07-16 17:57:47 -05:00
2008-07-11 15:49:21 +02:00
2008-07-25 10:53:48 -07:00
2008-07-20 17:12:36 -07:00
2008-07-25 10:53:44 -07:00
2008-07-24 10:47:20 -07:00
2008-07-25 10:53:30 -07:00
2008-07-20 17:12:36 -07:00
2008-07-04 08:47:41 -04:00
2008-07-23 15:25:21 +02:00
2008-07-24 10:47:19 -07:00
2008-07-14 22:38:31 +02:00
2008-07-20 07:18:38 -03:00
2008-07-14 22:38:36 +02:00
2008-07-26 12:00:03 -07:00
2008-07-24 22:53:33 +02:00
2008-07-25 10:53:42 -07:00
2008-07-08 14:15:59 -04:00
2008-07-18 18:05:19 -07:00
2008-07-14 22:18:19 -07:00
2008-07-14 22:49:06 -07:00
2008-07-03 17:51:57 +09:00
2008-07-10 14:47:38 +01:00
2008-07-09 12:09:28 -04:00
2008-07-25 10:53:39 -07:00
2008-07-26 12:00:04 -07:00
2008-07-24 10:47:29 -07:00
2008-07-21 00:55:14 -04:00
2008-07-15 10:39:22 -07:00
2008-07-03 13:21:14 +02:00
2008-07-16 23:27:07 +02:00
2008-07-25 10:53:42 -07:00
2008-07-21 13:41:16 -07:00
2008-07-25 10:53:26 -07:00
2008-07-20 17:12:37 -07:00
2008-07-11 19:27:31 -04:00
2008-07-25 10:53:29 -07:00
2008-07-25 12:01:37 -07:00
2008-07-25 10:53:29 -07:00
2008-07-26 12:00:04 -07:00
2008-07-25 10:53:28 -07:00
2008-07-24 10:47:17 -07:00
2008-07-25 10:53:30 -07:00
2008-07-04 10:40:05 -07:00
2008-07-24 19:11:15 -07:00
2008-07-20 12:42:40 +03:00
2008-07-20 12:42:32 +03:00
2008-07-24 10:47:40 -07:00
2008-07-23 09:49:56 +01:00
2008-07-23 09:49:56 +01:00
2008-07-14 15:59:33 -04:00
2008-07-14 16:11:52 +02:00
2008-07-25 10:53:27 -07:00
2008-07-24 10:47:41 -07:00
2008-07-25 10:53:37 -07:00
2008-07-24 10:47:21 -07:00
2008-07-24 10:47:21 -07:00
2008-07-26 12:00:04 -07:00
2008-07-24 10:47:21 -07:00
2008-07-25 10:53:40 -07:00
2008-07-26 12:00:05 -07:00
2008-07-09 16:30:45 +10:00
2008-07-14 10:02:12 +02:00
2008-07-24 10:47:24 -07:00
2008-07-11 19:27:31 -04:00
2008-07-03 17:51:57 +09:00
2008-07-03 17:51:57 +09:00
2008-07-25 10:53:34 -07:00
2008-07-25 10:53:29 -07:00
2008-07-22 14:09:06 -07:00
2008-07-08 02:36:40 -07:00
2008-07-09 12:09:40 -04:00
2008-07-09 12:09:17 -04:00
2008-07-09 12:09:24 -04:00
2008-07-09 12:09:16 -04:00
2008-07-25 10:53:40 -07:00
2008-07-25 10:53:37 -07:00
2008-07-22 10:39:30 +10:00
2008-07-24 10:47:15 -07:00
2008-07-26 12:00:06 -07:00
2008-07-26 12:00:03 -07:00
2008-07-24 11:50:15 -07:00
2008-07-25 10:53:48 -07:00
2008-07-07 16:26:50 -07:00
2008-07-22 14:25:38 -07:00
2008-07-11 19:27:31 -04:00
2008-07-25 10:53:46 -07:00
2008-07-25 10:53:45 -07:00
2008-07-14 20:36:32 -07:00
2008-07-20 00:08:47 -07:00
2008-07-14 14:30:21 -07:00
2008-07-24 10:47:23 -07:00
2008-07-16 23:27:07 +02:00
2008-07-25 10:53:44 -07:00
2008-07-25 10:53:27 -07:00
2008-07-14 15:01:47 +10:00
2008-07-03 13:25:05 +01:00
2008-07-25 10:53:35 -07:00
2008-07-25 10:53:35 -07:00
2008-07-26 12:00:06 -07:00
2008-07-25 10:53:29 -07:00
2008-07-04 10:40:07 -07:00
2008-07-25 10:53:29 -07:00
2008-07-25 10:53:33 -07:00
2008-07-25 10:53:26 -07:00
2008-07-26 12:00:04 -07:00
2008-07-25 10:53:37 -07:00
2008-07-24 10:47:33 -07:00
2008-07-20 00:08:47 -07:00
2008-07-23 14:42:09 +02:00
2008-07-26 12:00:04 -07:00
2008-07-04 10:40:08 -07:00
2008-07-24 10:47:22 -07:00
2008-07-25 10:53:42 -07:00
2008-07-24 08:31:21 -04:00
2008-07-18 04:07:44 -07:00
2008-07-24 10:47:24 -07:00
2008-07-08 10:33:01 -04:00
2008-07-24 10:47:29 -07:00
2008-07-14 22:49:06 -07:00
2008-07-24 10:47:21 -07:00
2008-07-04 10:40:04 -07:00
2008-07-25 10:53:30 -07:00
2008-07-25 10:53:26 -07:00
2008-07-12 21:52:41 +01:00
2008-07-26 12:00:04 -07:00
2008-07-19 22:35:47 -07:00
2008-07-25 10:53:26 -07:00
2008-07-20 17:12:37 -07:00
2008-07-24 10:47:13 -07:00
2008-07-26 12:00:04 -07:00
2008-07-22 13:03:29 -07:00
2008-07-25 17:28:49 -07:00
2008-07-21 21:55:02 -07:00
2008-07-21 21:55:01 -07:00
2008-07-25 10:53:47 -07:00
2008-07-19 00:07:02 -07:00
2008-07-08 02:38:40 -07:00
2008-07-18 18:10:28 +02:00
2008-07-24 10:47:29 -07:00
2008-07-22 13:03:28 -07:00
2008-07-20 17:12:34 -07:00
2008-07-22 13:03:27 -07:00
2008-07-25 10:53:26 -07:00
2008-07-21 21:54:55 -07:00
2008-07-21 15:16:40 -07:00
2008-07-21 15:15:55 -07:00
2008-07-20 07:17:02 -03:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:05 +10:00
2008-07-25 12:06:12 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:14 +10:00
2008-07-25 12:06:04 +10:00
2008-07-24 10:47:14 -07:00
2008-07-25 10:53:40 -07:00
2008-07-15 08:36:38 -07:00
2008-07-10 16:55:37 -07:00