KAMEZAWA Hiroyuki dbd4ea78f0
memcg: add lock to synchronize page accounting and migration ...
Introduce a new bit spin lock, PCG_MOVE_LOCK, to synchronize the page
accounting and migration code. This reworks the locking scheme of
_update_stat() and _move_account() by adding new lock bit PCG_MOVE_LOCK,
which is always taken under IRQ disable.
1. If pages are being migrated from a memcg, then updates to that
memcg page statistics are protected by grabbing PCG_MOVE_LOCK using
move_lock_page_cgroup(). In an upcoming commit, memcg dirty page
accounting will be updating memcg page accounting (specifically: num
writeback pages) from IRQ context (softirq). Avoid a deadlocking
nested spin lock attempt by disabling irq on the local processor when
grabbing the PCG_MOVE_LOCK.
2. lock for update_page_stat is used only for avoiding race with
move_account(). So, IRQ awareness of lock_page_cgroup() itself is not
a problem. The problem is between mem_cgroup_update_page_stat() and
mem_cgroup_move_account_page().
Trade-off:
* Changing lock_page_cgroup() to always disable IRQ (or
local_bh) has some impacts on performance and I think
it's bad to disable IRQ when it's not necessary.
* adding a new lock makes move_account() slower. Score is
here.
Performance Impact: moving a 8G anon process.
Before:
real 0m0.792s
user 0m0.000s
sys 0m0.780s
After:
real 0m0.854s
user 0m0.000s
sys 0m0.842s
This score is bad but planned patches for optimization can reduce
this impact.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com >
Signed-off-by: Greg Thelen <gthelen@google.com >
Reviewed-by: Minchan Kim <minchan.kim@gmail.com >
Acked-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp >
Cc: Andrea Righi <arighi@develer.com >
Cc: Balbir Singh <balbir@linux.vnet.ibm.com >
Cc: Wu Fengguang <fengguang.wu@intel.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2011-01-13 17:32:50 -08:00
2011-01-06 22:32:52 +00:00
2011-01-12 15:15:14 -08:00
2010-10-20 15:37:28 -07:00
2011-01-13 08:03:25 -08:00
2011-01-09 09:10:10 -08:00
2011-01-13 09:58:14 -08:00
2010-12-16 12:37:26 -05:00
2011-01-13 10:32:54 -08:00
2010-10-26 17:54:22 -07:00
2011-01-12 14:37:42 +09:00
2010-10-30 08:31:35 -07:00
2011-01-10 20:11:38 +01:00
2011-01-13 08:03:19 -08:00
2011-01-12 14:33:05 +00:00
2011-01-07 17:08:46 -08:00
2010-12-26 22:37:05 -08:00
2011-01-11 15:11:56 -08:00
2011-01-13 08:03:08 -08:00
2011-01-07 13:16:28 -08:00
2010-12-11 01:28:58 -05:00
2010-11-23 20:14:46 +00:00
2010-10-22 10:20:08 -07:00
2010-10-21 20:21:03 -04:00
2010-12-10 15:45:05 -08:00
2010-11-12 07:55:30 -08:00
2011-01-10 08:51:44 -08:00
2010-12-06 15:58:43 -05:00
2010-10-26 16:52:10 -07:00
2010-10-27 18:03:06 -07:00
2011-01-10 13:31:14 -08:00
2011-01-13 08:03:12 -08:00
2010-11-10 14:54:09 +01:00
2011-01-07 17:50:31 +11:00
2010-11-15 13:27:04 -05:00
2010-11-10 14:54:09 +01:00
2011-01-13 10:45:01 -08:00
2010-12-13 16:11:13 -08:00
2010-10-25 21:18:20 -04:00
2010-11-29 08:35:12 +11:00
2011-01-13 08:03:17 -08:00
2010-12-16 17:53:38 +01:00
2010-11-01 15:38:34 -04:00
2010-11-26 10:51:04 +00:00
2010-12-20 09:07:35 -08:00
2010-10-25 08:02:40 -07:00
2011-01-13 17:32:46 -08:00
2011-01-05 14:15:50 +01:00
2010-10-24 14:20:01 -07:00
2010-12-16 16:15:34 -08:00
2010-11-10 16:57:11 -08:00
2011-01-13 08:03:22 -08:00
2010-11-30 13:39:55 +00:00
2011-01-13 08:03:13 -08:00
2011-01-10 07:38:53 -08:00
2010-12-31 10:50:54 -08:00
2010-12-07 13:47:12 +01:00
2011-01-10 08:51:44 -08:00
2011-01-13 20:00:02 +00:00
2011-01-10 16:10:33 -08:00
2011-01-13 19:59:58 +00:00
2011-01-13 19:59:52 +00:00
2011-01-03 01:41:40 -08:00
2010-11-26 09:57:36 +01:00
2010-10-22 15:55:22 +02:00
2011-01-07 23:36:59 -05:00
2010-10-21 14:47:59 +02:00
2010-12-17 15:07:19 +01:00
2011-01-10 16:05:30 -08:00
2010-12-10 15:55:23 -08:00
2010-10-23 09:24:34 +09:00
2011-01-10 19:04:07 +01:00
2011-01-12 20:16:43 -05:00
2010-12-15 13:56:33 -05:00
2010-12-22 12:56:10 +09:00
2011-01-09 15:42:55 -08:00
2010-12-08 10:30:34 -08:00
2010-11-01 15:38:34 -04:00
2011-01-04 08:48:34 +01:00
2011-01-10 08:51:44 -08:00
2010-11-30 17:28:57 -05:00
2011-01-07 17:50:27 +11:00
2011-01-13 10:45:01 -08:00
2010-10-22 10:22:17 -07:00
2010-11-15 13:49:51 +09:00
2011-01-07 17:50:23 +11:00
2011-01-07 17:50:23 +11:00
2010-11-19 10:18:47 -05:00
2010-12-07 20:16:56 +01:00
2011-01-07 17:50:29 +11:00
2011-01-13 14:47:54 +01:00
2011-01-13 17:32:45 -08:00
2010-12-06 17:15:15 -08:00
2010-10-25 14:11:37 -07:00
2011-01-10 22:11:23 +01:00
2011-01-13 17:26:46 -08:00
2010-11-18 10:56:29 -08:00
2011-01-13 09:58:38 -08:00
2010-12-17 15:18:04 +01:00
2011-01-10 07:38:53 -08:00
2011-01-13 17:32:48 -08:00
2010-11-12 14:51:55 +01:00
2010-11-15 22:40:38 +01:00
2010-11-09 09:31:10 -08:00
2011-01-10 22:11:23 +01:00
2010-10-26 17:40:56 -07:00
2010-12-20 14:49:47 -05:00
2010-11-19 17:47:57 +08:00
2011-01-11 23:55:51 +01:00
2011-01-06 11:10:05 -08:00
2010-11-22 08:24:29 -08:00
2010-11-16 10:58:30 -08:00
2010-11-12 12:30:57 -08:00
2011-01-04 11:29:52 -08:00
2010-10-21 16:08:28 +02:00
2010-12-06 12:59:05 -08:00
2011-01-06 10:23:33 -08:00
2010-10-22 10:21:26 -07:00
2011-01-13 09:58:14 -08:00
2010-11-10 10:21:22 +01:00
2010-10-26 16:52:08 -07:00
2010-11-11 13:40:11 +01:00
2010-12-17 10:01:09 -08:00
2010-11-24 11:16:42 -08:00
2010-10-26 10:33:27 +02:00
2011-01-13 17:32:31 -08:00
2010-10-28 11:58:39 +02:00
2011-01-10 12:29:43 -05:00
2010-12-09 20:17:07 -08:00
2010-10-29 14:45:29 -04:00
2011-01-13 10:25:58 -08:00
2010-10-22 15:34:12 -05:00
2011-01-13 17:32:31 -08:00
2011-01-13 17:32:33 -08:00
2010-10-27 18:03:18 -07:00
2010-10-29 13:14:41 -05:00
2011-01-13 17:32:47 -08:00
2011-01-13 08:03:07 -08:00
2010-10-22 10:16:44 -07:00
2011-01-07 17:02:58 -08:00
2011-01-07 08:43:37 +01:00
2010-12-22 10:27:53 +01:00
2011-01-12 11:30:50 +02:00
2010-10-24 10:50:49 +02:00
2011-01-12 11:23:08 +02:00
2011-01-12 11:23:12 +02:00
2011-01-13 08:03:06 -08:00
2011-01-13 08:03:05 -08:00
2010-11-12 07:55:32 -08:00
2011-01-05 19:43:22 -05:00
2010-10-25 14:11:39 -07:00
2011-01-07 17:50:31 +11:00
2010-10-25 21:24:15 -04:00
2010-10-21 14:11:46 -07:00
2010-10-28 09:08:47 -05:00
2010-11-22 08:34:23 -08:00
2010-10-26 16:52:19 -07:00
2011-01-10 19:04:08 +01:00
2010-12-06 11:03:46 -08:00
2011-01-13 17:32:50 -08:00
2011-01-13 17:32:43 -08:00
2010-10-22 10:16:44 -07:00
2011-01-13 17:32:34 -08:00
2011-01-13 17:32:46 -08:00
2011-01-13 17:32:41 -08:00
2011-01-13 17:32:47 -08:00
2011-01-13 17:32:46 -08:00
2011-01-13 17:32:43 -08:00
2010-12-23 09:48:41 +01:00
2010-10-26 16:52:13 -07:00
2011-01-07 17:50:33 +11:00
2010-11-26 15:05:34 +01:00
2011-01-07 17:50:27 +11:00
2011-01-12 20:03:43 -05:00
2011-01-12 20:03:43 -05:00
2011-01-07 17:50:26 +11:00
2011-01-10 16:05:30 -08:00
2010-11-17 10:54:34 -08:00
2010-12-17 12:03:14 -08:00
2010-12-16 12:37:23 -05:00
2011-01-11 14:17:09 -05:00
2011-01-06 14:57:46 -05:00
2011-01-07 17:50:29 +11:00
2010-12-07 23:02:44 -05:00
2011-01-06 16:03:13 -05:00
2010-10-23 09:24:39 +09:00
2010-12-20 14:49:47 -05:00
2010-12-22 22:15:32 +01:00
2010-11-10 16:57:11 -08:00
2010-12-23 15:57:48 -07:00
2010-10-21 11:10:10 -06:00
2011-01-08 21:42:42 -07:00
2010-10-21 11:34:55 -06:00
2010-11-01 01:08:14 -04:00
2011-01-13 17:32:50 -08:00
2011-01-13 17:32:43 -08:00
2010-10-26 16:52:05 -07:00
2011-01-13 17:32:36 -08:00
2011-01-07 17:50:33 +11:00
2011-01-13 08:03:21 -08:00
2010-10-27 21:30:13 -04:00
2010-10-27 17:53:25 +02:00
2010-12-18 15:54:04 +01:00
2010-12-16 11:36:43 +01:00
2010-10-24 15:07:11 -07:00
2011-01-10 07:38:54 -08:00
2010-10-22 10:16:42 -07:00
2010-12-24 15:04:06 +01:00
2010-12-24 15:04:41 +01:00
2010-12-10 15:06:43 +01:00
2011-01-07 17:50:29 +11:00
2011-01-13 08:03:21 -08:00
2011-01-13 08:03:21 -08:00
2011-01-13 08:03:10 -08:00
2010-10-27 18:03:10 -07:00
2010-11-12 07:55:33 -08:00
2010-12-02 17:55:12 +01:00
2011-01-10 19:04:05 +01:00
2011-01-13 17:32:48 -08:00
2010-10-29 04:16:31 -04:00
2010-10-27 18:03:18 -07:00
2010-11-29 11:17:22 -08:00
2010-10-26 16:52:16 -07:00
2011-01-07 17:50:31 +11:00
2010-12-17 12:34:59 -08:00
2010-12-17 12:34:08 -08:00
2010-11-29 22:01:58 -08:00
2010-11-29 22:01:58 -08:00
2010-11-17 08:59:32 -08:00
2011-01-07 17:50:29 +11:00
2010-11-12 07:55:30 -08:00
2010-11-15 13:24:06 -05:00
2010-10-20 13:37:56 -04:00
2011-01-13 08:03:17 -08:00
2011-01-13 08:03:18 -08:00
2010-10-27 18:03:16 -07:00
2011-01-13 08:03:18 -08:00
2011-01-13 17:32:42 -08:00
2011-01-13 08:03:23 -08:00
2010-12-13 22:48:21 +01:00
2010-11-15 11:29:30 -08:00
2011-01-13 17:32:43 -08:00
2011-01-10 09:46:24 +11:00
2010-10-22 10:20:06 -07:00
2010-10-21 10:12:48 +11:00
2010-10-30 12:12:50 +02:00
2011-01-07 17:50:27 +11:00
2010-10-22 10:20:10 -07:00
2011-01-13 08:03:19 -08:00
2010-11-30 14:11:49 -08:00
2011-01-13 15:06:28 +09:00
2010-11-09 14:45:52 +01:00
2010-11-15 18:48:25 +09:00
2010-11-09 16:38:20 +09:00
2010-10-31 10:40:39 -04:00
2010-12-16 14:43:14 -08:00
2010-11-28 21:16:28 +02:00
2011-01-07 17:50:16 +11:00
2010-11-06 09:04:33 +02:00
2010-11-17 14:58:36 -08:00
2010-10-27 17:28:36 +01:00
2010-12-08 12:16:33 -08:00
2011-01-13 10:25:58 -08:00
2011-01-10 12:00:22 -05:00
2010-11-18 14:37:34 +01:00
2010-11-24 11:14:24 -08:00
2011-01-13 10:05:56 -08:00
2011-01-13 17:32:42 -08:00
2010-10-27 18:03:14 -07:00
2010-11-18 14:37:45 +01:00
2010-12-22 19:43:34 -08:00
2011-01-13 08:03:21 -08:00
2010-10-22 14:46:25 +02:00
2010-12-11 12:34:34 +01:00
2011-01-13 08:03:20 -08:00
2011-01-01 13:57:56 -08:00
2011-01-01 13:57:56 -08:00
2011-01-13 08:03:08 -08:00
2010-11-29 08:55:25 +11:00
2010-11-29 08:55:22 +11:00
2010-10-27 18:03:12 -07:00
2011-01-07 20:57:22 -05:00
2010-12-16 13:14:32 -08:00
2011-01-13 08:03:20 -08:00
2010-12-02 12:58:16 -08:00
2010-10-26 16:52:03 -07:00
2010-11-28 23:08:04 +01:00
2010-12-20 09:37:33 +01:00
2010-11-10 16:57:11 -08:00
2010-10-22 10:22:05 -07:00
2010-12-16 10:05:06 -08:00
2011-01-13 08:03:18 -08:00
2011-01-06 13:05:04 +10:00
2010-11-10 12:04:41 +09:00
2010-12-11 02:01:35 -05:00
2010-10-21 07:55:58 -02:00
2010-10-27 18:03:18 -07:00
2011-01-13 17:32:34 -08:00
2011-01-13 17:32:31 -08:00
2010-11-15 13:25:02 -05:00
2011-01-07 16:58:04 -08:00
2010-10-30 09:05:48 -07:00
2010-12-07 14:04:02 -08:00
2010-12-10 14:43:58 -08:00
2011-01-13 08:03:24 -08:00
dbd4ea78f0