Hagen Paul Pfeifer 01f2f3f6ef
net: optimize Berkeley Packet Filter (BPF) processing ...
Gcc is currenlty not in the ability to optimize the switch statement in
sk_run_filter() because of dense case labels. This patch replace the
OR'd labels with ordered sequenced case labels. The sk_chk_filter()
function is modified to patch/replace the original OPCODES in a
ordered but equivalent form. gcc is now in the ability to transform the
switch statement in sk_run_filter into a jump table of complexity O(1).
Until this patch gcc generates a sequence of conditional branches (O(n) of 567
byte .text segment size (arch x86_64):
7ff: 8b 06 mov (%rsi),%eax
801: 66 83 f8 35 cmp $0x35,%ax
805: 0f 84 d0 02 00 00 je adb <sk_run_filter+0x31d>
80b: 0f 87 07 01 00 00 ja 918 <sk_run_filter+0x15a>
811: 66 83 f8 15 cmp $0x15,%ax
815: 0f 84 c5 02 00 00 je ae0 <sk_run_filter+0x322>
81b: 77 73 ja 890 <sk_run_filter+0xd2>
81d: 66 83 f8 04 cmp $0x4,%ax
821: 0f 84 17 02 00 00 je a3e <sk_run_filter+0x280>
827: 77 29 ja 852 <sk_run_filter+0x94>
829: 66 83 f8 01 cmp $0x1,%ax
[...]
With the modification the compiler translate the switch statement into
the following jump table fragment:
7ff: 66 83 3e 2c cmpw $0x2c,(%rsi)
803: 0f 87 1f 02 00 00 ja a28 <sk_run_filter+0x26a>
809: 0f b7 06 movzwl (%rsi),%eax
80c: ff 24 c5 00 00 00 00 jmpq *0x0(,%rax,8)
813: 44 89 e3 mov %r12d,%ebx
816: e9 43 03 00 00 jmpq b5e <sk_run_filter+0x3a0>
81b: 41 89 dc mov %ebx,%r12d
81e: e9 3b 03 00 00 jmpq b5e <sk_run_filter+0x3a0>
Furthermore, I reordered the instructions to reduce cache line misses by
order the most common instruction to the start.
Signed-off-by: Hagen Paul Pfeifer <hagen@jauu.net >
Signed-off-by: David S. Miller <davem@davemloft.net >
2010-06-25 21:33:12 -07:00
2010-05-30 09:12:43 -07:00
2010-05-26 08:30:15 -07:00
2010-06-20 19:46:07 -07:00
2010-05-17 22:39:48 -07:00
2010-05-19 12:57:48 -03:00
2010-05-27 11:34:55 -07:00
2010-05-28 01:37:38 +02:00
2010-05-28 01:38:00 +02:00
2010-04-21 16:37:49 -07:00
2010-05-27 09:12:40 -07:00
2010-05-14 01:56:12 +01:00
2010-06-15 17:31:06 +02:00
2010-05-03 08:33:00 -04:00
2010-05-25 10:16:01 +01:00
2010-05-25 00:38:26 -06:00
2010-06-04 16:00:42 -04:00
2010-05-14 15:09:37 -04:00
2010-05-29 15:31:57 -07:00
2010-05-28 14:42:18 -07:00
2010-05-14 17:08:01 -04:00
2010-05-27 09:12:53 -07:00
2010-05-21 09:34:30 -07:00
2010-05-21 09:34:30 -07:00
2010-05-19 13:38:54 -04:00
2010-05-21 20:00:35 +02:00
2010-05-30 09:00:03 -07:00
2010-05-18 09:17:01 -07:00
2010-05-21 15:26:46 -07:00
2010-06-24 21:30:09 -07:00
2010-05-27 22:15:33 -04:00
2010-05-27 09:12:44 -07:00
2010-05-10 14:24:26 +02:00
2010-04-28 09:20:33 +02:00
2010-05-25 08:07:00 -07:00
2010-05-27 09:12:53 -07:00
2010-05-29 09:12:30 -07:00
2010-05-19 22:41:05 -04:00
2010-04-09 14:07:56 -07:00
2010-05-27 21:06:58 -04:00
2010-05-27 09:12:44 -07:00
2010-05-27 09:12:45 -07:00
2010-05-19 22:03:14 +03:00
2010-05-15 07:16:33 -04:00
2010-04-22 18:32:12 -07:00
2010-05-19 22:41:57 -04:00
2010-05-10 16:08:01 -07:00
2010-05-22 00:36:56 -06:00
2010-05-27 09:12:53 -07:00
2010-05-17 16:30:58 -07:00
2010-05-21 19:30:44 +02:00
2010-05-18 01:25:00 +02:00
2010-05-18 01:25:00 +02:00
2010-05-21 21:12:01 +02:00
2010-04-26 22:03:42 +04:00
2010-05-25 08:07:05 -07:00
2010-04-09 08:36:07 +02:00
2010-05-17 10:00:15 +02:00
2010-05-21 20:29:10 -07:00
2010-05-25 08:07:02 -07:00
2010-06-14 23:13:06 -07:00
2010-05-21 19:30:39 +02:00
2010-05-27 22:05:02 -04:00
2010-05-27 22:05:02 -04:00
2010-05-21 21:12:40 +02:00
2010-05-24 00:36:13 -07:00
2010-04-26 16:13:54 +02:00
2010-05-27 22:03:07 -04:00
2010-06-25 21:33:12 -07:00
2010-04-15 22:18:36 +02:00
2010-04-15 22:18:36 +02:00
2010-05-19 00:26:30 +02:00
2010-05-21 09:37:28 -07:00
2010-05-27 22:15:33 -04:00
2010-05-27 15:23:47 -07:00
2010-05-18 08:35:04 -07:00
2010-05-25 15:06:06 +02:00
2010-05-21 18:31:18 -04:00
2010-05-25 08:07:01 -07:00
2010-05-27 09:12:42 -07:00
2010-05-21 09:34:29 -07:00
2010-05-19 14:05:06 +02:00
2010-05-25 08:07:01 -07:00
2010-05-01 04:32:14 +02:00
2010-05-20 00:18:59 +01:00
2010-05-22 00:36:56 -06:00
2010-05-24 08:05:29 -07:00
2010-04-20 11:52:21 -04:00
2010-06-05 02:23:17 -07:00
2010-06-02 07:11:15 -07:00
2010-06-03 03:21:52 -07:00
2010-06-03 03:21:52 -07:00
2010-06-03 03:21:52 -07:00
2010-06-03 03:21:52 -07:00
2010-06-12 15:51:22 -07:00
2010-06-02 07:11:15 -07:00
2010-06-02 05:53:56 -07:00
2010-06-03 03:21:52 -07:00
2010-05-03 12:33:13 +03:00
2010-04-22 16:12:36 -07:00
2010-06-15 11:48:58 -07:00
2010-04-23 23:35:28 -07:00
2010-06-23 13:16:38 -07:00
2010-05-27 09:12:52 -07:00
2010-05-20 23:05:28 -07:00
2010-05-21 10:48:12 -07:00
2010-05-11 12:01:10 -07:00
2010-06-03 03:21:52 -07:00
2010-05-03 11:50:57 +02:00
2010-05-19 17:33:38 +09:30
2010-06-03 03:21:52 -07:00
2010-05-25 08:07:09 -07:00
2010-05-21 19:30:40 +02:00
2010-05-20 23:05:28 -07:00
2010-04-22 16:12:36 -07:00
2010-05-20 21:04:27 -05:00
2010-05-25 08:07:05 -07:00
2010-04-23 02:08:44 +02:00
2010-05-20 21:04:29 -05:00
2010-05-27 09:12:45 -07:00
2010-05-21 09:37:32 -07:00
2010-05-21 09:37:29 -07:00
2010-04-21 16:33:29 -07:00
2010-05-14 15:09:32 -04:00
2010-05-19 11:36:29 +03:00
2010-05-17 12:17:10 +03:00
2010-05-26 17:34:16 +01:00
2010-05-26 13:07:55 +01:00
2010-05-25 19:41:19 -04:00
2010-05-25 08:07:07 -07:00
2010-05-21 09:37:30 -07:00
2010-04-28 08:51:12 +10:00
2010-05-25 08:07:09 -07:00
2010-05-27 09:12:44 -07:00
2010-05-25 08:07:00 -07:00
2010-05-25 08:06:57 -07:00
2010-05-18 00:52:36 -03:00
2010-05-25 08:06:59 -07:00
2010-05-25 15:08:26 -07:00
2010-06-03 03:21:52 -07:00
2010-05-25 08:07:02 -07:00
2010-05-27 09:12:57 -07:00
2010-05-21 17:15:44 -07:00
2010-04-08 10:18:47 +02:00
2010-05-11 14:40:55 +02:00
2010-04-13 14:49:34 -07:00
2010-04-28 15:16:48 -07:00
2010-06-03 03:21:52 -07:00
2010-06-03 03:21:52 -07:00
2010-05-17 05:27:42 +02:00
2010-06-03 03:21:52 -07:00
2010-05-01 15:00:15 -07:00
2010-06-16 18:07:52 -07:00
2010-04-20 16:22:01 +02:00
2010-05-21 09:37:32 -07:00
2010-06-15 21:44:29 -07:00
2010-05-14 15:09:30 -04:00
2010-05-14 15:09:21 -04:00
2010-05-14 15:09:30 -04:00
2010-05-10 11:32:33 +09:00
2010-06-02 16:13:18 -04:00
2010-05-30 09:00:03 -07:00
2010-05-31 00:28:35 -07:00
2010-05-22 00:10:41 -06:00
2010-04-28 18:20:29 -06:00
2010-05-22 00:10:40 -06:00
2010-05-19 13:44:27 +10:00
2010-05-27 09:12:44 -07:00
2010-05-11 12:01:40 -07:00
2010-05-21 18:58:52 -07:00
2010-05-28 14:42:18 -07:00
2010-04-16 19:14:34 +04:00
2010-05-21 11:37:59 +02:00
2010-06-03 03:21:52 -07:00
2010-05-21 21:12:52 +02:00
2010-05-17 21:37:40 +02:00
2010-05-10 23:08:19 +02:00
2010-05-10 23:09:30 +02:00
2010-05-10 23:08:15 +02:00
2010-04-24 11:31:25 -07:00
2010-05-19 12:14:42 +04:00
2010-05-27 18:56:27 +02:00
2010-05-30 09:11:11 -07:00
2010-04-09 10:12:03 -07:00
2010-05-21 18:31:26 -04:00
2010-05-27 09:12:52 -07:00
2010-05-25 08:07:03 -07:00
2010-05-18 09:28:04 -07:00
2010-05-03 15:53:54 -07:00
2010-05-10 16:53:55 -07:00
2010-05-18 08:27:54 -07:00
2010-05-18 08:27:54 -07:00
2010-06-03 03:21:52 -07:00
2010-05-21 18:31:19 -04:00
2010-05-21 18:31:19 -04:00
2010-04-27 13:06:35 -04:00
2010-05-27 09:12:50 -07:00
2010-05-27 09:12:50 -07:00
2010-05-27 09:12:51 -07:00
2010-05-30 09:02:47 -07:00
2010-05-25 08:06:58 -07:00
2010-05-11 10:09:47 +02:00
2010-05-11 14:40:55 +02:00
2010-05-27 09:12:47 -07:00
2010-06-03 03:21:52 -07:00
2010-05-27 09:12:39 -07:00
2010-05-06 10:56:07 +10:00
2010-05-27 09:12:49 -07:00
2010-05-21 11:08:05 -07:00
2010-04-13 23:27:41 -07:00
2010-05-25 11:41:43 -04:00
2010-05-13 18:42:25 +09:00
2010-04-26 15:50:50 +09:00
2010-04-15 13:13:52 +09:00
2010-06-15 18:16:43 -07:00
2010-05-19 22:03:13 +03:00
2010-04-09 10:09:50 -07:00
2010-05-19 22:03:13 +03:00
2010-05-30 13:02:08 +03:00
2010-04-22 12:37:07 +02:00
2010-06-03 03:18:19 -07:00
2010-06-16 14:55:35 -07:00
2010-05-21 21:12:40 +02:00
2010-05-10 11:08:35 -07:00
2010-04-14 04:49:51 -07:00
2010-05-08 17:12:33 +02:00
2010-05-27 09:12:43 -07:00
2010-05-27 09:12:52 -07:00
2010-05-14 14:20:34 -04:00
2010-05-15 23:28:39 -07:00
2010-05-21 09:37:31 -07:00
2010-04-13 23:26:02 -07:00
2010-05-19 11:36:34 +03:00
2010-05-03 23:50:42 -07:00
2010-05-27 09:12:51 -07:00
2010-05-09 19:35:27 +02:00
2010-04-13 12:43:42 +02:00
2010-05-12 23:02:23 -07:00
2010-05-12 23:02:23 -07:00
2010-05-27 09:12:57 -07:00
2010-05-14 09:50:34 -04:00
2010-05-21 09:34:29 -07:00
2010-05-17 07:57:27 -07:00
2010-06-23 12:59:47 -07:00
2010-05-20 23:05:28 -07:00
2010-05-20 13:21:50 -07:00
2010-06-16 14:55:34 -07:00
2010-05-19 22:40:47 -04:00
2010-05-11 16:07:59 -06:00
2010-05-11 16:07:59 -06:00
2010-05-11 16:07:59 -06:00
2010-05-19 12:58:37 -03:00
2010-05-19 22:15:40 +09:30
2010-05-19 22:15:49 +09:30
2010-05-19 22:15:46 +09:30
2010-05-25 08:06:59 -07:00
2010-05-20 21:26:12 -07:00
2010-06-03 03:21:52 -07:00
2010-05-21 20:01:03 +02:00
2010-05-21 18:31:18 -04:00
2010-05-17 21:37:42 +02:00
01f2f3f6ef