Eric Dumazet bbaffaca48
rcu: Introduce hlist_nulls variant of hlist ...
hlist uses NULL value to finish a chain.
hlist_nulls variant use the low order bit set to 1 to signal an end-of-list marker.
This allows to store many different end markers, so that some RCU lockless
algos (used in TCP/UDP stack for example) can save some memory barriers in
fast paths.
Two new files are added :
include/linux/list_nulls.h
- mimics hlist part of include/linux/list.h, derived to hlist_nulls variant
include/linux/rculist_nulls.h
- mimics hlist part of include/linux/rculist.h, derived to hlist_nulls variant
Only four helpers are declared for the moment :
hlist_nulls_del_init_rcu(), hlist_nulls_del_rcu(),
hlist_nulls_add_head_rcu() and hlist_nulls_for_each_entry_rcu()
prefetches() were removed, since an end of list is not anymore NULL value.
prefetches() could trigger useless (and possibly dangerous) memory transactions.
Example of use (extracted from __udp4_lib_lookup())
struct sock *sk, *result;
struct hlist_nulls_node *node;
unsigned short hnum = ntohs(dport);
unsigned int hash = udp_hashfn(net, hnum);
struct udp_hslot *hslot = &udptable->hash[hash];
int score, badness;
rcu_read_lock();
begin:
result = NULL;
badness = -1;
sk_nulls_for_each_rcu(sk, node, &hslot->head) {
score = compute_score(sk, net, saddr, hnum, sport,
daddr, dport, dif);
if (score > badness) {
result = sk;
badness = score;
}
}
/*
* if the nulls value we got at the end of this lookup is
* not the expected one, we must restart lookup.
* We probably met an item that was moved to another chain.
*/
if (get_nulls_value(node) != hash)
goto begin;
if (result) {
if (unlikely(!atomic_inc_not_zero(&result->sk_refcnt)))
result = NULL;
else if (unlikely(compute_score(result, net, saddr, hnum, sport,
daddr, dport, dif) < badness)) {
sock_put(result);
goto begin;
}
}
rcu_read_unlock();
return result;
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com >
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl >
Signed-off-by: David S. Miller <davem@davemloft.net >
2008-11-16 19:37:55 -08:00
2008-10-20 08:52:40 -07:00
2008-10-17 17:15:43 -03:00
2008-10-22 01:19:37 +02:00
2008-10-04 17:12:27 -04:00
2008-10-19 22:54:12 +02:00
2008-10-22 15:38:42 -07:00
2008-11-08 21:37:46 +01:00
2008-11-05 14:40:25 +01:00
2008-10-14 11:58:31 -07:00
2008-10-08 11:35:19 +02:00
2008-10-08 11:35:00 +02:00
2008-09-29 17:56:56 -04:00
2008-10-21 13:25:28 +11:00
2008-10-13 21:51:51 +01:00
2008-09-03 15:41:57 -07:00
2008-10-30 11:38:47 -07:00
2008-11-10 13:50:19 -08:00
2008-10-31 00:56:49 -07:00
2008-09-12 16:30:20 -07:00
2008-10-20 16:07:19 +01:00
2008-09-17 16:54:26 +01:00
2008-07-25 10:53:47 -07:00
2008-10-10 23:50:21 -04:00
2008-10-20 11:01:52 -07:00
2008-08-12 10:13:38 +10:00
2008-10-16 11:21:51 -07:00
2008-07-24 10:47:27 -07:00
2008-10-10 22:39:31 +02:00
2008-07-24 10:47:41 -07:00
2008-07-24 10:47:24 -07:00
2008-10-16 11:21:39 -07:00
2008-10-16 11:21:39 -07:00
2008-07-25 15:44:39 +10:00
2008-10-20 08:50:25 -07:00
2008-10-20 08:52:42 -07:00
2008-10-16 11:21:38 -07:00
2008-11-06 08:41:55 +01:00
2008-10-23 08:20:34 -07:00
2008-10-21 07:48:52 -04:00
2008-10-17 08:46:57 +02:00
2008-08-12 16:07:27 -07:00
2008-10-20 08:52:32 -07:00
2008-08-12 16:07:30 -07:00
2008-08-14 22:59:43 +10:00
2008-10-21 07:47:22 -04:00
2008-11-07 22:56:00 -08:00
2008-10-30 11:38:45 -07:00
2008-10-16 11:21:32 -07:00
2008-09-05 11:11:51 +02:00
2008-08-21 09:50:24 +02:00
2008-11-09 11:17:33 -08:00
2008-07-26 20:53:14 -04:00
2008-07-25 10:53:33 -07:00
2008-10-16 11:21:33 -07:00
2008-10-14 10:35:08 +02:00
2008-08-26 10:26:54 +02:00
2008-07-31 16:21:13 -07:00
2008-07-30 09:41:45 -07:00
2008-10-14 12:12:02 -07:00
2008-09-08 19:25:24 +02:00
2008-10-09 13:52:43 -04:00
2008-11-09 21:09:54 +01:00
2008-09-06 19:22:15 +02:00
2008-10-23 18:51:22 +04:00
2008-08-14 09:35:23 +10:00
2008-08-29 15:50:04 +10:00
2008-07-20 17:12:38 -07:00
2008-07-22 17:30:57 -07:00
2008-10-23 05:13:16 -04:00
2008-11-12 00:47:26 -08:00
2008-07-21 21:54:59 -07:00
2008-07-25 10:53:47 -07:00
2008-10-23 10:23:07 -07:00
2008-10-22 20:21:31 +02:00
2008-10-13 09:51:43 -07:00
2008-07-25 10:53:34 -07:00
2008-08-28 11:49:15 -05:00
2008-08-28 11:49:15 -05:00
2008-08-07 02:22:54 -04:00
2008-07-21 12:00:37 +01:00
2008-10-21 17:45:06 +01:00
2008-10-18 14:29:15 +01:00
2008-07-22 10:39:36 +10:00
2008-09-19 10:20:41 +02:00
2008-10-16 16:53:04 +02:00
2008-10-13 16:05:06 +01:00
2008-10-15 12:46:51 +01:00
2008-10-16 09:24:47 -07:00
2008-10-16 16:53:13 +02:00
2008-07-21 21:54:53 -07:00
2008-10-09 08:56:13 +02:00
2008-10-12 12:05:55 +02:00
2008-08-07 02:22:08 -04:00
2008-07-24 10:47:29 -07:00
2008-07-24 10:47:29 -07:00
2008-08-20 13:19:51 -07:00
2008-10-16 11:21:30 -07:00
2008-10-20 08:52:37 -07:00
2008-10-16 11:21:30 -07:00
2008-10-20 08:52:36 -07:00
2008-10-09 08:56:19 +02:00
2008-10-08 19:44:18 -04:00
2008-10-21 07:47:06 -04:00
2008-10-15 22:21:11 +02:00
2008-08-12 16:07:31 -07:00
2008-10-30 11:38:45 -07:00
2008-07-22 19:09:51 -07:00
2008-07-26 20:53:20 -04:00
2008-08-23 12:14:12 -07:00
2008-10-30 11:38:45 -07:00
2008-10-31 00:59:46 -04:00
2008-10-21 07:47:06 -04:00
2008-10-23 16:00:24 +02:00
2008-10-16 16:08:57 +02:00
2008-07-30 00:41:37 -04:00
2008-07-20 17:12:36 -07:00
2008-10-23 17:57:37 +04:00
2008-07-24 10:47:20 -07:00
2008-09-22 07:29:31 +01:00
2008-10-16 11:21:40 -07:00
2008-07-20 17:12:36 -07:00
2008-10-27 15:06:01 +01:00
2008-10-14 23:51:02 +02:00
2008-07-31 18:45:41 +02:00
2008-11-06 15:44:19 -08:00
2008-10-23 13:52:40 +04:00
2008-10-22 20:21:30 +02:00
2008-10-17 17:25:11 -03:00
2008-08-08 15:13:27 +01:00
2008-10-22 20:21:33 +02:00
2008-10-16 11:21:38 -07:00
2008-10-24 12:54:18 -04:00
2008-10-15 16:03:01 -07:00
2008-10-21 07:47:30 -04:00
2008-07-25 10:53:42 -07:00
2008-11-10 15:17:41 -05:00
2008-10-13 19:01:08 -07:00
2008-10-13 19:01:08 -07:00
2008-10-13 19:01:08 -07:00
2008-09-22 21:28:11 -07:00
2008-10-01 01:30:19 -07:00
2008-08-15 19:52:19 -07:00
2008-10-09 11:59:55 -07:00
2008-11-04 14:49:57 -08:00
2008-11-03 18:16:50 -08:00
2008-10-13 19:01:08 -07:00
2008-08-02 18:36:10 +01:00
2008-11-16 19:32:39 -08:00
2008-10-07 15:34:37 -07:00
2008-09-05 21:35:30 -07:00
2008-10-29 22:02:09 +01:00
2008-07-24 10:47:29 -07:00
2008-07-30 00:41:40 -04:00
2008-10-18 14:29:15 +01:00
2008-10-20 13:23:01 -07:00
2008-11-03 18:21:45 +01:00
2008-10-16 11:21:33 -07:00
2008-10-16 15:17:40 -07:00
2008-10-15 14:24:08 +02:00
2008-08-15 09:26:14 +10:00
2008-07-25 10:53:42 -07:00
2008-07-21 13:41:16 -07:00
2008-10-18 14:05:18 +02:00
2008-07-25 10:53:26 -07:00
2008-10-21 15:54:40 +02:00
2008-09-22 01:15:02 -07:00
2008-07-20 17:12:37 -07:00
2008-10-12 09:37:01 -02:00
2008-08-16 16:46:57 -07:00
2008-10-16 20:00:24 -04:00
2008-10-23 08:55:01 -07:00
2008-10-16 11:21:32 -07:00
2008-10-16 11:21:31 -07:00
2008-10-20 08:52:40 -07:00
2008-10-20 13:23:01 -07:00
2008-10-31 00:17:34 -07:00
2008-08-15 08:35:42 -07:00
2008-10-12 11:44:37 -07:00
2008-10-09 08:56:04 +02:00
2008-10-16 11:21:29 -07:00
2008-07-24 10:47:17 -07:00
2008-10-14 10:34:22 +02:00
2008-07-24 19:11:15 -07:00
2008-10-28 14:21:34 +02:00
2008-10-28 14:22:15 +02:00
2008-09-23 22:01:33 +01:00
2008-07-23 09:49:56 +01:00
2008-10-20 22:34:12 +01:00
2008-11-10 08:04:47 -08:00
2008-10-14 10:35:08 +02:00
2008-11-16 19:37:55 -08:00
2008-09-01 09:47:16 +02:00
2008-08-11 15:25:07 +02:00
2008-10-13 09:47:43 +11:00
2008-10-09 08:56:06 +02:00
2008-10-10 23:16:16 -04:00
2008-08-04 10:58:24 +09:00
2008-10-14 10:39:28 +02:00
2008-11-16 18:59:45 -08:00
2008-10-20 08:52:39 -07:00
2008-07-24 10:47:21 -07:00
2008-07-24 10:47:21 -07:00
2008-09-13 14:41:52 -07:00
2008-10-20 08:50:25 -07:00
2008-07-27 01:56:38 +02:00
2008-07-27 01:54:58 +02:00
2008-08-02 16:28:50 +02:00
2008-10-20 08:52:31 -07:00
2008-10-20 08:52:39 -07:00
2008-10-20 08:52:30 -07:00
2008-10-14 10:37:17 +02:00
2008-07-28 16:30:21 -07:00
2008-10-20 08:52:39 -07:00
2008-10-29 00:51:25 +01:00
2008-10-22 10:00:23 +11:00
2008-10-22 10:00:23 +11:00
2008-10-16 11:21:30 -07:00
2008-08-29 14:37:23 -07:00
2008-08-29 14:37:23 -07:00
2008-11-06 15:41:21 -08:00
2008-09-05 06:33:59 +02:00
2008-10-23 05:13:20 -04:00
2008-08-26 10:37:20 -07:00
2008-11-07 22:54:20 -08:00
2008-10-08 11:35:01 +02:00
2008-10-01 07:03:24 -07:00
2008-10-06 20:08:26 -04:00
2008-10-17 13:06:45 -04:00
2008-10-07 17:22:20 -04:00
2008-10-14 19:23:17 -04:00
2008-11-10 15:17:40 -05:00
2008-09-08 19:25:24 +02:00
2008-07-25 10:53:37 -07:00
2008-07-22 10:39:30 +10:00
2008-07-25 22:34:40 -04:00
2008-10-13 10:55:47 +11:00
2008-10-21 15:17:48 +11:00
2008-10-23 08:55:02 -07:00
2008-10-20 08:52:30 -07:00
2008-10-20 08:52:32 -07:00
2008-10-20 08:50:26 -07:00
2008-10-16 11:21:49 -07:00
2008-10-13 10:10:37 -07:00
2008-10-22 16:42:43 -07:00
2008-10-23 10:20:36 -07:00
2008-10-22 16:42:35 -07:00
2008-08-18 13:48:04 -07:00
2008-07-28 14:56:09 -07:00
2008-11-03 14:31:18 -08:00
2008-10-09 12:50:59 -04:00
2008-09-05 09:24:30 +02:00
2008-10-05 13:33:42 -07:00
2008-09-14 17:24:26 +02:00
2008-10-26 23:07:25 -07:00
2008-10-08 16:38:41 -07:00
2008-10-16 11:21:48 -07:00
2008-08-20 15:40:32 -07:00
2008-08-29 14:37:23 -07:00
2008-11-07 22:56:00 -08:00
2008-09-12 16:29:34 -07:00
2008-10-16 09:24:51 -07:00
2008-08-05 14:33:50 -07:00
2008-10-16 11:21:30 -07:00
2008-10-23 00:11:07 -04:00
2008-09-05 21:35:03 -07:00
2008-09-24 15:45:48 +02:00
2008-09-01 02:42:54 +04:00
2008-09-05 21:35:30 -07:00
2008-10-23 18:54:05 +04:00
2008-10-23 08:55:01 -07:00
2008-08-15 17:15:23 +02:00
2008-10-20 08:52:39 -07:00
2008-09-02 19:21:38 -07:00
2008-10-16 11:21:46 -07:00
2008-10-16 11:21:46 -07:00
2008-07-26 12:00:06 -07:00
2008-10-02 15:53:13 -07:00
2008-07-25 10:53:29 -07:00
2008-10-03 10:36:08 +02:00
2008-11-16 19:37:55 -08:00
2008-10-29 11:19:58 -07:00
2008-09-30 12:08:41 +02:00
2008-08-15 17:54:39 +02:00
2008-08-15 08:35:42 -07:00
2008-10-21 07:49:04 -04:00
2008-10-16 11:21:30 -07:00
2008-07-26 20:53:14 -04:00
2008-07-26 12:00:04 -07:00
2008-09-05 14:39:37 -07:00
2008-10-30 11:38:45 -07:00
2008-08-22 16:29:57 -04:00
2008-10-14 10:39:05 +02:00
2008-10-20 08:52:40 -07:00
2008-07-26 12:00:12 -07:00
2008-10-16 11:21:30 -07:00
2008-09-22 19:51:15 -07:00
2008-07-23 14:42:09 +02:00
2008-11-11 08:01:43 +01:00
2008-10-30 11:38:47 -07:00
2008-07-25 10:53:42 -07:00
2008-07-24 08:31:21 -04:00
2008-10-20 08:52:39 -07:00
2008-10-13 09:51:39 -07:00
2008-10-13 09:51:40 -07:00
2008-07-23 14:01:49 -04:00
2008-10-01 16:13:54 +09:00
2008-07-26 20:53:14 -04:00
2008-07-24 10:47:29 -07:00
2008-10-31 19:02:14 -04:00
2008-10-23 15:20:06 +04:00
2008-08-05 09:28:47 +03:00
2008-07-25 10:53:30 -07:00
2008-07-25 10:53:26 -07:00
2008-09-23 08:09:13 -07:00
2008-09-09 17:41:42 +02:00
2008-10-22 07:00:38 -04:00
2008-11-06 09:05:33 +01:00
2008-11-06 00:58:40 -05:00
2008-07-30 03:27:25 -07:00
2008-10-05 11:16:16 -07:00
2008-08-11 09:30:24 +02:00
2008-08-11 09:30:24 +02:00
2008-10-03 18:22:18 -07:00
2008-07-20 17:12:37 -07:00
2008-08-26 00:19:27 +10:00
2008-10-03 11:46:14 -05:00
2008-11-02 10:15:07 -08:00
2008-08-15 08:35:42 -07:00
2008-10-20 08:52:40 -07:00
2008-10-20 08:52:31 -07:00
2008-10-16 11:21:28 -07:00
2008-07-22 13:03:29 -07:00
2008-07-25 17:28:49 -07:00
2008-10-16 11:21:47 -07:00
2008-07-21 21:55:02 -07:00
2008-10-20 08:52:42 -07:00
2008-07-27 16:12:28 -07:00
2008-10-16 11:21:32 -07:00
2008-07-25 10:53:47 -07:00
2008-10-07 14:43:06 -07:00
2008-11-11 09:30:23 -08:00
2008-10-13 09:51:40 -07:00
2008-09-05 21:35:01 -07:00
2008-10-17 18:13:38 +02:00
2008-10-22 09:48:06 +02:00
2008-11-06 08:42:48 +01:00
2008-07-24 10:47:29 -07:00
2008-09-24 17:33:13 +02:00
2008-11-07 16:09:23 +01:00
2008-09-05 14:39:38 -07:00
2008-10-14 10:39:31 +02:00
2008-10-13 09:51:43 -07:00
2008-07-20 17:12:34 -07:00
2008-10-13 09:51:44 -07:00
2008-07-25 10:53:26 -07:00
2008-10-21 07:47:06 -04:00
2008-08-20 23:22:22 +01:00
2008-07-21 21:54:55 -07:00
2008-10-17 14:41:02 -07:00
2008-07-21 15:15:55 -07:00
2008-09-17 16:54:35 +01:00
2008-10-21 14:31:15 -02:00
2008-07-27 11:07:12 -03:00
2008-07-26 13:25:25 -03:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:05 +10:00
2008-07-25 12:06:12 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:04 +10:00
2008-07-25 12:06:14 +10:00
2008-07-25 12:06:04 +10:00
2008-10-23 15:48:28 +04:00
2008-10-23 17:35:04 +04:00
2008-10-13 09:51:42 -07:00
2008-10-16 11:21:31 -07:00
2008-09-17 16:54:23 +01:00
2008-10-02 22:48:34 +01:00
2008-11-06 09:05:33 +01:00
2008-10-16 10:09:17 -04:00
2008-10-28 16:01:07 -07:00
bbaffaca48