Download raw body.
UDP send in parallel
On 2024-01-05 23:06 +01, Alexander Bluhm <alexander.bluhm@gmx.net> wrote:
> Hi,
>
> Sending UDP packets via datagram socket is MP safe now. Same applies
> to raw IPv4 and IPv6, and divert sockets. Switch sosend() from
> exclusive net lock to shared net lock in combination with per socket
> lock. TCP and GRE still use exclusive net lock.
>
> Please test this diff if you have applications that run multithreaded
> and send lots of UDP packets.
So I ran dnsbench and things do not look good.
current:
1 client(s) for 120s
Queries per second: 25202.641044
2 client(s) for 120s
Queries per second: 26331.337927
4 client(s) for 120s
Queries per second: 25651.039089
8 client(s) for 120s
Queries per second: 26005.807446
10 client(s) for 120s
Queries per second: 25824.787767
1 client(s) for 120s with DO bit set
Queries per second: 25573.938322
2 client(s) for 120s with DO bit set
Queries per second: 25383.326988
4 client(s) for 120s with DO bit set
Queries per second: 25284.565750
8 client(s) for 120s with DO bit set
Queries per second: 24415.760138
10 client(s) for 120s with DO bit set
Queries per second: 25475.886520
With the diff:
1 client(s) for 120s
Queries per second: 1620.002447
2 client(s) for 120s
Queries per second: 4526.044116
4 client(s) for 120s
Queries per second: 7079.784528
8 client(s) for 120s
Queries per second: 9210.908147
10 client(s) for 120s
Queries per second: 14360.313588
1 client(s) for 120s with DO bit set
Queries per second: 5997.335569
2 client(s) for 120s with DO bit set
Queries per second: 7326.934034
4 client(s) for 120s with DO bit set
Queries per second: 2622.538847
8 client(s) for 120s with DO bit set
Queries per second: 8985.788030
10 client(s) for 120s with DO bit set
Queries per second: 2591.206131
I ran dnsbench on a different machine. The target is a random amd64 kvm
VM I have lying around with 8 cores. I'm starting 8 nsd servers.
I'm adding the full log, nsd.conf and some scripts:
current:
========
1 client(s) for 120s
Statistics:
Queries sent: 3026586
Queries completed: 3024320 (99.93%)
Queries lost: 2266 (0.07%)
Response codes: NOERROR 3024320 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000122
Queries per second: 25202.641044
Average Latency (s): 0.000258 (min 0.000054, max 3.222699)
Latency StdDev (s): 0.002885
2 client(s) for 120s
Statistics:
Queries sent: 3161990
Queries completed: 3159763 (99.93%)
Queries lost: 2227 (0.07%)
Response codes: NOERROR 3159763 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000093
Queries per second: 26331.337927
Average Latency (s): 0.000289 (min 0.000052, max 0.374885)
Latency StdDev (s): 0.001140
4 client(s) for 120s
Statistics:
Queries sent: 3080387
Queries completed: 3078129 (99.93%)
Queries lost: 2258 (0.07%)
Response codes: NOERROR 3078129 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000168
Queries per second: 25651.039089
Average Latency (s): 0.000274 (min 0.000052, max 0.233587)
Latency StdDev (s): 0.000842
8 client(s) for 120s
Statistics:
Queries sent: 3122931
Queries completed: 3120699 (99.93%)
Queries lost: 2232 (0.07%)
Response codes: NOERROR 3120699 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000081
Queries per second: 26005.807446
Average Latency (s): 0.000281 (min 0.000051, max 0.625339)
Latency StdDev (s): 0.001165
10 client(s) for 120s
Statistics:
Queries sent: 3101371
Queries completed: 3099152 (99.93%)
Queries lost: 2219 (0.07%)
Response codes: NOERROR 3099152 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.006872
Queries per second: 25824.787767
Average Latency (s): 0.000292 (min 0.000053, max 1.417694)
Latency StdDev (s): 0.001889
1 client(s) for 120s with DO bit set
Statistics:
Queries sent: 3071148
Queries completed: 3068876 (99.93%)
Queries lost: 2272 (0.07%)
Response codes: NOERROR 3068876 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000133
Queries per second: 25573.938322
Average Latency (s): 0.000251 (min 0.000042, max 0.801752)
Latency StdDev (s): 0.001281
2 client(s) for 120s with DO bit set
Statistics:
Queries sent: 3048263
Queries completed: 3046000 (99.93%)
Queries lost: 2263 (0.07%)
Response codes: NOERROR 3046000 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000030
Queries per second: 25383.326988
Average Latency (s): 0.000280 (min 0.000055, max 0.366167)
Latency StdDev (s): 0.000907
4 client(s) for 120s with DO bit set
Statistics:
Queries sent: 3036399
Queries completed: 3034151 (99.93%)
Queries lost: 2248 (0.07%)
Response codes: NOERROR 3034151 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000123
Queries per second: 25284.565750
Average Latency (s): 0.000288 (min 0.000051, max 0.536886)
Latency StdDev (s): 0.001050
8 client(s) for 120s with DO bit set
Statistics:
Queries sent: 2932157
Queries completed: 2929894 (99.92%)
Queries lost: 2263 (0.08%)
Response codes: NOERROR 2929894 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000114
Queries per second: 24415.760138
Average Latency (s): 0.000277 (min 0.000052, max 2.126242)
Latency StdDev (s): 0.002415
10 client(s) for 120s with DO bit set
Statistics:
Queries sent: 3059310
Queries completed: 3057110 (99.93%)
Queries lost: 2200 (0.07%)
Response codes: NOERROR 3057110 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000142
Queries per second: 25475.886520
Average Latency (s): 0.000314 (min 0.000058, max 1.122629)
Latency StdDev (s): 0.001273
------------------------------------------------------------------------
with the diff
=============
1 client(s) for 120s
Statistics:
Queries sent: 196737
Queries completed: 194415 (98.82%)
Queries lost: 2322 (1.18%)
Response codes: NOERROR 194415 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.009078
Queries per second: 1620.002447
Average Latency (s): 0.002164 (min 0.000060, max 4.914079)
Latency StdDev (s): 0.093896
2 client(s) for 120s
Statistics:
Queries sent: 545388
Queries completed: 543126 (99.59%)
Queries lost: 2262 (0.41%)
Response codes: NOERROR 543126 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000156
Queries per second: 4526.044116
Average Latency (s): 0.001541 (min 0.000060, max 4.913738)
Latency StdDev (s): 0.069487
4 client(s) for 120s
Statistics:
Queries sent: 851846
Queries completed: 849575 (99.73%)
Queries lost: 2271 (0.27%)
Response codes: NOERROR 849575 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000121
Queries per second: 7079.784528
Average Latency (s): 0.000986 (min 0.000055, max 4.909502)
Latency StdDev (s): 0.050539
8 client(s) for 120s
Statistics:
Queries sent: 1107606
Queries completed: 1105310 (99.79%)
Queries lost: 2296 (0.21%)
Response codes: NOERROR 1105310 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000111
Queries per second: 9210.908147
Average Latency (s): 0.000709 (min 0.000056, max 4.911321)
Latency StdDev (s): 0.035660
10 client(s) for 120s
Statistics:
Queries sent: 1725511
Queries completed: 1723240 (99.87%)
Queries lost: 2271 (0.13%)
Response codes: NOERROR 1723240 (100.00%)
Average packet size: request 30, response 348
Run time (s): 120.000165
Queries per second: 14360.313588
Average Latency (s): 0.000469 (min 0.000059, max 4.910076)
Latency StdDev (s): 0.025829
1 client(s) for 120s with DO bit set
Statistics:
Queries sent: 722037
Queries completed: 719681 (99.67%)
Queries lost: 2356 (0.33%)
Response codes: NOERROR 719681 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000122
Queries per second: 5997.335569
Average Latency (s): 0.000454 (min 0.000054, max 4.915879)
Latency StdDev (s): 0.027013
2 client(s) for 120s with DO bit set
Statistics:
Queries sent: 881534
Queries completed: 879233 (99.74%)
Queries lost: 2301 (0.26%)
Response codes: NOERROR 879233 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000125
Queries per second: 7326.934034
Average Latency (s): 0.000693 (min 0.000059, max 4.999180)
Latency StdDev (s): 0.041681
4 client(s) for 120s with DO bit set
Statistics:
Queries sent: 317082
Queries completed: 314705 (99.25%)
Queries lost: 2377 (0.75%)
Response codes: NOERROR 314705 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000129
Queries per second: 2622.538847
Average Latency (s): 0.000791 (min 0.000057, max 4.816053)
Latency StdDev (s): 0.031375
8 client(s) for 120s with DO bit set
Statistics:
Queries sent: 1080608
Queries completed: 1078300 (99.79%)
Queries lost: 2308 (0.21%)
Response codes: NOERROR 1078300 (100.00%)
Average packet size: request 41, response 708
Run time (s): 120.000605
Queries per second: 8985.788030
Average Latency (s): 0.000602 (min 0.000055, max 4.913466)
Latency StdDev (s): 0.034349
10 client(s) for 120s with DO bit set
Statistics:
Queries sent: 313334
Queries completed: 310945 (99.24%)
Queries lost: 2389 (0.76%)
Response codes: NOERROR 310945 (100.00%)
Average packet size: request 41, response 709
Run time (s): 120.000102
Queries per second: 2591.206131
Average Latency (s): 0.000515 (min 0.000065, max 4.206721)
Latency StdDev (s): 0.013964
------------------------------------------------------------------------
$ cat gen_input.sh
#! /bin/sh
dig +noall +answer +onesoa @k.root-servers.net AXFR . |
awk '$4 !~ /NSEC|RRSIG|ZONEMD/ {print $1,$4}' | sort -u
------------------------------------------------------------------------
$ cat run.sh
#! /bin/sh
RUNTIME=120
for NUMCLIENTS in 1 2 4 8 10; do
echo ${NUMCLIENTS} 'client(s)' for ${RUNTIME}s
dnsperf -c ${NUMCLIENTS} -l ${RUNTIME} -d dnsperf.input -s $1 |
sed -n '/^Statistics/,$p'
done
for NUMCLIENTS in 1 2 4 8 10; do
echo ${NUMCLIENTS} 'client(s)' for ${RUNTIME}s with DO bit set
dnsperf -D -c ${NUMCLIENTS} -l ${RUNTIME} -d dnsperf.input -s $1 |
sed -n '/^Statistics/,$p'
done
------------------------------------------------------------------------
nsd.conf:
server:
server-count: 8
hide-version: yes
verbosity: 1
database: "" # disable database
ip-address: xxx
remote-control:
control-enable: yes
control-interface: /var/run/nsd.sock
zone:
name: "."
request-xfr: 192.228.79.201 NOKEY # b.root-servers.net
request-xfr: 192.33.4.12 NOKEY # c.root-servers.net
request-xfr: 192.5.5.241 NOKEY # f.root-servers.net
request-xfr: 192.112.36.4 NOKEY # g.root-servers.net
request-xfr: 193.0.14.129 NOKEY # k.root-servers.net
request-xfr: 192.0.47.132 NOKEY # xfr.cjr.dns.icann.org
request-xfr: 192.0.32.132 NOKEY # xfr.lax.dns.icann.org
request-xfr: 2001:500:84::b NOKEY # b.root-servers.net
request-xfr: 2001:500:2f::f NOKEY # f.root-servers.net
request-xfr: 2001:7fd::1 NOKEY # k.root-servers.net
request-xfr: 2620:0:2830:202::132 NOKEY # xfr.cjr.dns.icann.org
request-xfr: 2620:0:2d0:202::132 NOKEY # xfr.lax.dns.icann.org
--
In my defence, I have been left unsupervised.
UDP send in parallel