Index | Thread | Search

From:
Florian Obser <florian@openbsd.org>
Subject:
Re: UDP send in parallel
To:
Alexander Bluhm <alexander.bluhm@gmx.net>
Cc:
tech@openbsd.org
Date:
Mon, 08 Jan 2024 11:25:18 +0100

Download raw body.

Thread
On 2024-01-05 23:06 +01, Alexander Bluhm <alexander.bluhm@gmx.net> wrote:
> Hi,
>
> Sending UDP packets via datagram socket is MP safe now.  Same applies
> to raw IPv4 and IPv6, and divert sockets.  Switch sosend() from
> exclusive net lock to shared net lock in combination with per socket
> lock.  TCP and GRE still use exclusive net lock.
>
> Please test this diff if you have applications that run multithreaded
> and send lots of UDP packets.

So I ran dnsbench and things do not look good.

current:
1 client(s) for 120s
  Queries per second:   25202.641044
2 client(s) for 120s
  Queries per second:   26331.337927
4 client(s) for 120s
  Queries per second:   25651.039089
8 client(s) for 120s
  Queries per second:   26005.807446
10 client(s) for 120s
  Queries per second:   25824.787767
1 client(s) for 120s with DO bit set
  Queries per second:   25573.938322
2 client(s) for 120s with DO bit set
  Queries per second:   25383.326988
4 client(s) for 120s with DO bit set
  Queries per second:   25284.565750
8 client(s) for 120s with DO bit set
  Queries per second:   24415.760138
10 client(s) for 120s with DO bit set
  Queries per second:   25475.886520

With the diff:
1 client(s) for 120s
  Queries per second:   1620.002447
2 client(s) for 120s
  Queries per second:   4526.044116
4 client(s) for 120s
  Queries per second:   7079.784528
8 client(s) for 120s
  Queries per second:   9210.908147
10 client(s) for 120s
  Queries per second:   14360.313588
1 client(s) for 120s with DO bit set
  Queries per second:   5997.335569
2 client(s) for 120s with DO bit set
  Queries per second:   7326.934034
4 client(s) for 120s with DO bit set
  Queries per second:   2622.538847
8 client(s) for 120s with DO bit set
  Queries per second:   8985.788030
10 client(s) for 120s with DO bit set
  Queries per second:   2591.206131

I ran dnsbench on a different machine. The target is a random amd64 kvm
VM I have lying around with 8 cores. I'm starting 8 nsd servers.

I'm adding the full log, nsd.conf and some scripts:

current:
========
1 client(s) for 120s
Statistics:

  Queries sent:         3026586
  Queries completed:    3024320 (99.93%)
  Queries lost:         2266 (0.07%)

  Response codes:       NOERROR 3024320 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000122
  Queries per second:   25202.641044

  Average Latency (s):  0.000258 (min 0.000054, max 3.222699)
  Latency StdDev (s):   0.002885

2 client(s) for 120s
Statistics:

  Queries sent:         3161990
  Queries completed:    3159763 (99.93%)
  Queries lost:         2227 (0.07%)

  Response codes:       NOERROR 3159763 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000093
  Queries per second:   26331.337927

  Average Latency (s):  0.000289 (min 0.000052, max 0.374885)
  Latency StdDev (s):   0.001140

4 client(s) for 120s
Statistics:

  Queries sent:         3080387
  Queries completed:    3078129 (99.93%)
  Queries lost:         2258 (0.07%)

  Response codes:       NOERROR 3078129 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000168
  Queries per second:   25651.039089

  Average Latency (s):  0.000274 (min 0.000052, max 0.233587)
  Latency StdDev (s):   0.000842

8 client(s) for 120s
Statistics:

  Queries sent:         3122931
  Queries completed:    3120699 (99.93%)
  Queries lost:         2232 (0.07%)

  Response codes:       NOERROR 3120699 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000081
  Queries per second:   26005.807446

  Average Latency (s):  0.000281 (min 0.000051, max 0.625339)
  Latency StdDev (s):   0.001165

10 client(s) for 120s
Statistics:

  Queries sent:         3101371
  Queries completed:    3099152 (99.93%)
  Queries lost:         2219 (0.07%)

  Response codes:       NOERROR 3099152 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.006872
  Queries per second:   25824.787767

  Average Latency (s):  0.000292 (min 0.000053, max 1.417694)
  Latency StdDev (s):   0.001889

1 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         3071148
  Queries completed:    3068876 (99.93%)
  Queries lost:         2272 (0.07%)

  Response codes:       NOERROR 3068876 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000133
  Queries per second:   25573.938322

  Average Latency (s):  0.000251 (min 0.000042, max 0.801752)
  Latency StdDev (s):   0.001281

2 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         3048263
  Queries completed:    3046000 (99.93%)
  Queries lost:         2263 (0.07%)

  Response codes:       NOERROR 3046000 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000030
  Queries per second:   25383.326988

  Average Latency (s):  0.000280 (min 0.000055, max 0.366167)
  Latency StdDev (s):   0.000907

4 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         3036399
  Queries completed:    3034151 (99.93%)
  Queries lost:         2248 (0.07%)

  Response codes:       NOERROR 3034151 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000123
  Queries per second:   25284.565750

  Average Latency (s):  0.000288 (min 0.000051, max 0.536886)
  Latency StdDev (s):   0.001050

8 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         2932157
  Queries completed:    2929894 (99.92%)
  Queries lost:         2263 (0.08%)

  Response codes:       NOERROR 2929894 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000114
  Queries per second:   24415.760138

  Average Latency (s):  0.000277 (min 0.000052, max 2.126242)
  Latency StdDev (s):   0.002415

10 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         3059310
  Queries completed:    3057110 (99.93%)
  Queries lost:         2200 (0.07%)

  Response codes:       NOERROR 3057110 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000142
  Queries per second:   25475.886520

  Average Latency (s):  0.000314 (min 0.000058, max 1.122629)
  Latency StdDev (s):   0.001273

------------------------------------------------------------------------

with the diff
=============

1 client(s) for 120s
Statistics:

  Queries sent:         196737
  Queries completed:    194415 (98.82%)
  Queries lost:         2322 (1.18%)

  Response codes:       NOERROR 194415 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.009078
  Queries per second:   1620.002447

  Average Latency (s):  0.002164 (min 0.000060, max 4.914079)
  Latency StdDev (s):   0.093896

2 client(s) for 120s
Statistics:

  Queries sent:         545388
  Queries completed:    543126 (99.59%)
  Queries lost:         2262 (0.41%)

  Response codes:       NOERROR 543126 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000156
  Queries per second:   4526.044116

  Average Latency (s):  0.001541 (min 0.000060, max 4.913738)
  Latency StdDev (s):   0.069487

4 client(s) for 120s
Statistics:

  Queries sent:         851846
  Queries completed:    849575 (99.73%)
  Queries lost:         2271 (0.27%)

  Response codes:       NOERROR 849575 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000121
  Queries per second:   7079.784528

  Average Latency (s):  0.000986 (min 0.000055, max 4.909502)
  Latency StdDev (s):   0.050539

8 client(s) for 120s
Statistics:

  Queries sent:         1107606
  Queries completed:    1105310 (99.79%)
  Queries lost:         2296 (0.21%)

  Response codes:       NOERROR 1105310 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000111
  Queries per second:   9210.908147

  Average Latency (s):  0.000709 (min 0.000056, max 4.911321)
  Latency StdDev (s):   0.035660

10 client(s) for 120s
Statistics:

  Queries sent:         1725511
  Queries completed:    1723240 (99.87%)
  Queries lost:         2271 (0.13%)

  Response codes:       NOERROR 1723240 (100.00%)
  Average packet size:  request 30, response 348
  Run time (s):         120.000165
  Queries per second:   14360.313588

  Average Latency (s):  0.000469 (min 0.000059, max 4.910076)
  Latency StdDev (s):   0.025829

1 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         722037
  Queries completed:    719681 (99.67%)
  Queries lost:         2356 (0.33%)

  Response codes:       NOERROR 719681 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000122
  Queries per second:   5997.335569

  Average Latency (s):  0.000454 (min 0.000054, max 4.915879)
  Latency StdDev (s):   0.027013

2 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         881534
  Queries completed:    879233 (99.74%)
  Queries lost:         2301 (0.26%)

  Response codes:       NOERROR 879233 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000125
  Queries per second:   7326.934034

  Average Latency (s):  0.000693 (min 0.000059, max 4.999180)
  Latency StdDev (s):   0.041681

4 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         317082
  Queries completed:    314705 (99.25%)
  Queries lost:         2377 (0.75%)

  Response codes:       NOERROR 314705 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000129
  Queries per second:   2622.538847

  Average Latency (s):  0.000791 (min 0.000057, max 4.816053)
  Latency StdDev (s):   0.031375

8 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         1080608
  Queries completed:    1078300 (99.79%)
  Queries lost:         2308 (0.21%)

  Response codes:       NOERROR 1078300 (100.00%)
  Average packet size:  request 41, response 708
  Run time (s):         120.000605
  Queries per second:   8985.788030

  Average Latency (s):  0.000602 (min 0.000055, max 4.913466)
  Latency StdDev (s):   0.034349

10 client(s) for 120s with DO bit set
Statistics:

  Queries sent:         313334
  Queries completed:    310945 (99.24%)
  Queries lost:         2389 (0.76%)

  Response codes:       NOERROR 310945 (100.00%)
  Average packet size:  request 41, response 709
  Run time (s):         120.000102
  Queries per second:   2591.206131

  Average Latency (s):  0.000515 (min 0.000065, max 4.206721)
  Latency StdDev (s):   0.013964


------------------------------------------------------------------------

$ cat gen_input.sh
#! /bin/sh

dig +noall +answer +onesoa @k.root-servers.net AXFR .  |
    awk '$4 !~ /NSEC|RRSIG|ZONEMD/ {print $1,$4}' | sort -u

------------------------------------------------------------------------

$ cat run.sh
#! /bin/sh

RUNTIME=120

for NUMCLIENTS in 1 2 4 8 10; do
	echo ${NUMCLIENTS} 'client(s)' for ${RUNTIME}s
	dnsperf -c ${NUMCLIENTS} -l ${RUNTIME} -d dnsperf.input -s $1 |
	    sed -n '/^Statistics/,$p'
done

for NUMCLIENTS in 1 2 4 8 10; do
	echo ${NUMCLIENTS} 'client(s)' for ${RUNTIME}s with DO bit set
	dnsperf -D -c ${NUMCLIENTS} -l ${RUNTIME} -d dnsperf.input -s $1 |
	    sed -n '/^Statistics/,$p'
done


------------------------------------------------------------------------

nsd.conf:

server:
	server-count: 8
	hide-version: yes
	verbosity: 1
	database: "" # disable database
	ip-address: xxx
remote-control:
	control-enable: yes
	control-interface: /var/run/nsd.sock
zone:
	name: "."
	request-xfr: 192.228.79.201 NOKEY # b.root-servers.net
	request-xfr: 192.33.4.12 NOKEY    # c.root-servers.net
	request-xfr: 192.5.5.241 NOKEY    # f.root-servers.net
	request-xfr: 192.112.36.4 NOKEY   # g.root-servers.net
	request-xfr: 193.0.14.129 NOKEY   # k.root-servers.net
	request-xfr: 192.0.47.132 NOKEY   # xfr.cjr.dns.icann.org
	request-xfr: 192.0.32.132 NOKEY   # xfr.lax.dns.icann.org
	request-xfr: 2001:500:84::b NOKEY # b.root-servers.net
	request-xfr: 2001:500:2f::f NOKEY # f.root-servers.net
	request-xfr: 2001:7fd::1 NOKEY    # k.root-servers.net
	request-xfr: 2620:0:2830:202::132 NOKEY  # xfr.cjr.dns.icann.org
	request-xfr: 2620:0:2d0:202::132 NOKEY  # xfr.lax.dns.icann.org


-- 
In my defence, I have been left unsupervised.