On Mon, 03 Feb 2025 21:28:12 +0100,
Vitaliy Makkoveev <mvs@openbsd.org> wrote:
> 
> > On 3 Feb 2025, at 20:25, Kirill A. Korinsky <kirill@korins.ky> wrote:
> > 
> > On Mon, 03 Feb 2025 11:44:42 +0100,
> > Vitaliy Makkoveev <mvs@openbsd.org> wrote:
> >> 
> >> First split them by filt_sor*() and filt_soe*() functions for
> >> `soread_filtops' and `soexcept_filtops' respectively. The
> >> filt_soexcept() path is like filt_sowrite() and the `sb_mtx'
> >> serialization is pretty enough, but filt_soread() is more complicated
> >> for the connection oriented sockets. It is not yet clean is the
> >> concurrent SO_ACCEPTCONN transition safe for select(2) and poll(2), so
> >> keep socket lock together with `sb_mtx' lock for that case.
> >> 
> >> Also get rid of sofilt_*lock() functions. They were temporary made to
> >> take `sb_mtx' mutex with shared netlock for inet sockets case or with
> >> socket lock for other cases. Now shared netlock is not enough and should
> >> be taken together with `so_lock' rwlock. We have special solock_shared()
> >> function for that purpose, so use it directly in corresponding
> >> filt_sor*() handlers.
> >> 
> >> Kirill, could you run sys/netinet/tcpthread/ and sys/kern/sosplice on
> >> your arm64 machine?
> >> 
> > 
> 
> Thanks for testing. The regress/sys/kern/sosplice was fixed with
> the ifq [1] diff I committed today. You already tested it by my private
> request. This diff should appear in the most recent snap. The diff from
> this thread only adds some performance. 
> 
> 1. http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/net/ifq.c.diff?r1=1.55&r2=1.56
>

Indeed, without your diff it doesn't fail on the kernel from cvs.

> > Sure. I've used current snapshot as baseline.
> > 
> > 1. regress/sys/netinet/tcpthread
> > 
> > Your diff improves tcpthread in term of sys column time. Significant.
> > 
> > your diff
> > 
> >        ==== run-default ====
> >        time  ./tcpthread
> >        count: connect 171, accept 170, send 118, recv 278, close 170, drop 0
> >               10.03 real        24.54 user         5.41 sys
> > 
> >        ==== run-sock100 ====
> >        ulimit -n 500; time  ./tcpthread -n100
> >        count: connect 174558, accept 9961, send 286661, recv 65474, close 45805, drop 0
> >            0m10.04s real     0m09.19s user     0m14.41s system
> > 
> >        ==== run-splice ====
> >        time  ./tcpthread -n10 -S2
> >        count: connect 170624, splice 0, unsplice 0, max 0, idle 0, accept 0, send 0, recv 85097, close 85527, drop 0
> >               10.12 real        23.98 user         7.31 sys
> > 
> >        ==== run-unsplice ====
> >        time  ./tcpthread -n10 -S2 -U1
> >        count: connect 142843, splice 0, unsplice 0, max 0, idle 0, accept 0, send 0, recv 71286, close 71547, drop 0
> >               10.12 real        33.44 user         6.74 sys
> > 
> >        ==== run-max ====
> >        time  ./tcpthread -n10 -S4 -M100 -s2 -r2
> >        count: connect 250205, splice 0, unsplice 0, max 0, idle 0, accept 0, send 4138, recv 124616, close 121443, drop 0
> >               10.13 real        38.94 user        11.47 sys
> > 
> >        ==== run-idle ====
> >        time  ./tcpthread -n10 -S4 -I100 -s2 -r2
> >        count: connect 325965, splice 0, unsplice 0, max 0, idle 0, accept 0, send 325972, recv 0, close 0, drop 0
> >               10.13 real        45.97 user         7.95 sys
> > 
> >        ==== run-drop ====
> >        time doas  ./tcpthread -n10 -D3 -o0
> >        count: connect 39142, accept 0, send 39492, recv 0, close 0, drop 0
> >               10.12 real        49.39 user         0.82 sys
> > 
> >        ==== run-splicedrop ====
> >        time doas  ./tcpthread -n10 -S2 -D3 -o0
> >        count: connect 61005, splice 0, unsplice 0, max 0, idle 0, accept 0, send 61727, recv 0, close 0, drop 0
> >               10.12 real        48.96 user         1.42 sys
> > 
> > and the snapshot:
> > 
> >        ==== run-default ====
> >        time  ./tcpthread
> >        count: connect 199, accept 198, send 164, recv 355, close 198, drop 0
> >               10.04 real        24.73 user         5.17 sys
> > 
> >        ==== run-sock100 ====
> >        ulimit -n 500; time  ./tcpthread -n100
> >        count: connect 12079, accept 12079, send 6561, recv 18550, close 12029, drop 0
> >            0m10.43s real     0m21.34s user     0m10.07s system
> > 
> >        ==== run-splice ====
> >        time  ./tcpthread -n10 -S2
> >        count: connect 39071, splice 129, unsplice 0, max 0, idle 0, accept 1222, send 626064, recv 474297, close 44484, drop 0
> >               10.04 real         9.38 user        10.21 sys
> > 
> >        ==== run-unsplice ====
> >        time  ./tcpthread -n10 -S2 -U1
> >        count: connect 321093, splice 12, unsplice 749, max 0, idle 0, accept 10, send 52780, recv 11, close 276654, drop 0
> >               10.04 real        26.03 user        17.15 sys
> > 
> >        ==== run-max ====
> >        time  ./tcpthread -n10 -S4 -M100 -s2 -r2
> >        count: connect 20, splice 6, unsplice 0, max 0, idle 0, accept 9, send 5, recv 12, close 13, drop 0
> >               10.10 real        31.41 user        18.78 sys
> > 
> >        ==== run-idle ====
> >        time  ./tcpthread -n10 -S4 -I100 -s2 -r2
> >        count: connect 298272, splice 349, unsplice 0, max 0, idle 7, accept 506, send 43897, recv 28, close 273104, drop 0
> >               10.04 real        15.28 user        37.83 sys
> > 
> >        ==== run-drop ====
> >        time doas  ./tcpthread -n10 -D3 -o0
> >        count: connect 37, accept 10, send 319, recv 6, close 0, drop 10
> >               10.02 real         1.92 user        42.07 sys
> > 
> >        ==== run-splicedrop ====
> >        time doas  ./tcpthread -n10 -S2 -D3 -o0
> >        count: connect 1821, splice 427, unsplice 0, max 0, idle 0, accept 516, send 53614, recv 1135, close 0, drop 499
> >               10.04 real        41.18 user         8.23 sys
> > 
> > 
> > 2 regress/sys/kern/sosplice 
> > 
> > On snapshot, when I run it as doas make, it fails on:
> > 
> >        ==== run-udp-splice ====
> >        pkill -x tcpbench
> >        *** Error 1 in target 'run-udp-splice' (ignored)
> >        ./splice -u -4 12345 127.0.0.1 12346 &  rpid=$! ;  tcpbench -u -4 -t 7 -s -p 12346 &  sleep 1 ;  tcpbench -u -4 -t 5 127.0.0.1 ;  kill $rpid ;  pkill -x tcpbench
> >        splice: setsockopt rcvbuf: No buffer space available
> >        tcpbench: write: Connection refused
> >        --- 127.0.0.1 tcpbench statistics ---
> >        94208 bytes sent over 0.000 seconds
> >        bandwidth min/avg/max/std-dev = inf/0.000/0.000/0.000 Mbps
> >        *** Error 1 in perf (Makefile:30 'run-udp-splice')
> >        FAILED
> >        *** Error 1 in perf (<bsd.regress.mk>:105 'regress': if make -C /usr/src/regress/sys/kern/sosplice/perf run-udp-splice; then  echo -n "SUCCE...)
> >        *** Error 2 in /usr/src/regress/sys/kern/sosplice (<bsd.subdir.mk>:48 'all': @for entry in  error tcp udp loop scapy perf; do  set -e; if te...)
> > 
> > and with your diff it runs till the end.
> > 
> > As far as I understood the code, it is reads well, but I not think that my
> > OK here means anything. Anyway, you have it.
> > 
> > -- 
> > wbr, Kirill
> 

-- 
wbr, Kirill