On 5.2.2025. 19:25, Alexander Bluhm wrote:
> Hi,
> 
> To run TCP input in parallel, each packet needs the socket lock.
> As lock and unlock per packet kills throughput for single stream
> TCP, diff below keeps the socket locked when consecutive packets
> belong to the same socket.
> 
> I keep a list of TCP packets in struct softnet.  So each softnet
> thread has its own list to accumulate packets.  After processing
> all packets from an interface input queue, TCP input is called for
> each TCP packet.  The socket lock is only unlocked and locked if
> the TCP socket changes between the packets.  This solves the
> performance issue.
> 
> To remember the softnet thread and packet offset, I use the cookie
> field in the mbuf header.  Hopefully nothing else uses cookies in
> this path.
> 
> Please test with various interfaces and pseudo devices.
> Are there setups where my idea does not work?
> How much is TCP stack getting faster?
> Are there performance regressions?
> 
> Note that at most 4 CPUs are used for network.  You might want to
> increase NET_TASKQ.  Also you need network interfaces that support
> multiqueue.  ix, ixl, igc, bnxt, vio, vmx, ...
> 
> bluhm

Hi,

I've tested ix, bnxt and mcx on same machine and it seems that mcx have
performance problems with 1 tcp stream with or without your diff. With
this diff mcx get nice performance boost with more than 1 tcp stream
which is not the case without this diff...


NET_TASKQ=8

1 tcp stream
ix - 9.38 Gbits/sec
mcx - 4.21 Gbits/sec
bnxt - 6.95 Gbits/sec

8 tcp streams
ix - 9.41 Gbits/sec
mcx - 5.15 Gbits/sec
bnxt - 6.68 Gbits/sec




with diff
1 tcp stream
ix - 9.38 Gbits/sec
mcx - 4.20 Gbits/sec
bnxt - 7.88 Gbits/sec

8 tcp streams
ix - 9.36 Gbits/sec
mcx - 9.21 Gbits/sec
bnxt - 9.34 Gbits/sec


spreading interrupts over queues is done by mixing source and
destination ip, not src or dst tcp/udp ports? is that right? if so, to
test this properly I need to fire up some virtual hosts ?