Hi Hrvoje,

On 09.01.25 10:03, Hrvoje Popovski wrote:

>> this diff finally enables multiqueue for vio(4). It goes on top of the
>> "virtio: Support unused virtqueues" diff from my previous mail.
>>
>> The distribution of of packets to the enabled queues is not optimal. To
>> improve this, one would need the optional RSS (receive-side scaling)
>> feature which is difficult to configure with libvirt/qemu and therefore
>> usually not available on hypervisors. Things may improve with future
>> libvirt versions. RSS support is not included in this diff. But even
>> without RSS, we have seen some nice performance gains.


> 
> I'm hitting this diff with forwarding setup over ipsec for two days and
> doing ifconfig up/down and hosts seems stable. Forwarding performance is
> the same as without this diff.
> 
> I'm sending traffic from host connected to obsd1 vio2 then that traffic
> goes over ipsec link between obsd1 vio1 - obsd2 vio1 and traffic exits
> from obsd2 vio3 to other host


Thanks for testing. Since the traffic distribution is done heuristically 
by the hypervisor, it is often not optimal. I think it is particularily 
bad for your case because the hypervisor will think that all ipsec 
traffic belongs to one flow and put it into the same queue.

I will try to improve it a bit, but in general things get better if you 
communicate with many peers.

Cheers,
Stefan


> 
> 
> obsd1# vmstat -i
> 
> interrupt                       total     rate
> irq144/com0                      2749        0
> irq52/ehci1                        20        0
> irq65/vio0:1                       15        0
> irq66/vio0:2                    27709        0
> irq67/vio0:3                    37917        0
> irq68/vio0:4                      267        0
> irq69/vio0:5                    17490        0
> irq71/vio1:1                       81        0
> irq72/vio1:2                   106539        1
> irq73/vio1:3                       81        0
> irq74/vio1:4                       21        0
> irq77/vio2:1                       45        0
> irq78/vio2:2               5444311772    68168
> irq79/vio2:3               5529889506    69239
> irq80/vio2:4               5550808699    69501
> irq81/vio2:5               5560299463    69620
> irq83/vio3:1                        1        0
> irq56/vioscsi0:3                57609        0
> irq0/clock                   44969768      563
> irq0/ipi                       933325       11
> Total                     22131463077   277107
> 
> 
> interrupt                       total     rate
> irq144/com0                      3453        0
> irq52/ehci1                        20        0
> irq54/vioblk0:1                 52021        0
> irq65/vio0:1                       15        0
> irq66/vio0:2                    26751        0
> irq67/vio0:3                    36743        0
> irq68/vio0:4                      161        0
> irq69/vio0:5                    17215        0
> irq71/vio1:1                      501        0
> irq72/vio1:2                151685670     1898
> irq73/vio1:3                159400783     1995
> irq74/vio1:4                217874776     2727
> irq75/vio1:5                221039736     2766
> irq77/vio2:1                        1        0
> irq83/vio3:1                        9        0
> irq84/vio3:2                   101817        1
> irq85/vio3:3                       22        0
> irq0/clock                   42318639      529
> irq0/ipi                    687262836     8602
> Total                      1479821169    18522
>