On 14.1.2025. 13:36, Stefan Fritsch wrote:
> Hi Hrvoje,
> 
> On 09.01.25 10:03, Hrvoje Popovski wrote:
> 
>>> this diff finally enables multiqueue for vio(4). It goes on top of the
>>> "virtio: Support unused virtqueues" diff from my previous mail.
>>>
>>> The distribution of of packets to the enabled queues is not optimal. To
>>> improve this, one would need the optional RSS (receive-side scaling)
>>> feature which is difficult to configure with libvirt/qemu and therefore
>>> usually not available on hypervisors. Things may improve with future
>>> libvirt versions. RSS support is not included in this diff. But even
>>> without RSS, we have seen some nice performance gains.
> 
> 
>>
>> I'm hitting this diff with forwarding setup over ipsec for two days and
>> doing ifconfig up/down and hosts seems stable. Forwarding performance is
>> the same as without this diff.
>>
>> I'm sending traffic from host connected to obsd1 vio2 then that traffic
>> goes over ipsec link between obsd1 vio1 - obsd2 vio1 and traffic exits
>> from obsd2 vio3 to other host
> 
> 
> Thanks for testing. Since the traffic distribution is done heuristically
> by the hypervisor, it is often not optimal. I think it is particularily
> bad for your case because the hypervisor will think that all ipsec
> traffic belongs to one flow and put it into the same queue.
> 
> I will try to improve it a bit, but in general things get better if you
> communicate with many peers.

Hi,

it seems that even with plain forwarding all traffic on egress
interfaces are going to one queue. On ingress interface interrupts are
spread nicely. Maybe because of that forwarding performance is same as
without multiqueue vio.


I'm sending random 10.111/24 from host connected to vio2 to random
10.222/24 to host connected to vio3.


netstat
Destination        Gateway            Flags   Refs      Use   Mtu  Prio
Iface

10.111/16          192.168.111.11     UGS        0        0     -     8 vio2
10.222/16          192.168.222.11     UGS        1 1957400238     -
8 vio3


obsd1# vmstat -iz
interrupt                       total     rate
irq144/com0                       484        0
irq48/acpi0                         0        0
irq49/uhci0                         0        0
irq50/uhci1                         0        0
irq51/uhci2                         0        0
irq52/ehci0                         0        0
irq49/ppb0                          0        0
irq49/ppb1                          0        0
irq49/ppb2                          0        0
irq49/ppb3                          0        0
irq49/uhci3                         0        0
irq50/uhci4                         0        0
irq51/uhci5                         0        0
irq52/ehci1                        20        0
irq64/vio0:0                        0        0
irq65/vio0:1                        9        0
irq66/vio0:2                      286        0
irq67/vio0:3                     1467        0
irq68/vio0:4                      543        0
irq69/vio0:5                      841        0
irq70/vio1:0                        0        0
irq71/vio1:1                        9        0
irq72/vio1:2                        1        0
irq73/vio1:3                        1        0
irq74/vio1:4                        0        0
irq75/vio1:5                        1        0
irq76/vio2:0                        0        0
irq77/vio2:1                        9        0
irq78/vio2:2                 91031269    37170
irq79/vio2:3                154334709    63019
irq80/vio2:4                152971269    62462
irq81/vio2:5                155102163    63332
irq82/vio3:0                        0        0
irq83/vio3:1                        9        0
irq84/vio3:2                  1928679      787
irq85/vio3:3                        4        0
irq86/vio3:4                        1        0
irq87/vio3:5                        0        0
irq53/vioscsi0:0                    0        0
irq54/vioscsi0:1                    0        0
irq55/vioscsi0:2                    0        0
irq56/vioscsi0:3                45856       18
irq57/ahci0                         0        0
irq49/ichiic0                       0        0
irq145/pckbc0                       0        0
irq146/pckbc0                       0        0
irq0/clock                    1358939      554
irq0/ipi                     10597761     4327
Total                       567374330   231675


Thank you for mq vio ....