I put this on the most recent run of the arm64.p package builders,
nothing exploded.  I wasn't able to do a comparison of the time taken to
build all of the packages as a big one was added since the last run.

-peter


On 2024 Nov 13 (Wed) at 23:41:10 +1000 (+1000), David Gwynne wrote:
:it's become obvious that heavily contended rwlocks put a lot of
:pressure on the scheduler, and we have enough contended locks that
:there's a benefit to changing rwlocks to try and mitigate against
:that pressure.
:
:when a thread is waiting on an rwlock, it sets a bit in the rwlock
:to indicate that when the current owner of the rwlock leaves the
:critical section, it should wake up the waiting thread to try and
:take the lock. if there's no waiting thread, the owner can skip the
:wakeup.
:
:the problem is that rwlocks can't tell the difference between one
:waiting thread and more than one waiting thread. so when the "there's
:a thread waiting" bit is cleared, all the waiting threads are woken
:up. one of these woken threads will take ownership of the lock, but
:also importantly, the other threads will end up setting the "im
:waiting" bit again, which is necessary for them to be woken up by
:the 2nd thread that won the race to become the owner of the lock.
:
:this is compounded by pending writers and readers waiting on the
:same wait channel. an rwlock may have one pending writer trying to
:take the lock, but many readers waiting for it too. it would make
:sense to wake up only the writer so it can take the lock next, but
:we end up waking the readers at the same time.
:
:the result of this is that contended rwlocks wake up a lot of
:threads, which puts a lot of pressure on the scheduler. this is
:noticeable as a lot of contention on the scheduler lock, which
:increases the system time used by the system. this is a pretty
:classic thundering herd problem.
:
:the diff below mitigates against these wakeups by adding counters
:to rwlocks for the number threads waiting to take write and read
:locks instead of relying on bits. when a thread needs to wait for
:a rwlock it increments the relevant counter before sleeping. after
:it is woken up and takes the lock it decrements that counter. this
:means rwlocks know how many threads are waiting at all times without
:having to wake everything up to rebuild state every time a thread
:releases the lock.
:
:pending writers and readers also wait on separate wchans. this
:allows us to prioritise writers and to wake them up one at a time.
:once there's no pending writers all pending readers can be woken
:up in one go so
:they can share the lock as soon as possible.
:
:if you are suffering a contended rwlock, this should reduce the
:amount of time spent spinning on the sched lock, which in turn may
:also reduce the wall clock time doing that work.
:
:the only downside to this diff in my opinion is that it grows struct
:rwlock by 8 bytes.
:
:i've been hitting this diff hard, but would appreciate more tests.
:rwlocks are a very fundamental part of the kernel so they need to
:work. benchmarks and witness tests are also welcome.
:

-- 
Due to circumstances beyond your control, you are master of your fate
and captain of your soul.