From: Stuart Henderson Subject: Re: sysupgrade/ftp: use a 'needle' to poke through caching layers To: Job Snijders , tech@openbsd.org Date: Fri, 3 May 2024 11:33:27 +0100 On 2024/05/03 04:17, Theo de Raadt wrote: > Stuart Henderson wrote: > > > SHA256.sig on the origin cannot be relied upon to be in sync with the > > tgz files. Part of this was due to non-atomic syncs which AIUI have > > now been improved, but another part AFAIK is an artefact of the way in > > which builds are done and that's harder to change. > > Nope. > > The first few steps of pushing build pieces has always been correct. > I only push directories that are complete and correct. > > The only weird thing is that base and x components are handled a bit > seperately (so you can get older X, with newer base, for a small window > of time until the new X build completes). architectures with install*.* > files that is solved, because the signing specifically waits for those > files, and then of course they are also correct in the hash. From memory, it usually is the X sets. > > I fetch base snaps and run signify to check the hashes. Despite only > > fetching them once a day (so I guess the chances of running into > > any individual breakage are probably fairly low) I've had 2 failures > > in the last month. (See examples below). > > I don't know where you fetched from. Even if you fetched from > ftp.openbsd.org, it could get de-sync'd, until about 18 hours ago. From > cdn.openbsd.org it could get VERY desync'd. Currently fetching from ftp.fr but I've moved that around a bit. When I've noticed this quickly enough to check in the past, I've checked other mirrors and all that I've checked have been the same. > From 2nd and 3rd tier > mirrors it is probably even worse. We asked 2nd/3rd tier mirrors to use --delete-delay --delay-updates since 2016 or so though I think perhaps some 2nd levels had to stop using it when the fanout was at uofa and had slow/interrupted transfers (or for disk space reasons). > I think we have fixed very well on ftp.openbsd.org, and on > cdn.openbsd.org to the degree that job describes. Other mirrors can get > get fixed incrementally following this. > > So please check again. Will keep an eye on it. > Regardless you can *still lose*, unless we put all of OpenBSD into 1 file. Yes, there is the situation where files change while you're in the middle of fetching them for an install. At least for base, signify will pick this up.