Download raw body.
Adding support for mirrors on the Tor network.
Dear all,
In this email, I have attached a diff that adds support for updating the
system, applying patches, installing firmware, and managing ports
through a mirror on the Tor network. This implementation is designed to
be as non-invasive to the base system as possible, and requires the
prior installation of two ports (net/tor and net/torsocks) for this
feature to work. In addition, the documentation has been updated to
reflect these changes.
Below are the details of the changes:
1. Changes in the Documentation:
- A section has been added to the manual page (installurl.5) stating
that using Tor mirrors requires the net/tor and net/torsocks ports to be
installed and the Tor daemon to be running.
2. Changes in the sysupgrade, syspatch and fw_update scripts:
- Direct invocation of the "ftp -N" command has been replaced with the
"$FTP_CMD" variable.
- A new function "set_proxy()" has been added to parse the mirror URL.
If the URL contains ".onion" (indicating a Tor mirror), the function
configures "$FTP_CMD" to use torsocks along with ftp (e.g.
"/usr/local/bin/torsocks /usr/bin/ftp -N"). Otherwise, the normal ftp
command will be used.
3. Changes in the perl module for pkg_add (PackageRepository.pm):
- A check has been added to the "parse_url" function to determine if the
host in the URL ends with ".onion". If so, a flag ("use_torsocks") is set.
- The "ftp_cmd" function uses this flag to prepend torsocks to the ftp
command when appropriate.
Please note that these changes do not allow you to install the system
via a mirror on the Tor network; the system and the necessary ports must
already be installed for this feature to work.
I have tested it on amd64 with Solène Rapenne's mirror [1] and it works
fine.
Best regards,
David.
References:
[1]:
https://dataswamp.org/~solene/2024-05-25-openbsd-privacy-friendly-mirror.html
Index: src/share/man/man5/installurl.5
===================================================================
RCS file: /cvs/src/share/man/man5/installurl.5,v
diff -u -p -u -r1.8 installurl.5
--- src/share/man/man5/installurl.5 27 Apr 2019 22:30:05 -0000 1.8
+++ src/share/man/man5/installurl.5 18 Mar 2025 12:33:10 -0000
@@ -34,6 +34,17 @@ It is created by the installer script du
Empty lines and lines beginning with
.Sq #
in the file are ignored.
+.Pp
+To use Onion (Tor) mirrors, the following tools from the ports tree are required:
+.Bl -bullet
+.It
+net/tor
+.It
+net/torsocks
+.El
+.Pp
+Additionally, the Tor daemon must be running.
+.El
.Sh FILES
.Bl -tag -width /etc/installurl -compact
.It Pa /etc/installurl
Index: src/usr.sbin/syspatch/syspatch.sh
===================================================================
RCS file: /cvs/src/usr.sbin/syspatch/syspatch.sh,v
diff -u -p -u -r1.168 syspatch.sh
--- src/usr.sbin/syspatch/syspatch.sh 13 Dec 2023 17:50:23 -0000 1.168
+++ src/usr.sbin/syspatch/syspatch.sh 18 Mar 2025 12:33:43 -0000
@@ -134,7 +134,7 @@ fetch_and_verify()
[[ -n ${_tgz} ]]
[[ -t 0 ]] || echo "${_title} ${_tgz}"
- unpriv -f "${_TMP}/${_tgz}" ftp -N syspatch -VD "${_title}" -o \
+ unpriv -f "${_TMP}/${_tgz}" $FTP_CMD syspatch -VD "${_title}" -o \
"${_TMP}/${_tgz}" "${_MIRROR}/${_tgz}"
(cd ${_TMP} && sha256 -qC ${_TMP}/SHA256 ${_tgz})
@@ -168,7 +168,7 @@ ls_missing()
local _c _f _cmd _l="$(ls_installed)" _p _sha=${_TMP}/SHA256
# don't output anything on stdout to prevent corrupting the patch list
- unpriv -f "${_sha}.sig" ftp -N syspatch -MVo "${_sha}.sig" \
+ unpriv -f "${_sha}.sig" $FTP_CMD syspatch -MVo "${_sha}.sig" \
"${_MIRROR}/SHA256.sig" >/dev/null
unpriv -f "${_sha}" signify -Veq -x ${_sha}.sig -m ${_sha} -p \
/etc/signify/openbsd-${_OSrev}-syspatch.pub >/dev/null
@@ -181,7 +181,7 @@ ls_missing()
while read _c; do _c=${_c##syspatch${_OSrev}-} &&
[[ -n ${_l} ]] && echo ${_c} | grep -qw -- "${_l}" || echo ${_c}
done | while read _p; do
- _cmd="ftp -N syspatch -MVo - \
+ _cmd="$FTP_CMD syspatch -MVo - \
${_MIRROR}/syspatch${_OSrev}-${_p}.tgz"
unpriv "${_cmd}" | tar tzf - | while read _f; do
# no earlier version of _all_ files contained in the tgz
@@ -284,10 +284,22 @@ set -A _KERNV -- $(sysctl -n kern.versio
_OSrev=${_KERNV[0]%.*}${_KERNV[0]#*.}
[[ -n ${_OSrev} ]]
+set_proxy() {
+ case "$1" in
+ *.onion/*)
+ FTP_CMD="/usr/local/bin/torsocks /usr/bin/ftp -N"
+ ;;
+ *)
+ FTP_CMD="/usr/bin/ftp -N"
+ ;;
+ esac
+}
+
_MIRROR=$(while read _line; do _line=${_line%%#*}; [[ -n ${_line} ]] &&
print -r -- "${_line}"; done </etc/installurl | tail -1) 2>/dev/null
[[ ${_MIRROR} == @(file|ftp|http|https)://* ]] ||
_MIRROR=https://cdn.openbsd.org/pub/OpenBSD
+set_proxy "$_MIRROR"
_MIRROR="${_MIRROR}/syspatch/${_KERNV[0]}/$(machine)"
_PATCH_APPLIED=false
Index: src/usr.sbin/sysupgrade/sysupgrade.sh
===================================================================
RCS file: /cvs/src/usr.sbin/sysupgrade/sysupgrade.sh,v
diff -u -p -u -r1.58 sysupgrade.sh
--- src/usr.sbin/sysupgrade/sysupgrade.sh 3 Feb 2025 18:55:55 -0000 1.58
+++ src/usr.sbin/sysupgrade/sysupgrade.sh 18 Mar 2025 12:34:50 -0000
@@ -72,6 +72,17 @@ rmel() {
echo -n "$_c"
}
+set_proxy() {
+ case "$1" in
+ *.onion/*)
+ FTP_CMD="/usr/local/bin/torsocks /usr/bin/ftp -N"
+ ;;
+ *)
+ FTP_CMD="/usr/bin/ftp -N"
+ ;;
+ esac
+}
+
SNAP=false
FILE=false
FORCE=false
@@ -111,6 +122,7 @@ case $# in
;;
*) usage
esac
+set_proxy "$MIRROR"
[[ $MIRROR == @(file|ftp|http|https)://* ]] ||
FILE=true
$FORCE_VERSION && $SNAP &&
@@ -136,10 +148,10 @@ install -d -o 0 -g 0 -m 0755 ${SETSDIR}
cd ${SETSDIR}
echo "Fetching from ${URL}"
-if ! unpriv -f SHA256.sig ftp -N sysupgrade -Vmo SHA256.sig ${URL}SHA256.sig; then
+if ! unpriv -f SHA256.sig $FTP_CMD sysupgrade -Vmo SHA256.sig ${URL}SHA256.sig; then
if [[ -n ${ALT_URL} ]]; then
echo "Fetching from ${ALT_URL}"
- unpriv -f SHA256.sig ftp -N sysupgrade -Vmo SHA256.sig ${ALT_URL}SHA256.sig
+ unpriv -f SHA256.sig $FTP_CMD sysupgrade -Vmo SHA256.sig ${ALT_URL}SHA256.sig
URL=${ALT_URL}
NEXT_VERSION=${VERSION}
else
@@ -162,7 +174,7 @@ if ! [[ -r /etc/signify/$KEY ]]; then
BUNDLE=sigbundle-${HAVEKEY}.tgz
FWKEY=$(echo $KEY | sed -e 's/base/fw/')
echo "Adding missing keys from bundle $BUNDLE"
- unpriv -f ${BUNDLE} ftp -N sysupgrade -Vmo $BUNDLE https://ftp.openbsd.org/pub/OpenBSD/signify/$BUNDLE
+ unpriv -f ${BUNDLE} $FTP_CMD sysupgrade -Vmo $BUNDLE https://ftp.openbsd.org/pub/OpenBSD/signify/$BUNDLE
signify -Vzq -m - -x $BUNDLE | (cd /etc/signify && tar xfz - $KEY $FWKEY)
rm $BUNDLE
fi
@@ -175,7 +187,7 @@ if cmp -s /var/db/installed.SHA256 SHA25
exit 0
fi
-unpriv -f BUILDINFO ftp -N sysupgrade -Vmo BUILDINFO ${URL}BUILDINFO
+unpriv -f BUILDINFO $FTP_CMD sysupgrade -Vmo BUILDINFO ${URL}BUILDINFO
unpriv cksum -qC SHA256 BUILDINFO
if [[ -e /var/db/installed.BUILDINFO ]]; then
@@ -205,7 +217,7 @@ done
[[ -n ${OLD_FILES} ]] && rm ${OLD_FILES}
for f in ${DL}; do
- unpriv -f $f ftp -N sysupgrade -Vmo ${f} ${URL}${f}
+ unpriv -f $f $FTP_CMD sysupgrade -Vmo ${f} ${URL}${f}
done
if [[ -n ${DL} ]]; then
Index: src/usr.sbin/fw_update/fw_update.sh
===================================================================
RCS file: /cvs/src/usr.sbin/fw_update/fw_update.sh,v
diff -u -p -u -r1.62 fw_update.sh
--- src/usr.sbin/fw_update/fw_update.sh 24 Nov 2024 21:27:04 -0000 1.62
+++ src/usr.sbin/fw_update/fw_update.sh 18 Mar 2025 12:35:28 -0000
@@ -108,6 +108,17 @@ spin() {
}>/dev/tty
}
+set_proxy() {
+ case "$FWURL" in
+ *.onion/*)
+ FTP_CMD="/usr/local/bin/torsocks /usr/bin/ftp -N"
+ ;;
+ *)
+ FTP_CMD="/usr/bin/ftp -N"
+ ;;
+ esac
+}
+
fetch() {
local _src="${FWURL}/${1##*/}" _dst=$1 _user=_file _exit _error=''
local _ftp_errors="$FD_DIR/ftp_errors"
@@ -123,13 +134,13 @@ fetch() {
esac
if ! "$DROP_PRIVS"; then
- /usr/bin/ftp -N error -D 'Get/Verify' $_flags -o- "$_src" > "$_dst"
+ $FTP_CMD error -D 'Get/Verify' $_flags -o- "$_src" > "$_dst"
elif [ -x /usr/bin/su ]; then
exec /usr/bin/su -s /bin/ksh "$_user" -c \
- "/usr/bin/ftp -N error -D 'Get/Verify' $_flags -o- '$_src'" > "$_dst"
+ "$FTP_CMD -N error -D 'Get/Verify' $_flags -o- '$_src'" > "$_dst"
else
exec /usr/bin/doas -u "$_user" \
- /usr/bin/ftp -N error -D 'Get/Verify' $_flags -o- "$_src" > "$_dst"
+ $FTP_CMD -N error -D 'Get/Verify' $_flags -o- "$_src" > "$_dst"
fi
) & FTPPID=$!
set +o monitor
@@ -384,7 +395,7 @@ add_firmware () {
2|3) _flags=-Vm ;;
esac
- ftp -N "${0##/}" -D "$_m" "$_flags" -o- "file:${1}" |
+ $FTP_CMD "${0##/}" -D "$_m" "$_flags" -o- "file:${1}" |
tar -s ",^\+,${FWPKGTMP}/+," \
-s ",^firmware,${DESTDIR}/etc/firmware," \
-C / -zxphf - "+*" "firmware/*"
Index: src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm
===================================================================
RCS file: /cvs/src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm,v
diff -u -p -u -r1.177 PackageRepository.pm
--- src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm 25 Nov 2023 10:29:23 -0000 1.177
+++ src/usr.sbin/pkg_add/OpenBSD/PackageRepository.pm 18 Mar 2025 12:37:24 -0000
@@ -590,6 +590,9 @@ sub parse_url($class, $r, $state)
$o->can_be_empty;
$$r = $class->urlscheme."://$o->{host}$o->{release}:$$r";
}
+ if ($host =~ /\.onion$/i) {
+ $o->{use_torsocks} = 1;
+ }
$o->setup_session;
return $o;
} else {
@@ -757,7 +760,12 @@ sub fetch_id($self)
sub ftp_cmd($self)
{
- return OpenBSD::Paths->ftp;
+ my $ftp = OpenBSD::Paths->ftp;
+ if ($self->{use_torsocks}) {
+ return "/usr/local/bin/torsocks $ftp";
+ } else {
+ return $ftp;
+ }
}
sub drop_privileges_and_setup_env($self)
Adding support for mirrors on the Tor network.