From: Theo Buehler <tb@theobuehler.org>
Subject: ftp: send 'Accept: */*' header
To: tech@openbsd.org
Cc: bket@openbsd.org
Date: Wed, 17 Apr 2024 10:24:44 +0100

Ports bulk builders ran across a server config that would refuse
downloads without accept header, presumably before there is no default
fallback for no MIME type in their apache config's content negotiation:

$ ftp  https://download.filezilla-project.org/client/FileZilla_3.67.0_src.tar.xz
Trying 2a01:4f8:242:52d0::2...
Requesting https://download.filezilla-project.org/client/FileZilla_3.67.0_src.tar.xz
ftp: Error retrieving https://download.filezilla-project.org/client/FileZilla_3.67.0_src.tar.xz: 403 Forbidden

The patch below adds adds the (optional per spec) accept header to
signal we accept all MIME types, with which the download succeeds:

$ ftp  https://download.filezilla-project.org/client/FileZilla_3.67.0_src.tar.xz
Trying 2a01:4f8:242:52d0::2...
Requesting https://download.filezilla-project.org/client/FileZilla_3.67.0_src.tar.xz
100% |**********************************************************************|  3963 KB    00:01
4058264 bytes received in 1.75 seconds (2.20 MB/s)

I'm not sure how much testing for this is required. I also only tested
the non-proxy path.

Index: fetch.c
===================================================================
RCS file: /cvs/src/usr.bin/ftp/fetch.c,v
diff -u -p -r1.216 fetch.c
--- fetch.c	28 Jun 2023 17:35:06 -0000	1.216
+++ fetch.c	17 Apr 2024 09:17:27 -0000
@@ -705,7 +705,7 @@ noslash:
 		 */
 		ftp_printf(fin, "GET %s HTTP/1.1\r\n"
 		    "Connection: close\r\n"
-		    "Host: %s\r\n%s%s\r\n",
+		    "Host: %s\r\n%s%s\r\nAccept: */*\r\n",
 		    epath, proxyhost, buf ? buf : "", httpuseragent);
 		if (credentials)
 			ftp_printf(fin, "Authorization: Basic %s\r\n",
@@ -773,7 +773,7 @@ noslash:
 			ftp_printf(fin, "\r\nIf-Modified-Since: %s", tmbuf);
 #endif /* SMALL */
 
-		ftp_printf(fin, "\r\n%s%s\r\n",
+		ftp_printf(fin, "\r\n%s%s\r\nAccept: */*\r\n",
 		    buf ? buf : "", httpuseragent);
 		if (credentials)
 			ftp_printf(fin, "Authorization: Basic %s\r\n",