Index | Thread | Search

From:
Claudio Jeker <cjeker@diehard.n-r-g.com>
Subject:
Re: httpd: add cache controls for static files
To:
job@bsd.nl, tech@openbsd.org
Date:
Wed, 13 May 2026 20:42:25 +0200

Download raw body.

Thread
On Wed, May 13, 2026 at 07:00:01PM +0200, Kirill A. Korinsky wrote:
> On Wed, 13 May 2026 17:36:24 +0200,
> Job Snijders <job@bsd.nl> wrote:
> > 
> > On Wed, May 13, 2026 at 05:33:09PM +0200, Kirill A. Korinsky wrote:
> > > You missed that it is not just Cache-Control: no-cache, but it actually
> > > needs to Vary: Accept-Encoding that makes [no] cache-control no-cache not
> > > that clear I think.
> > > 
> > > Next, send-header or add-header is possible approach, but my diff adds it
> > > out of the box because mtime / Last-Modified for static files makes it safe.
> > 
> > Good points, and I like that your approach does a bunch of sane things
> > out of the box.
> > 
> > Let's go with 'static-cache-control' like you proposed.
> > 
> 
> Here the diff, the only things which I don't like is lookup() where
> long "static-cache-control" string makes it a bit odd visually.
> 
> But I can't justify reformat other strings, so, I think it's acceptable.
> 
> Ok?
> 
> Index: config.c
> ===================================================================
> RCS file: /home/cvs/src/usr.sbin/httpd/config.c,v
> diff -u -p -r1.71 config.c
> --- config.c	11 May 2026 22:33:10 -0000	1.71
> +++ config.c	13 May 2026 16:56:00 -0000
> @@ -633,6 +633,11 @@ config_getserver_config(struct httpd *en
>  			    sizeof(srv_conf->path));
>  		}
>  
> +		f = SRVFLAG_STATIC_CACHE_CONTROL |
> +		    SRVFLAG_NO_STATIC_CACHE_CONTROL;
> +		if ((srv_conf->flags & f) == 0)
> +			srv_conf->flags |= parent->flags & f;
> +
>  		f = SRVFLAG_SERVER_HSTS;
>  		srv_conf->flags |= parent->flags & f;
>  		srv_conf->hsts_max_age = parent->hsts_max_age;
> Index: httpd.conf.5
> ===================================================================
> RCS file: /home/cvs/src/usr.sbin/httpd/httpd.conf.5,v
> diff -u -p -r1.129 httpd.conf.5
> --- httpd.conf.5	18 Jan 2026 16:38:02 -0000	1.129
> +++ httpd.conf.5	13 May 2026 16:56:00 -0000
> @@ -254,6 +254,18 @@ If
>  is omitted, enable the banner for the current
>  .Ic server
>  if it was disabled globally.
> +.It Oo Ic no Oc Ic static-cache-control
> +Send
> +.Dq Cache-Control: no-cache
> +with static file responses.
> +When
> +.Ic gzip-static
> +is enabled, also send
> +.Dq Vary: Accept-Encoding
> +with static file responses.
> +This is enabled by default; use
> +.Ic no static-cache-control
> +to disable these headers.
>  .It Ic block drop
>  Drop the connection without sending an error page.
>  .It Ic block Op Ic return Ar code Op Ar uri
> Index: httpd.h
> ===================================================================
> RCS file: /home/cvs/src/usr.sbin/httpd/httpd.h,v
> diff -u -p -r1.171 httpd.h
> --- httpd.h	11 May 2026 22:33:10 -0000	1.171
> +++ httpd.h	13 May 2026 16:56:00 -0000
> @@ -393,6 +393,8 @@ SPLAY_HEAD(client_tree, client);
>  #define SRVFLAG_NO_GZIP_STATIC		0x0000000010000000ULL
>  #define SRVFLAG_LOCATION_FOUND		0x0000000040000000ULL
>  #define SRVFLAG_LOCATION_NOT_FOUND	0x0000000080000000ULL
> +#define SRVFLAG_STATIC_CACHE_CONTROL	0x0000000100000000ULL
> +#define SRVFLAG_NO_STATIC_CACHE_CONTROL	0x0000000200000000ULL
>  
>  #define SRVFLAG_BITS							\
>  	"\10\01INDEX\02NO_INDEX\03AUTO_INDEX\04NO_AUTO_INDEX"		\
> @@ -401,7 +403,8 @@ SPLAY_HEAD(client_tree, client);
>  	"\21AUTH\22NO_AUTH\23BLOCK\24NO_BLOCK\25LOCATION_MATCH"		\
>  	"\26SERVER_MATCH\27SERVER_HSTS\30DEFAULT_TYPE\31PATH_REWRITE"	\
>  	"\32NO_PATH_REWRITE\34NO_BANNER\33GZIP_STATIC"			\
> -	"\35NO_GZIP_STATIC\37LOCATION_FOUND\40LOCATION_NOT_FOUND"
> +	"\35NO_GZIP_STATIC\37LOCATION_FOUND\40LOCATION_NOT_FOUND"	\
> +	"\41STATIC_CACHE_CONTROL\42NO_STATIC_CACHE_CONTROL"
>  
>  #define TCPFLAG_NODELAY		0x01
>  #define TCPFLAG_NNODELAY	0x02
> Index: parse.y
> ===================================================================
> RCS file: /home/cvs/src/usr.sbin/httpd/parse.y,v
> diff -u -p -r1.133 parse.y
> --- parse.y	11 May 2026 22:33:10 -0000	1.133
> +++ parse.y	13 May 2026 16:56:00 -0000
> @@ -142,7 +142,7 @@ typedef struct {
>  %token	TIMEOUT TLS TYPE TYPES HSTS MAXAGE SUBDOMAINS DEFAULT PRELOAD REQUEST
>  %token	ERROR INCLUDE AUTHENTICATE WITH BLOCK DROP RETURN PASS REWRITE
>  %token	CA CLIENT CRL OPTIONAL PARAM FORWARDED FOUND NOT
> -%token	ERRDOCS GZIPSTATIC BANNER
> +%token	ERRDOCS GZIPSTATIC BANNER STATIC_CACHE_CONTROL
>  %token	<v.string>	STRING
>  %token  <v.number>	NUMBER
>  %type	<v.port>	port
> @@ -275,7 +275,8 @@ server		: SERVER optmatch STRING	{
>  			    SERVER_REQUESTTIMEOUT;
>  			s->srv_conf.maxrequests = SERVER_MAXREQUESTS;
>  			s->srv_conf.maxrequestbody = SERVER_MAXREQUESTBODY;
> -			s->srv_conf.flags = SRVFLAG_LOG;
> +			s->srv_conf.flags = SRVFLAG_LOG |
> +			    SRVFLAG_STATIC_CACHE_CONTROL;
>  			if ($2)
>  				s->srv_conf.flags |= SRVFLAG_SERVER_MATCH;
>  			s->srv_conf.logformat = LOG_FORMAT_COMMON;
> @@ -558,6 +559,7 @@ serveroptsl	: LISTEN ON STRING opttls po
>  		| root
>  		| directory
>  		| banner
> +		| static_cache_control
>  		| logformat
>  		| fastcgi
>  		| authenticate
> @@ -1252,6 +1254,16 @@ gzip_static	: NO GZIPSTATIC		{
>  		}
>  		;
>  
> +static_cache_control	: NO STATIC_CACHE_CONTROL	{
> +			srv_conf->flags &= ~SRVFLAG_STATIC_CACHE_CONTROL;
> +			srv_conf->flags |= SRVFLAG_NO_STATIC_CACHE_CONTROL;
> +		}
> +		| STATIC_CACHE_CONTROL	{
> +			srv_conf->flags &= ~SRVFLAG_NO_STATIC_CACHE_CONTROL;
> +			srv_conf->flags |= SRVFLAG_STATIC_CACHE_CONTROL;
> +		}
> +		;
> +
>  tcpip		: TCP '{' optnl tcpflags_l '}'
>  		| TCP tcpflags
>  		;
> @@ -1511,6 +1523,7 @@ lookup(char *s)
>  		{ "sack",		SACK },
>  		{ "server",		SERVER },
>  		{ "socket",		SOCKET },
> +		{ "static-cache-control",	STATIC_CACHE_CONTROL },
>  		{ "strip",		STRIP },
>  		{ "style",		STYLE },
>  		{ "subdomains",		SUBDOMAINS },
> Index: server_http.c
> ===================================================================
> RCS file: /home/cvs/src/usr.sbin/httpd/server_http.c,v
> diff -u -p -r1.161 server_http.c
> --- server_http.c	2 Mar 2026 19:24:58 -0000	1.161
> +++ server_http.c	13 May 2026 16:56:00 -0000
> @@ -1608,6 +1608,16 @@ server_response_http(struct client *clt,
>  	    kv_set(cl, "%lld", (long long)size) == -1))
>  		return (-1);
>  
> +	if (srv_conf->flags & SRVFLAG_STATIC_CACHE_CONTROL) {
> +		if (kv_add(&resp->http_headers, "Cache-Control",
> +		    "no-cache") == NULL)
> +			return (-1);
> +		if (srv_conf->flags & SRVFLAG_GZIP_STATIC &&
> +		    kv_add(&resp->http_headers, "Vary",
> +		    "Accept-Encoding") == NULL)
> +			return (-1);
> +	}
> +
>  	/* Set last modification time */
>  	if (server_http_time(mtime, tmbuf, sizeof(tmbuf)) <= 0 ||
>  	    kv_add(&resp->http_headers, "Last-Modified", tmbuf) == NULL)
> 
> 

From my vague memories running big websites it feels this button is
somewhat limiting. IIRC Cache-Control, Vary and other headers had to be
fumbled with to make caches work properly.
So my worry is that this is not enough but it paints us into a corner.

The diff itself looks OK. I would have split static-cache-control into
multiple tokens but that's just a bikeshed right now.

-- 
:wq Claudio