Download raw body.
rpki-client: indicate instances of suboptimal HTTP content encoding
In verbose mode, draw some attention to inefficient transfers by logging
a message when objects larger than a megabyte were transferred without
using gzip or deflate compression.
Example:
# rm -rf /var/cache/rpki-client/.rrdp
# rpki-client -v 2>&1 | grep --line-buffered -e downloaded -e Proces -e New | ts
Jun 06 20:45:45 rpki-client: https://rrdp.ripe.net/172322cf-c642-4e6f-806c-bd2375d8001a/130462/snapshot-b... (2.18.121.80): downloaded 243840080 bytes without content encoding compression
Jun 06 20:46:09 rpki-client: https://rrdp.lacnic.net/rrdp/66b09eea-03c7-435f-bb29-5bee810bd657/209590/d51... (172.233.4.134): downloaded 61735570 bytes without content encoding compression
Jun 06 20:47:46 rpki-client: https://rrdp.paas.rpki.ripe.net/1c33ba5d-4e16-448d-9a22-b12599ef1cba/229024/... (104.123.50.160): downloaded 18427607 bytes without content encoding compression
Jun 06 20:48:38 rpki-client: https://rpki-rrdp.us-east-2.amazonaws.com/rrdp/20aa329b-fc52-4c61-bf53-09725... (108.156.60.53): downloaded 12421100 bytes without content encoding compression
...
OK?
Index: http.c
===================================================================
RCS file: /cvs/src/usr.sbin/rpki-client/http.c,v
diff -u -p -r1.93 http.c
--- http.c 11 Mar 2025 14:53:03 -0000 1.93
+++ http.c 6 Jun 2025 20:44:49 -0000
@@ -797,7 +797,6 @@ http_inflate_advance(struct http_connect
if (zctx->zdone) {
/* all compressed data processed */
- conn->gzipped = 0;
http_inflate_done(conn);
if (conn->iosz == 0) {
@@ -911,7 +910,9 @@ http_done(struct http_connection *conn,
if (conn->gzipped) {
conn->gzipped = 0;
http_inflate_done(conn);
- }
+ } else if (conn->totalsz > (1024 * 1024))
+ logx("%s: downloaded %zu bytes without content encoding "
+ "compression", conn_info(conn), conn->totalsz);
conn->state = STATE_IDLE;
conn->idle_time = getmonotime() + HTTP_IDLE_TIMEOUT;
rpki-client: indicate instances of suboptimal HTTP content encoding