Hi, 

I've spent several hours trying to find a solution for this problem. My goal is to prepare a patch. I would appreciate any advice. 

I've expolored three paths: 

1. Simple change in rc scripts: At the boot time some default rules are loaded at first. There is no "set limit anchors" in there yet. But earlier /etc/rc.conf and /etc/rc.conf.local are loaded. Easiest solution for this problem may be to define some variable settable in that config files (ex. pf_startup_anchors_limit) and loaded with this default ruleset that will be replaced later by loading /etc/pf.conf. This only requires small changes in /etc/rc, /etc/rc.d/rc.subr and optional changes in /etc/rc.conf that may be overwritten in /etc/rc.conf.local. 

2. Change in the pfctl and the kernel: 
a) pfctl: currently limit values (including limit anchors) are set at the very end of loading rules, just before executing commit. In src/pfctl/pfctl.c there is a call to pfctl_load_options that will set limits too. This is called after rules, anchors, queues, and tables are loaded. So I think setting of the limits should be moved out of the pfctl_load_options and set earlier or at least setting limit of anchors should be moved there. imits are set by calling DIOCSETLIMIT ioctl. 
b) kernel: Currently, when loading rules in sys/net/pf_ruleset.c function pf_create_anchor is called where new anchor is get from the pool. with "anchor = rs_pool_get_anchor();". The rs_pool_get_anchor(); is defined as shortcut to pool_get with pool_get(&pf_anchor_pl, PR_WAITOK|PR_LIMITFAIL|PR_ZERO). While pf_anchor_pl is global structure that holds all anchors due to the flag PR_LIMITFAIL it is ment to fail if there are more anchors than pool limit. I suppose that proper way should be use temprary pool that will be allocated in DIOCXBEGIN, this new pool should have hard limit set at DIOCSETLIMIT, any anchors created by DIOCADDRULE and DIOCCHANGERULE sould be created within this new pool, and during DIOCXCOMMIT old pool should be replaced with new one and old one should be freed. I think this is an invasive change too. 

3. Change in pfctl: I have figured out is that pfctl loads all the configuration, including limits in one transaction. I was wondering if it is possible to Begin transaction, set limits, commit transaction, and then open another one doing the rest of the staff. I've found out that in sbin/pfctl/pfctl.c there is a function "pfctl_rules" that is responsible for this. As I understood transaction is opened, then configuration file is parsed but at parsing time some tables and I think some other stuff like queues is created, and some structures that are later used are also populated. So transaction has to be already opened at parsing time. If we want to set limits in separate transaction, we have to get rid of any changes requiring transaction to be open at file parsing time. But as I see it now it is a massive change in logic pfctl operates and having two transactions looks odd to me and doesn't feel to be the right way. 


I do have any working solution (bot not yet a pach) for 1 but im not sure if it should be treated as pemament one or as workaround. I'm not sure if I will be capable to prepare 2 or 3 but if someone that have expirence in the topic will say one of those is a goal I will try to acheave it. 





From: "Rafał Ramocki" <rafal.ramocki@eo.pl> 
To: "tech" <tech@openbsd.org> 
Sent: Thursday, April 17, 2025 5:48:47 PM 
Subject: Error when at startup more than 512 anchors are loaded from pf.conf 

Hello, 

I'm facing problem with loading pf.conf at system startup when more than 512 anchors are defined despite set limit anchors 1024 is set in pf.conf. When loading that rulset defined in pf.conf and "load anchor" direcvies pf prints following error: 

pfctl: DIOCADDRULE: Invalid argument. 

I think that problem was introduced in 7.2 with this change: 

[ https://github.com/openbsd/src/commit/fa90ac5c787b88bcd714c01650b5cded334d9725 | https://github.com/openbsd/src/commit/fa90ac5c787b88bcd714c01650b5cded334d9725 ] 

At startup there is limit of 512 anchors. When in one transaction there are more than default limit of 512 anchors loaded and in the same transaction the limit is increased we are getting the error as above. 

I think that my use case is valid. I use one anchor per one vpn user to load separate rules for each one of them: 

This configuration is in pf.conf: 

anchor "vpn" in on tun0 
load anchor vpn from "/etc/anchors/vpn/users.conf" 

And users.conf looks like this: 

anchor "user1" in from 172.31.210.101 { 
pass in quick log proto tcp to X.X.X.X port 8080 
pass in quick log proto tcp to Y.Y.Y.Y port { 80 443 } 
(...) 
} 
anchor "user2" in from 172.31.210.105 { 
pass in quick log proto tcp to Z.Z.Z.Z port 3306 
(...) 
} 
(...) 

I wonder how can I workaround this problem. 
-- 
Rafal Ramocki