From: David Gwynne <david@gwynne.id.au>
Subject: libpcap + DLT_ENC: use the ip version nibble to match with
To: tech@openbsd.org
Date: Thu, 9 Oct 2025 15:19:58 +1000

DLT_ENC specifies a special encapsulation header (struct enchdr)
for prefixing the packet payload, but i think we've screwed up both the
setup of it in the kernel and use of it tcpdump, which makes it almost
useless.

packet payloads (including encapsulation headers) should have a
consistent byte order, but the kernel uses the host byte order when
setting up the fields in struct enchdr. one of these fields is the
address family of the packet payload.

because of this screwup, the DLT_ENC handling in libpcap is set up
weirdly so it effectively nops out the IP version matching and both ipv4
and ipv6 checks will end up matching each other.

tcpdump tries to parse the enchdr in the host byte order. it's address
family check ends up assuming everything is ipv4 if the comparisons it
does dont find anything.

i kind of want to fix the population of enchdr in the kernel so
everything is stored in network byte order. i can make tcpdump cope with
this by doing it's comparisons in both host and network byte order.

in the meantime, we can teach libpcap to ignore the enchdr when doing
address family checks by having it look at the first nibble in the
packet, like what's done for DLT_RAW.

it basically turns this:

# tcpdump -ni enc0 -d host 192.168.0.1 
(000) ld       #0x0
(001) ld       [24]
(002) jeq      #0xc0a80001      jt 5    jf 3
(003) ld       [28]
(004) jeq      #0xc0a80001      jt 5    jf 6
(005) ret      #116
(006) ret      #0

into this:

# tcpdump -ni enc0 -d host 192.168.0.1
(000) ldb      [12]
(001) rsh      #4
(002) jeq      #0x4             jt 3	jf 8
(003) ld       [24]
(004) jeq      #0xc0a80001      jt 7	jf 5
(005) ld       [28]
(006) jeq      #0xc0a80001      jt 7	jf 8
(007) ret      #116
(008) ret      #0

Index: gencode.c
===================================================================
RCS file: /cvs/src/lib/libpcap/gencode.c,v
diff -u -p -r1.68 gencode.c
--- gencode.c	6 Jun 2025 00:04:33 -0000	1.68
+++ gencode.c	9 Oct 2025 05:01:04 -0000
@@ -748,8 +748,8 @@ init_linktype(int type)
 		return;
 
 	case DLT_ENC:
-		off_linktype = -1;
-		off_nl = 12;
+		off_linktype = 12; /* ENC_HDRLEN */
+		off_nl = 12;
 		return;
 
 	case DLT_PFLOG:
@@ -883,7 +883,6 @@ gen_linktype(int proto)
 		break;
 
 	case DLT_LOOP:
-	case DLT_ENC:
 	case DLT_NULL:
 	{
 		int v;
@@ -912,6 +911,7 @@ gen_linktype(int proto)
 		return (gen_cmp(0, BPF_W, (bpf_int32)v));
 		break;
 	}
+	case DLT_ENC:
 	case DLT_RAW: {
 		struct slist *s0, *s1;
 		int ipv;