bug: dhcpcd deletes the shared 192.168.2.0/24 route on lease loss, orphaning cable-guy's static eth0 address (ENETUNREACH, 40 min comms loss underway)

[vshie]

Bug description

BlueOS	1.5.0-beta.39-0-g7adad3f6
Hardware	Raspberry Pi 4 Model B Rev 1.5
OS / kernel	Debian 12 (bookworm), 5.10.33-v7l+
Vehicle	BlueBoat, ArduRover 4.7.0 (1511f271)
eth0 config	static 192.168.2.12/24 (cable-guy, AddressMode.Unmanaged) and host dhcpcd leasing 192.168.2.143/24 — both on the same subnet
MAVLink endpoints	UDP clients to 192.168.2.1:14550 and 192.168.2.128:14550
Date of incident	2026-07-27 / 2026-07-28, all times UTC

Summary

eth0 carries two IPv4 configurations on the same subnet: the static 192.168.2.12/24 that cable-guy applies, and a 192.168.2.143/24 lease that the host's dhcpcd obtains. They share one connected route, 192.168.2.0/24, and dhcpcd believes it owns it.

A physical-layer fault caused 38 eth0 carrier losses in under an hour. Every time dhcpcd lost its lease it also ran eth0: deleting route to 192.168.2.0/24. At 23:37:32 it deleted that route for the last time and never re-added it. The static 192.168.2.12 remained on the interface, but with no route to its own subnet every outbound packet to 192.168.2.x failed with ENETUNREACH — for 40 minutes, including the final 27½ minutes during which the carrier was up and stable.

mavlink-server logged 11,220 send failures on this boot, 11,085 of them to 192.168.2.1:14550 and 192.168.2.128:14550, all identical:

Failed to send message: Os { code: 101, kind: NetworkUnreachable, message: "Network unreachable" }

From the operator's point of view 192.168.2.12 was dead while every other device on the boat network still responded. The vehicle was underway, station-keeping in GUIDED at a commanded position. Recovery required physically reaching it and power-cycling it.

Evidence that the address survived and the route did not

• avahi-daemon withdrew and re-registered 192.168.2.143 on every single flap. It registered 192.168.2.12 once at 21:28:30 and never withdrew it — so the static address was on the interface for the entire incident.
• ENETUNREACH (errno 101) is what the kernel returns when there is no route to the destination network. A missing address or a dead link produce different errors.
• dhcpcd logged eth0: adding route to 192.168.2.0/24 followed one second later by eth0: pid 679 deleted route to 192.168.2.0/24, twice (22:47:01/02 and 22:54:07). That is the two configurations fighting over the same prefix route.
• After the carrier stabilised at 23:50:09, dhcpcd logged exactly one solicitation (23:50:10) and then nothing at all for 27½ minutes. Nothing else in the stack noticed or intervened.

Independent confirmation from a second process

A separate logger running on the same Pi polls the vehicle's Mikrotik radio over 192.168.2.x at ~1.5 s intervals. It went blind at exactly the same moments as mavlink-server — each of its long outages begins 2–5 s after a dhcpcd route deletion and ends 2–14 s after the route is restored — while its position column, read locally over MAVLink, kept updating normally throughout. The Pi itself was healthy; only the path to 192.168.2.0/24 was gone.

That log also rules out the radio as the cause of the 40-minute outage:

• The vehicle drifted from 531 m to 208 m during the blackout — steadily closer to the topside station and into stronger signal — and never recovered.
• Measured median SNR by range on the same afternoon: 45 dB at 100–199 m, 41 dB at 200–299 m, 37 dB at 300–399 m. The blackout was spent entirely inside the range band that had been carrying full-rate traffic all day.
• At 23:07:02 the kernel logged Link is Down, the carrier returned at 23:07:05, and the radio answered a poll at 23:07:05 with live data (27 dB SNR, 18 Mbps). A RouterOS device that had just rebooted could not answer three seconds later — the radio never went down. Polls only began failing at 23:07:10, the instant the lease expired.
• When the link was finally re-tested at 00:21:24 from 168 m, it associated instantly at 43 dB and 72.2 Mbps.

What actually restored service

Not DHCP. On the following boot dhcpcd solicited from 00:17:18 and gave up with timed out at 00:17:44; no lease was ever granted again. Comms returned at 00:18:58, when cable-guy applied the static 192.168.2.12 — and this time there was no DHCP client on the same subnet to delete its prefix route. That is the entire difference between the working and non-working states.

This is not a one-off

The same signature appears on a previous outing: 2026-07-23, 22:12:15–22:30:59, 6,078 ENETUNREACH failures split evenly between 192.168.2.1 and 192.168.2.128.

Question for maintainers

Does cable-guy add the static address in a way that sets IFA_F_NOPREFIXROUTE (or otherwise suppresses the kernel's automatic prefix route)? If so, dhcpcd's route is the only route for 192.168.2.0/24 and deleting it is immediately fatal, which matches everything observed here. If the kernel prefix route does normally exist, then dhcpcd is deleting a route it did not create, which is also a problem.

Steps to reproduce

1. Configure eth0 with a cable-guy static address on a subnet that also has a DHCP server (the stock 192.168.2.0/24 tether subnet).
2. Let the host dhcpcd obtain a lease on that same subnet.
3. Flap the link repeatedly — unplug/replug ethernet, or use a marginal Wi-Fi bridge — until dhcpcd cannot complete a handshake.
4. Restore the link and leave it up. The interface still holds the static address, but ip route no longer has 192.168.2.0/24 dev eth0, and everything on that subnet is unreachable indefinitely.

Timeline (UTC)

Time	Event	Source
21:17:01	Boot	journal
21:28:30	avahi registers 192.168.2.12 on eth0 — never withdrawn for the rest of the boot	journal
22:47:01	First DHCP lease 192.168.2.143; adding route to 192.168.2.0/24	journal
22:47:02	pid 679 deleted route to 192.168.2.0/24 — static and DHCP configs collide	journal
22:53:02	First eth0 carrier loss	journal
22:59:17	First of 17 Pi 5 V under-voltage warnings	journal
23:07:02	Carrier loss; mavlink-server begins failing with ENETUNREACH	journal + mavlink-server
23:07:05	Carrier back; Mikrotik still answering with live link data — radio never went down	mikrotik log
23:07:10	Lease expires; subnet becomes unreachable	journal
23:07:16	ArduPilot declares GCS failsafe	dataflash
23:10:55	New lease, adding route to 192.168.2.0/24 → 23:10:58 failsafe cleared	journal + dataflash
23:22:10 / 23:28:31	Two more deleting route / adding route cycles; both self-recovered	journal
23:36:12	deleting route; start of a burst of 33 carrier drops in 14 minutes	journal
23:37:16	Last successful DHCP lease; route re-added	journal
23:37:32	eth0: deleting route to 192.168.2.0/24 — never re-added	journal
23:37:52	GCS failsafe declared; never cleared again	dataflash
23:50:09	eth0: carrier acquired — link stable from here to the end	journal
23:50:10	One soliciting a DHCP lease, then no eth0 activity logged at all	journal
23:50:10 – 00:17:44	27½ min: carrier up, static address present, no route, no recovery	journal
00:17:44	Pi hard-resets (no shutdown sequence in the journal)	both
00:17:18 – 00:17:44	Next boot: dhcpcd solicits, then timed out. DHCP never works again	journal
00:18:58	cable-guy applies static 192.168.2.12; comms restored	journal

Primary pain point(s)

An autonomous surface vehicle underway at sea was unreachable for 40 minutes, while its radio, its autopilot and its towfish payload all continued to function normally. Recovery required physically reaching the vehicle.

Expected behaviour

• A transient carrier loss causes a brief packet gap, not the removal of the interface's routing.
• A DHCP client must not be able to delete the connected route belonging to a statically configured address on the same interface.
• If the primary interface has carrier but cannot reach its own subnet for more than a few seconds, something escalates and repairs it. A watchdog that only checks "is the address present?" will not catch this — the address was present.
• A MAVLink endpoint unreachable for minutes is surfaced as a health condition, not only as repeated log lines.

Actual behaviour

None of the above. The interface sat with carrier, with its static address, and with no route to its own subnet, indefinitely.

Possible mitigations

• Do not run dhcpcd on an interface that cable-guy has statically configured on the same subnet — either exclude it in dhcpcd.conf via denyinterfaces, or have cable-guy stop the DHCP client when it applies a static address.
• Ensure the static address keeps its own kernel prefix route independent of any DHCP lease.
• Extend the cable-guy watchdog to verify routability (address and a matching route, or an actual ARP/ping to the gateway) rather than address presence alone, and to re-assert the static configuration when that check fails.

Additional context

Logs: https://drive.google.com/drive/folders/1tqOLXCzkcC6oYky4V-z25GpODL4YBcQT?usp=sharing

The physical-layer fault that triggered this (38 carrier drops, plus 17 Pi 5 V under-voltage warnings) is a problem on our vehicle and not a BlueOS bug. The 40-minute unrecoverable outage that a 3-second carrier glitch was able to cause is the part being reported here.

Prerequisites

• I have checked to make sure that a similar request has not already been filed or fixed.

[joaoantoniocardoso]

Thanks for the report.

Just so we don't confuse the situation: this issue is not solved by #3996.