Reaching a machine you cannot route to

The architecture post describes the happy path: a cheap office PC, a fingerprint or face terminal on the same network, a read every minute, punches spooled to disk and sent to a cloud attendance platform. This post is about the offices where that path does not exist. Nothing is broken in them; the machine and the PC are just not where IP routing would put them next to each other.

Four shapes came up often enough to build for: a machine on the same cable in another range; a PC behind its own router with the machine on the internet provider's LAN; a machine that pushes; and a PC that cannot reach the cloud at all. Addresses are illustrative; shapes and numbers are real.

One cable, two ranges

ZKTeco terminals ship at 192.168.1.201. Hikvision's ship at 192.168.1.64, with a fallback in the 192.0.0.x block. An installer who plugs one into an office on 10.40.40.0/24 and leaves has left a machine that is physically next to the PC, on the same switch, and unreachable. The PC has no route for 192.168.1.0/24, so it hands the packet to its default gateway, and the router has no such network either.

The fix nobody has to drive for is a second address: an IP alias in the machine's /24 on the PC's own adapter, next to its DHCP address. With 192.168.1.250 there, 192.168.1.0/24 is on-link; the PC asks for the machine by ARP and talks to it down the cable. The connector picks from .250 down to .240, skipping the machine's address; on Windows, one that duplicate detection marks Duplicate is taken off and the next one tried. Aliases go into the OS's active store only, so a reboot removes them and an uninstalled connector leaves nothing behind; a note per alias in the spool folder lets it put back the ones still needed at start.

A second address changes how the PC routes, so it is fenced in: private ranges only (10/8, 172.16/12, 192.168/16 and the 192.0.0.x fallback), at most four, never overlapping a network the PC is on, and only on evidence. The overlap rule has one subtle case. A PC on 192.168.1.10/25 owns .0 to .127. A machine at 192.168.1.201 is outside that /25, so "is it on my network?" says it needs an alias, but a /24 alias would contain the PC's own /25 and steal its traffic. So the check is containment, both ways:

n := slash24(ip)
for _, l := range locals {
	// Either one inside the other: an alias would steal traffic
	// from a network the PC really is on.
	if l.Contains(n.IP) || n.Contains(l.IP) {
		return nil, fmt.Errorf("%s overlaps this PC's network %s", n, l)
	}
}

The table test that pins it puts the /25 case next to the near misses, and no test touches a real adapter: every OS command goes through a replaceable runner (testing against hardware we do not own).

Only on evidence

The first version added an alias for any machine whose address was private and not local. That is wrong in any office bigger than one switch, where a private address that is not local is usually a routed network (another floor, another VLAN) that the router already delivers to. An alias there tells the PC the range is on-link, the PC ARPs for the machine instead of using the router, and it cuts itself off from a network that worked a minute earlier. So a second address now needs evidence that the machine is on this PC's cable: this PC's own search saw it answer from that range (a SADP or DHIP reply whose source matched the address it claimed), or find_machine found it there. The search writes a note per address, and the alias code asks for it:

// Never for an address that is merely not local: in a bigger office
// that is a routed network, and an alias would cut this PC off from it.
func (r *Runner) aliasAllowed(host string) bool {
	ip := net.ParseIP(host).To4()
	return ip != nil && r.notes.Get("off_range::"+ip.String()) != ""
}

At start the runner adopts the aliases a previous run left, puts back the ones with evidence that a reboot removed, and prunes every one without it, because earlier builds had added them freely. The guard gives the runner an unseen address and checks no alias command ran, then feeds it an off-range search result and checks exactly one alias appears. Networks one router away never earn the note: an alias there would break the routing that reaches them.

One bug here cancelled itself. Once 192.168.1.250 was on the adapter, 192.168.1.0/24 was a local network, so the next search saw the machine on a local network, stopped calling it off-range, and removed the reason for the alias. Search replies are now judged against the PC's own networks: every local network minus the ones a second address created.

Better than an alias: move the machine

The real fix is a machine on the office's range, and for Hikvision the PC can make it. The set_dhcp instruction reads the machine's own network document over ISAPI, changes one field and sends it back, so nothing else on the machine moves:

var addressingType = regexp.MustCompile(`(<addressingType>)\s*[A-Za-z]*\s*(</addressingType>)`)

raw, err := r.send(ctx, "GET", ipAddressPath, nil)
// ...
_, err = r.send(ctx, "PUT", ipAddressPath, addressingType.ReplaceAll(raw, []byte("${1}dynamic${2}")))

Then the connector looks for the machine by serial and MAC every 15 seconds for up to three minutes. When it turns up, the platform is told, the alias comes off, and the next read goes to the new address. When it does not, the instruction fails with a message saying the second address stays and keeps working, which it does. There is no ZKTeco equivalent yet.

For a machine lost at an unknown address, find_machine stops at the first step that works: SADP for a Hikvision; the typed address, or else the three factory ones, each with a temporary alias kept only if the machine is there; then a TCP sweep of 192.168.0.x to 192.168.2.x and 10.0.0.x, 64 at a time. A hit counts only if its device info carries the right serial. It reports each step, stops at ten minutes, and runs only when someone asks.

A fixed address block: the machine link

Second addresses solve reading. Pushing has the opposite problem. A terminal that pushes to the PC is configured with the PC's address, typed once on its keypad. That address comes from DHCP, and after a power cut the router may hand out another one; the terminal then sends to nothing, and nobody notices until payroll. So the PC holds a machine link: a small private block on its default-route adapter, beside the DHCP address. The PC keeps .1, the machines get .2 to .14 on a /28, and neither end moves when the router restarts.

After 20 refused blocks the connector asks IT to set one with connector link 10.20.30.0/28; connector link auto goes back, never discarding a block the connector picked, since machines use it. The block lives in the protected data folder. Holding it is a tick at start, on a network change and every minute that checks before it acts (in place, it runs only an ARP read), follows the default route, puts the address back if the OS dropped it, and never fights: if another device answers at .1, the connector stops using it and says so. A stranger's MAC at a machine's address is only reported; that address is the machine's to keep.

On Windows the first version needed "DHCP/static IP coexistence" switched on, because adding a fixed address the old way turned the adapter's DHCP off. Now it goes in through the IP Helper API, in the active store, marked skip-as-source so the PC uses it for nothing but the machines. Measured on a hosted Windows CI runner, DHCP stays on and keeps its address either way. Why it starts no netsh is its own post. When the link address appears or moves, the running relays rebind to it. And the link has one limit that matters next: it only works on one cable.

One router away

The office where this was found had its Wi-Fi on its own router, the PC on that network, and two ZKTecos plugged into the internet provider's router, one network up. At home the same build had worked, because at home there was one network. In the office the PC reached the machine in two hops, port 4370 answered, reads worked, and nothing the machine sent ever arrived. I settled it with a packet capture on the PC, filtered for the machine's MAC and for ARP asking who had the link's .1: silence, while the router's own broadcasts kept arriving. Then I turned DHCP on at the machine and watched it land on the provider's network.

The office router does NAT. Machine to PC is impossible: it will not forward an unsolicited connection inward. Machine to cloud is possible on paper and was ruled out; the person who runs the product put it plainly, "switch to cloud is not reliable". A machine pushing straight to the cloud has no disk to wait on when the internet drops, and nobody in the office can see whether it works. PC to machine is outbound, the one direction NAT is built to allow.

So the PC reads, and three changes made that automatic. The search also looks one router away, so the machine appears in the list at all (finding machines that do not announce themselves has how). A pushing machine can be read as well: when nothing has arrived from one for 15 minutes the platform marks it read_too, the PC reads it every poll, and its batches say so; Sintech, which only pushes, is never read. And a locked ZKTeco's comm key is asked for once, then used for every read. Measured in that office afterwards, a punch reached the platform about 50 seconds after the finger left the sensor: the 60-second read interval doing what it says.

Across a router a failed read is more often a firewall than a dead machine, so each is sorted into port_closed, timeout, no_route or comm_key, with the port. The line for IT, naming the port and this PC's address on the route, appears only for a closed port on another network, a path IT can open; on the PC's own network the machine itself refused, and that line would send IT hunting for a firewall rule that does not exist.

Pushing through the office PC

Some offices want their machines to push: ZKTeco's push mode (ADMS, the /iclock/ HTTP protocol) and Sintech's UDP push. The connector stands in that path as a relay. A relay starts only when the machine list has a machine set to push through this PC and stops when it has none, so an install that never uses one never opens a port.

The ADMS relay listens on TCP 8085, the vendor's own convention, and is dumb on purpose: it never builds, parses or changes protocol content and has no commands of its own. Each request (method, path, query, base64 body, client address) travels inside the PC's own signed call to the platform, and the answer goes back to the terminal as it came. What it does own:

• Where it listens: every private address the PC has, office address and machine link alike, never 0.0.0.0. Its Windows Firewall rule is the Private profile only, because the port carries no credential.
• Who may call: private IPv4, loopback, and unique-local (fc00::/7) or link-local (fe80::/10) IPv6 with the zone (%en0) stripped; 120 requests a minute per source. The IPv6 rule came from a push terminal seen reaching the cloud over IPv6, from an address built out of its MAC. The listeners still bind IPv4 only, so the check is ahead of the socket.
• Which paths: cdata, getrequest, devicecmd and querydata under /iclock/, folded to one lowercase form first, because firmware writes /iclock/CDATA.aspx, /iclock/cdata/ and /iclock/cdata for one thing. querydata is where newer push firmware answers a DATA QUERY tablename=... command, added after a terminal acknowledged a request for its user list and sent no rows. Whether that firmware answers at all is not proven yet.
• Offline: a GET gets a bare OK, because the relay cannot invent a command; a POST is stored and gets OK too, so the terminal moves on. The store holds 50,000 requests or 200 MB, drops the oldest only as a last resort, and drains in order, 50 every 15 seconds. A request the platform refuses outright is dropped so it cannot block the rest; the platform dedupes by hash, so a second copy is harmless.

One thing I did not expect it to own was the machine's clock. A ZKTeco in push mode sets its clock from the Date header of every answer, and Go's HTTP server fills that header from the PC's clock, so a PC with a wrong clock was quietly putting every machine behind it wrong too. The relay now sends the platform's time, from the offset the runner already measures (time is the hardest field in a punch has the rest):

// The machine sets its clock from this header: the platform's time, not this PC's.
w.Header().Set("Date", time.Now().Add(s.offset()).UTC().Format(http.TimeFormat))

The Sintech relay is its UDP twin on port 5055. It never decodes a frame or composes a reply: each datagram goes up in base64 with its source and arrival time, and the only bytes sent back are the platform's, verbatim. A datagram over 512 bytes is dropped, never cut. A live one is a batch of one, because its reply must go back on the same socket while the exchange is open; offline frames are stored with no reply owed and drain later in batches of up to 200, a refused batch resent frame by frame to find the bad one.

When the cloud is what you cannot reach

The last shape: the machines are fine and the PC cannot reach the platform. That support call used to start with "is the internet working?" and end an hour later at a proxy nobody had mentioned. Now every failed request gets a kind support can act on: dns; connect_timeout (a firewall dropping the traffic); connect_refused; connect_reset (a firewall filtering by site name); proxy_auth (a 407); tls_inspected (a middlebox's own certificate); tls_other (often the PC's clock); http_blocked (a block or sign-in page answered instead); server_error (the platform's own 5xx).

What makes the classifier trustworthy is that the platform always answers JSON: a JSON reply came from the platform whatever its status, and a 200 with an HTML page is a block page. Windows' socket errors are matched by number (10061, 10054), because there they are not syscall.ECONNREFUSED and syscall.ECONNRESET. Around it:

• Proxy: its own setting, or HTTPS_PROXY. No PAC files or per-user Internet Options: the service runs as SYSTEM and would not see them.
• HTTPS inspection: the certificate is always fully checked against the system's store. An inspection box works only if IT installed its certificate; otherwise it is tls_inspected, and nothing skips the check.
• VPN capture: if the route to a machine or the router leaves through a tunnel adapter, or another adapter than the one owning that network, the heartbeat says vpn_captures_lan and names it.
• Firewall rule: one inbound rule, UDP, to the installed program only, from the local subnet, Private and Domain profiles, so replies to the search's multicast probe are not dropped.
• Wi-Fi isolation: guest Wi-Fi and AP isolation let a device reach the router and nothing else. If the PC is on a Wi-Fi adapter and only the router answered in the range its sweep covered, the heartbeat says wifi_isolated; false if anything else answered or a machine was found; nothing when it cannot tell. Routers are only the ones the system reports, never a guessed .1.

connector doctor runs all of it and prints one line per check, written to be read out on the phone, with a fix under each failure. An illustrative run:

[ OK ] No proxy: this PC connects to the platform directly.
[FAIL] The platform could not be reached. The connection was cut off part-way,
       which is what a firewall filtering by website name does. [connect_reset]
       Fix: Ask IT to allow the platform's address in their firewall's website filter.
[FAIL] A VPN (wg0) is taking traffic meant for the office machines, so they cannot be reached.
       Fix: Disconnect the VPN on this PC, or ask IT to turn on "allow local network access".
[ OK ] Windows Firewall rule for machine replies is in place.
[ OK ] Search found 2 machine(s).

It exits non-zero when any line fails, and on a paired PC it signs its probe, so the platform logs no failed sign-in for every check-up.

Noticing that the PC moved

A laptop that moves office, or an office with a new router, is on a network nobody has searched. The slow loop notices and searches once by itself: at most once per network a day, never during another search, the machine link moved first. The first version keyed on every adapter, and Hyper-V and WSL give theirs a new 172.x/20 on every boot, so every reboot looked like a move. The key is now only the adapter holding the default route:

func (r *Runner) watchKey() string {
	local := r.routeLocal() // a UDP "dial": the OS names the route, no packet sent
	for _, n := range r.localNets() {
		if n.IP.Equal(local) {
			return label(n) // "10.40.40.x"; Hyper-V, WSL and Docker adapters never match
		}
	}
	return ""
}

The alias and the link go on that adapter too, unless it is a VPN tunnel with no broadcast, where an address reaches no machine; then the first real broadcast adapter with a private address wins, virtual switches skipped.

What I would tell someone building this

• Same cable is not same network, and the reverse. Check layer 2 first: capture for the machine's MAC, turn DHCP on at the machine, see where it lands. Ten minutes of that told "a bug in the link" from "the machine is behind another router" in an office where I had assumed the first.
• Add routing state only on evidence, and judge with the state you did not create. An alias is right for a machine seen on your cable, and once it is up it changes what your own checks see.
• Hold addresses, never fight for them. Check before acting, follow the default route, back off the moment something else claims one.
• NAT picks the direction. The PC can always dial out, so let it call the machine, and put a push behind the PC only on a shared cable.
• Keep the relay dumb. Forward bytes, store them offline, and own only where it listens, who may call and the time it hands out.
• Name the failure in the user's words. A kind per failure and a fix line per kind turned an hour of "is the internet working?" into one line read out on the phone.