Operator-native network assurance

Prove your network is behaving correctly.

Trigpoint puts a lightweight agent in every POP and cross-measures latency, loss, jitter, DNS and HTTP across your mesh. It learns what normal looks like on every path, and when something drifts, it tells the NOC what broke, where, who's affected, and why — before the customer calls.

  • <1 MB static agent binary
  • 10 s measurement windows
  • 1 node carries a 50-POP mesh
major alert a7f3-02e1 02:15:34 UTC

Latency level shift — atl ↔ chi

8 paths affected · onset 02:13:41 · +18 ms vs nearest mode (≈60σ)

10 20 30 40 ms 01:50 02:00 02:10 02:20 02:30 baseline 18–22 ms · median ± MAD · hour-of-week opened +1 m 53 s rtt p50 · atl→chi · icmp
bgp
0 events ±120 s — clean
is-is
no adjacency or SPF events — clean
ifaces
inferred link set: no drops, optics nominal
trace
path_hash unchanged on 8/8 flows
cause
transport-layer change (carrier / DWDM) · confidence 0.87
impact
14 customers ride atl-core1 ↔ chi-core2

The gap

Device telemetry is a solved problem. Delivery isn't.

Operators have excellent visibility into devices and links, and almost none into what the network actually delivers between locations. Interface counters are nominal, optics are in spec, every BGP session is established — and a customer in Atlanta is quietly getting 36 ms to Chicago on a path that has run at 19 ms for two years. Router health is green; the customer experience is a question mark.

Trigpoint closes that gap by measuring the thing you actually sell — path performance — continuously, from inside your own POPs. The measurement mesh is table stakes. The product is the intelligence layer that explains why a measurement changed.

How it works

Measure. Learn. Detect. Explain.

01

Measure everything, from everywhere

A single static Rust binary — under a megabyte, no runtime dependencies — registers with the control plane over mTLS and executes a signed test schedule. ICMP, UDP with DSCP marking, TCP connect, DNS against the resolvers your customers actually use, HTTP with full TLS phase timing, STAMP with hardware timestamping where the reflector offers it, and Paris traceroute — UDP, keyed on source port — that enumerates the ECMP paths a LAG fabric really hashes on instead of smearing them. Router-destined ICMP is kept honest as control-plane reachability and never folded into dataplane latency, because a CoPP-rate-limited echo measures the control plane, not transit. Agents ship 10-second aggregates and keep raw 1-second samples in a ring buffer, pulled on demand when the correlator wants forensic detail. When the control plane is unreachable, results spool locally — the moments you most need data are the moments the network is broken.

34 ms 50 ms 20 ms 18 ms 20 ms 16 ms lax dal atl chi nyc
Fixed points, cross-measured — the way a country gets surveyed.
02

Learn what normal means, per path, per hour

The magic is never "latency is 42 ms". It's "latency is normally 18–22 ms on this path at this hour". The baseliner builds per-path, per-metric, hour-of-week baselines using robust statistics — median and MAD, not mean and standard deviation, because network latency distributions are heavy-tailed and often multimodal. A path that ECMPs across two fiber routes has two normal latencies; Trigpoint stores both modes and measures deviation from the nearest one. Anomalous windows are excluded from baseline updates, so an incident never teaches the system that broken is normal.

03

Detect deviations without crying wolf

Deviation scoring with hysteresis and multi-window confirmation — no flapping page at 03:14 because one probe crossed a threshold. The detector recognizes distinct signatures: a level shift (the routing-change shape), variance increase with p99 inflating before p50 (congestion), loss onset with flat RTT (hardware or optics, not congestion), and correlated multi-path anomalies — when many paths through one POP degrade together, the POP is the problem, not the paths.

04

Explain it with your own routing state

Given an anomaly window and the affected path set, the correlator queries the event record: did BGP paths change? Did the IGP reconverge? Did an interface on the inferred path show drops, errors, or optical degradation? Did the forwarding path itself change — a one-comparison check, because every traceroute carries a stable path fingerprint? In an MPLS core, where no ttl-propagate collapses the backbone into one invisible hop, live IGP topology is the primary path source rather than traceroute: SPF over the link graph proves which paths crossed a failed link instead of merely coinciding with it in time. The output is a ranked hypothesis with evidence and a confidence score. Correlation is rule-based and explainable first; every root-cause claim ships with the checks behind it, and "unexplained" is an honest, first-class category — never a guess dressed up as an answer.

Where the agent sits

Four planes, one mesh

"Deploy an agent in every POP" hides a design question: attached to what, measuring through what? Trigpoint models four distinct measurement planes, each with its own targets and cadence — and the agent attaches to the production forwarding plane, never the management network, because measuring the OOB network measures the wrong network entirely.

Device

Agent → each local core, aggregation, and PE router — STAMP to line-card reflectors with hardware timestamps.

1 s · localizes a sick device in seconds

Infrastructure

Agent ↔ agent across the backbone, global table, one measured path per attachment pair.

1–10 s primary · 30–60 s cross-attachment

Service

Agent in a customer VRF → remote PE loopbacks, riding the same LSPs and QoS classes your customers do.

30–60 s · the measurement that backs an SLA

External

Agent → transit and peering next-hops, eyeball targets, resolvers, and CDNs, per exit.

30–60 s · per upstream, scorecard-ready

The payoff is localization without inference. Each agent is dual-homed on two routed uplinks — one per aggregation router, each with its own source address — not a LAG that deliberately hides which router forwarded the packet. So when every path sourced via agr2 degrades while the agr1 twins to the same destinations stay clean, the aggregation uplink is the fault and the alert says exactly that: a group-by, not a guess. Path identity carries the attachment, the VRF, and the target's role, so "which layer is sick" becomes a query — and the cheap intra-POP device plane means the extra dimensions barely move the probe budget.

Features

Built the way operators run networks

A full probe suite

ICMP, UDP with per-class DSCP marking, TCP connect, path MTU discovery, DNS, HTTP(S) with DNS/connect/TLS/TTFB phase timing, source-port Paris traceroute, and STAMP with hardware timestamping. STAMP and TWAMP-Light interoperate with the reflectors already in your routers — Nokia, Juniper, and Arista line cards become schedulable vantage points, so aggregation and PE routers get measured without an agent sitting on them.

Baselines that respect reality

Median + MAD per path, metric, and hour of week. Multimodal baselines for ECMP twin-route paths. Operator-declared physics floors (fiber-distance latency) so gross violations alert on day one, while the system is honest about reduced confidence during its learning period.

Alerts that carry their evidence

Every alert states what deviated and by how much, when it started to the sample, the blast radius, each evidence source checked with its result, and a ranked cause with confidence. If Trigpoint can't say most of that, it holds the alert and keeps gathering — a wrong root cause destroys trust faster than a slightly late alert.

Route changes as a first-class signal

Each traceroute's hop sequence hashes to a stable path fingerprint, so detecting a forwarding change is a single comparison — the correlator's cheapest, highest-value input. "Latency shifted and the path changed 40 seconds earlier" is a root cause, not a mystery.

Your control plane as context

BMP feeds from your route reflectors, IS-IS topology via BGP-LS, gNMI interface and optics telemetry, RPKI validation state over RTR, inventory from NetBox. Trigpoint delivers value with the mesh alone and gets dramatically smarter with each feed you attach.

Impact, not just anomaly

Anomalous paths joined against topology and customer-port inventory answer the question that makes a NOC take an alert seriously: which services and which customers are riding the thing that just broke.

An agent you'd let near production

One static Rust binary, under 1 MB, no GC pauses polluting microsecond timing, kernel timestamping where available. Agents are deliberately dumb: they execute signed schedules and never decide what to test, so fleet behavior stays predictable and auditable. Clock-sync quality ships with every sample; one-way numbers are only trusted when the clocks deserve it.

A mesh that doesn't melt

Full mesh is quadratic: 50 POPs is 1,225 paths, 500 is 124,750. Trigpoint tiers the mesh — core paths at 1–10 s, regional at 30–60 s, a background sweep feeding baselines — then escalates frequency and depth automatically where something smells wrong. With an IGP feed it prunes to the distinct link-path cover, testing your links rather than your permutations.

Your data, your metal

Self-hosted first — operators don't ship their topology to a SaaS. ClickHouse stores measurements and routing events side by side so correlation is a SQL join, not a cross-system export. One modest node (8 vCPU, 32 GB, NVMe) carries a 50-POP mesh with years of rollups. NOCs live in Grafana, so Trigpoint meets them there.

Incidents, not alert storms

When forty paths through one link degrade together, that's one event. Trigpoint rolls anomalies sharing a device, link, or attachment into a single incident with a timeline — thirty path alarms become one — and exports it as an evidence pack, the RFO artifact engineers otherwise assemble by hand at 4 a.m.

Maintenance-aware

Declare a window and anomalies inside it are suppressed and annotated, never dropped — detection never stops, paging does. When the window closes, an automatic post-change report answers the question every change ticket begs: did everything return to baseline, per path and device touched?

On-demand bursts

Mid-incident, fire a traceroute storm, a STAMP burst, or an MTU sweep at a suspect region straight from the alert, the API, or a trig CLI — the same adaptive escalation the scheduler runs on its own, now under an engineer's hand. The first tool you reach for at 03:14.

Per-class truth

The mesh runs per QoS class — every EF and AF path has a best-effort twin, so "EF is slow while best-effort is clean" reads as a policer fault, not a transport mystery. And STAMP reflectors echo the DSCP they received: when a hop silently strips markings, latency stays perfect, the SLA quietly dies, and Trigpoint is the only witness — "sent EF, arrived best-effort" is an alert, with the hop range to check.

The contract with the NOC

Anatomy of an alert

An alert you can't act on is noise with a timestamp. Every Trigpoint alert answers six questions, or it doesn't fire:

  1. What deviated. Metric, path set, and magnitude against the learned baseline — "+18 ms vs the nearest mode, ≈60σ", not "threshold exceeded".
  2. When it started. To the sample, including the gap between onset and confirmation.
  3. Blast radius. POPs, services, and customers affected — from inventory, not guesswork.
  4. Evidence checked. BGP: no change. IS-IS: no events. Interface et-0/0/1: output drops rising. Every source consulted, with its result — including the exculpatory ones.
  5. Ranked likely cause, with confidence. Congestion, route change, hardware, service failure — or, honestly, unexplained.
  6. The drill-down. Links straight to path forensics: raw samples around the window, traceroute history, the routing events that coincided.

And the loop closes: operators confirm or reject every hypothesis, that verdict is recorded next to the evidence, and the correlator's hit rate is itself a tracked metric. The system earns trust the same way an engineer does — by being right, and by being checkable when it isn't.

One platform

Three questions every operator has to answer

Service Trust

"Is the network delivering the experience it should — POP to POP, POP to Internet?"

The continuous measurement mesh, self-learning baselines, anomaly detection, and path forensics. Deploy a container in every POP; get a self-baselining anomaly map of your network.

Routing Trust

"Is the control plane doing what it should?"

Observed BGP compared against RPKI validation state. Hijack and leak detection on your prefixes and your customers'. External vantage answers "does the Internet see us correctly?" while the internal mesh answers "are we seeing ourselves correctly?"

Configuration Trust

"Are the devices configured the way we intend?"

Golden-config drift detection, compliance and security-posture checks, and digital-twin queries — "if this link fails, what happens to these paths?" — grounded in the same live topology the correlator already keeps.

Why another monitoring tool

Inside-out, not outside-in

Enterprise synthetics platforms watch the Internet from the outside: useful if you're a company consuming networks, incomplete if you're the company running one. They don't speak your IGP, they can't see your route reflectors, and they have no idea which customers ride which links.

Trigpoint is built for the other side of the demarc. It understands your AS, your IS-IS topology, your POPs, and your customer inventory — so when a path degrades, the answer isn't a red dot on a world map. It's "the ATL–CHI link changed transport path at 02:13, no IGP or BGP involvement, 14 customers affected, and here is the evidence for each claim."

No product today — commercial or open source — combines a synthetic measurement mesh with the operator's own routing state for root-cause correlation. That's the gap Trigpoint exists to close.

Architecture

Small parts, sharp edges

Storage that matches the questions

Measurement tables cluster by path and time — the shape of "this path, this window". Event tables cluster by time — the shape of "what happened between 02:11 and 02:14". Raw data keeps 14 days, minute rollups 90, hourly rollups two years, because "was it always like this?" is a real question and deserves a real answer.

Deployment without ceremony

Docker Compose for small footprints, Helm for large ones; agents as containers or systemd units. Everything is Rust — the same properties that make the agent trustworthy make the control plane cheap to run. Ingest for a 50-POP tiered mesh is about 38 rows a second; sizing is not the risk.

Where it stands

Being proven the honest way

Trigpoint is in active development and validated continuously against a lab network with thirteen scripted, labeled fault classes: latency steps, bufferbloat congestion, clean packet loss, IS-IS reroutes, dead DNS at 03:14 with the WAN untouched, an aggregation-uplink degrade only one attachment sees, a customer prefix withdrawn upstream, a killed LSP, a starved QoS class, a degrading transit provider, a hop that silently strips DSCP markings, and a resolver that quietly stops validating DNSSEC. The detector catches every one with the correct signature and the correct discrimination — and has held zero false positives on a quiet mesh throughout.

The concept alert fires end-to-end, evidence and blast radius included. An IS-IS reroute is caught as an adjacency-down event within a second, and SPF over the live topology proves which paths crossed the failed link — including a cross-pair set no time-window could have attributed. A customer outage traced to a BMP withdraw event says "routing pull, not fault," naming the prefix and the peers that saw it vanish. Mesh loss joins against interface counters to name the dropping port. A dead LSP surfaces as the alert only a service-plane mesh can produce: the VPN is down and the underlay beneath it is provably clean. A starved EF class is called a class fault because its best-effort twins stay healthy; stripped DSCP markings are caught by the reflector echo with latency perfect; a degrading upstream is named because its sibling transit is clean. A paired probe — one validly-signed name, one deliberately bogus — tells a resolver that stopped validating apart from a zone whose signatures expired, faults that are invisible to every reachability check because resolution keeps "working". Each alert ships as a plain-language narrative — what happened, why we believe it, what to check — with the affected customers attached and the raw evidence one click below.

The loop closes in the tooling that exists today: a NOC web UI with a live mesh matrix and one-click verdicts, a trig CLI for bursts and triage, and a tracked per-hypothesis hit rate — the correlator's accuracy is itself a metric, judged by the operators it serves. The operator plane is production-shaped too: directory (LDAP) sign-in with PingID two-factor, every verdict and rebaseline attributed to the operator who made it, and an intelligence layer that survives reboots honestly — a restarted correlator reconciles every claim it left open, so history never shows an alert that nobody closed.

That bar — a zero-false-positive week followed by caught, explained, customer-attributed degradations — is the standard the production release has to clear. If you run POPs and want a self-baselining anomaly map of your own network, we'd like to talk while the roadmap is still wet.