Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,27 @@ changes; the linked API and deployment documentation contains operational detail
- See `docs/operations/configuration.md#executor-tesla-key-schedule`.

### Added
- API 1.16: RIPE Atlas-style probe addressing in `GET /executors`. Each entry
reports `is_public` and the last dispatcher-observed `address_v4` and
`address_v6` with their `prefix_v4/v6` and `asn_v4/v6` from the offline ASN
database, and `address_observations` (`via` control or reflection,
`observed_at`, lookup source and reason). Only addresses the dispatcher saw
on the executor's authenticated connections are listed, never
`public_host`. Executors call the existing address reflection over each
family every 10 minutes, resolving their `dispatcher.addr` and verifying the
dispatcher certificate, so both families are observed without a configured
reflector (`[connectivity] observe_addresses = false` turns this off).
`[metadata] address_opt_out = true` makes an executor private: its
addresses are withheld from everyone but operators, while prefix, ASN and
location stay public, as for a private RIPE Atlas probe. It is independent
of `location_opt_out`. Results record the same fields in
`provenance.vantage_point.addressing`. `dbl nodes` adds `ADDRESS_V4` and
`ADDRESS_V6` columns. **Privacy:** executors are public by default,
including executors that predate the setting, so their control-connection
address, operator-only until now, is listed once the dispatcher is
upgraded. To keep one private, upgrade it and set `address_opt_out`
before the dispatcher is upgraded
(`docs/vantage-points.md#privacy`).
- `dbl verify` and `client.Verify`: offline probe verification (#73,
`docs/verification.md`, delivery step 3). `client.ReadCapture` reads pcap
and pcapng (Ethernet, raw IP, Linux SLL/SLL2, loopback) up to 64 MiB and
Expand Down
141 changes: 140 additions & 1 deletion api/openapi.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
openapi: "3.0.3"
info:
title: "Debuglet dispatcher HTTP API"
version: "1.15"
version: "1.16"
description: "Wire contract of the dispatcher's public HTTP API. Bandwidth values are bits per second, timeouts are milliseconds, TESLA delays are seconds, TESLA anchor timestamps are Unix nanoseconds and every other timestamp is Unix seconds unless a field says otherwise. A client states the contract version it requires with the Debuglet-API-Version request header, declared on every operation below; every response carries the version this dispatcher implements in the same header. Requests are authenticated with a server-issued session: a native client presents it as an Authorization bearer token, a browser client in the session_token cookie plus the X-Debuglet-CSRF header on state-changing requests. Each operation states its own 401 and 403. Access to another account's run or payment order answers 404, so a response is no existence oracle. A dispatcher explicitly configured for the local development profile additionally serves the unauthenticated routes below as its own local operator. See docs/api.md for the access matrix and the compatibility and deprecation policy."
servers:
- url: "/"
Expand Down Expand Up @@ -7042,6 +7042,42 @@ components:
$ref: "#/components/schemas/ObservedList"
clock:
$ref: "#/components/schemas/ObservedClock"
is_public:
type: boolean
nullable: true
description: "Added in API 1.16. False when the executor set metadata.address_opt_out: the public listing then withholds address_v4 and address_v6, and nothing else. Prefixes, ASNs, location and every other field stay public, as for a private RIPE Atlas probe. An established operator always sees the addresses. True by default, including for executors that predate the setting."
address_v4:
type: string
nullable: true
description: "Added in API 1.16. The last IPv4 address the dispatcher itself observed for the executor: the peer of its control connection, or of an authenticated address reflection call over IPv4 (address_observations.v4 says which and when). Never the executor's own claim such as public_host. Null when none was observed in the current control session, or when is_public is false and the caller is not an operator. It may be a non-global address when the executor reaches the dispatcher over a private network."
address_v6:
type: string
nullable: true
description: "Added in API 1.16. As address_v4, for IPv6."
prefix_v4:
type: string
nullable: true
description: "Added in API 1.16. The prefix of the offline ASN database record covering the observed IPv4 address. Public even when is_public is false. Null when the address is unknown, not global or not in the database."
prefix_v6:
type: string
nullable: true
description: "Added in API 1.16. As prefix_v4, for IPv6."
asn_v4:
type: integer
format: int64
minimum: 1
maximum: 4294967295
nullable: true
description: "Added in API 1.16. The origin ASN of prefix_v4. Public even when is_public is false."
asn_v6:
type: integer
format: int64
minimum: 1
maximum: 4294967295
nullable: true
description: "Added in API 1.16. As asn_v4, for IPv6."
address_observations:
$ref: "#/components/schemas/AddressObservations"
last_seen:
type: integer
format: int64
Expand Down Expand Up @@ -7279,6 +7315,107 @@ components:
nullable: true
description: "Dispatcher receipt time of the report in Unix seconds."
description: "Added in API 1.9. All fields are null together."
AddressObservations:
type: object
nullable: true
additionalProperties: false
required:
- v4
- v6
properties:
v4:
$ref: "#/components/schemas/AddressObservation"
v6:
$ref: "#/components/schemas/AddressObservation"
description: "Added in API 1.16. How and when address_v4 and address_v6 were observed, one record per family. It never carries the address, so it is the same for public and private executors."
AddressObservation:
type: object
nullable: true
additionalProperties: false
required:
- source
- via
- observed_at
- lookup_source
- lookup_reason
properties:
source:
type: string
enum:
- dispatcher-observed
via:
type: string
enum:
- control
- reflection
description: "control: the peer address of the executor's control connection, preferred for its family. reflection: the peer address of the executor's last authenticated ReflectAddress call over this family."
observed_at:
type: integer
format: int64
description: "Unix seconds of the last observation: the last heartbeat of the control session that carries the control address, or the receipt of the last reflection call."
lookup_source:
type: string
nullable: true
pattern: "^database:[^@\\s]+@[1-9][0-9]*$"
description: "The offline ASN database the prefix and ASN come from. Null when no lookup happened."
lookup_reason:
type: string
enum:
- ""
- "no_database"
- "no_address"
- "not_ip"
- "non_global"
- "opted_out"
- "not_found"
- "lookup_error"
- "invalid_record"
description: "Empty when the database knew the address; otherwise why the prefix and ASN are null."
description: "Null when no address of the family was observed in the current control session."
ProbeAddressing:
type: object
nullable: true
additionalProperties: false
required:
- is_public
- address_v4
- address_v6
- prefix_v4
- prefix_v6
- asn_v4
- asn_v6
- address_observations
properties:
is_public:
type: boolean
nullable: true
address_v4:
type: string
nullable: true
address_v6:
type: string
nullable: true
prefix_v4:
type: string
nullable: true
prefix_v6:
type: string
nullable: true
asn_v4:
type: integer
format: int64
minimum: 1
maximum: 4294967295
nullable: true
asn_v6:
type: integer
format: int64
minimum: 1
maximum: 4294967295
nullable: true
address_observations:
$ref: "#/components/schemas/AddressObservations"
description: "The listing's RIPE Atlas-style addressing fields (see ExecutorResponse) as known at admission. The addresses are recorded for a private executor too, since a result is visible only to the run's owner and operators."
ExecutorList:
type: array
nullable: true
Expand Down Expand Up @@ -8218,6 +8355,8 @@ components:
$ref: "#/components/schemas/ResultLabelledClock"
platform:
$ref: "#/components/schemas/ResultLabelledPlatform"
addressing:
$ref: "#/components/schemas/ProbeAddressing"
description: "Added in format 1.1. Where the run was to execute, as known at admission. Each value names its source; no label means verified. Null for runs admitted before format 1.1, and always absent from 1.0 files."
nullable: true
ResultSourceLabel:
Expand Down
6 changes: 4 additions & 2 deletions api/spec.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ var OpenAPI []byte
const (
// Version is the HTTP API contract version as major.minor. It equals the
// info.version field of OpenAPI.
Version = "1.15"
Version = "1.16"
// Major is the compatibility number of Version. Clients that require a
// different major version are answered with an explicit incompatibility
// error instead of a best-effort response.
Expand All @@ -39,7 +39,9 @@ const (
// payment features, usage allowances (GET /me/allowance, POST
// /operator/accounts/{id}/allowance) and executor earnings
// (GET /operator/executors/{id}/earnings).
Minor = 15
// 1.16 added the RIPE Atlas-style is_public, address, prefix and ASN
// fields of each address family to GET /executors.
Minor = 16
// VersionHeader carries the contract version a client requires on requests
// and the version the dispatcher implements on responses.
VersionHeader = "Debuglet-API-Version"
Expand Down
10 changes: 8 additions & 2 deletions cmd/dbl/capabilities_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ func TestNodesShowsVantageColumnsAndFiltersISDAS(t *testing.T) {
mux.HandleFunc("GET /executors", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`[{"id":"old","ready":true},
{"id":"lab","ready":true,"admission":"maintenance",
{"id":"lab","ready":true,"admission":"maintenance","is_public":false,"address_v4":null,"address_v6":null,"asn_v4":64500,
"display":{"display_name":{"value":"ETH lab","source":"operator"},"city":{"value":"Zurich","source":"operator"},"country":{"value":"CH","source":"operator"},"network":{"value":null,"source":null}},
"scion_isd_as":{"value":"1-ff00:0:110","source":"executor-reported","observed_at":1},
"listeners":{"value":["udp","scion"],"source":"executor-reported","observed_at":1}}]`))
Expand All @@ -98,7 +98,7 @@ func TestNodesShowsVantageColumnsAndFiltersISDAS(t *testing.T) {
code, stdout, stderr := runCLI(context.Background(), "--endpoint", fx.endpoint(), "nodes")
assertCode(t, code, exitOK, stdout, stderr)
lines := strings.Split(strings.TrimSpace(stdout), "\n")
const header = "ID READY NAME LOCATION ISD_AS ASN LOCATION_SOURCE LAST_SEEN VERSION PRICE_PER_BW CURRENCY PROTOCOLS ENFORCEMENT CAPACITY_BPS ATTRIBUTION IPV4 IPV6 TCP_LISTENER UDP_LISTENER"
const header = "ID READY NAME LOCATION ISD_AS ASN ADDRESS_V4 ADDRESS_V6 LOCATION_SOURCE LAST_SEEN VERSION PRICE_PER_BW CURRENCY PROTOCOLS ENFORCEMENT CAPACITY_BPS ATTRIBUTION IPV4 IPV6 TCP_LISTENER UDP_LISTENER"
if len(lines) != 3 || strings.Join(strings.Fields(lines[0]), " ") != header {
t.Fatalf("table: %q", stdout)
}
Expand All @@ -108,6 +108,12 @@ func TestNodesShowsVantageColumnsAndFiltersISDAS(t *testing.T) {
if !strings.Contains(lines[2], "ETH lab Zurich,CH 1-ff00:0:110") {
t.Fatalf("vantage row: %q", lines[2])
}
if fields := strings.Fields(lines[1]); fields[6] != "-" || fields[7] != "-" {
t.Fatalf("legacy addresses: %q", lines[1])
}
if fields := strings.Fields(lines[2]); fields[7] != "private" || fields[8] != "private" {
t.Fatalf("private addresses: %q", lines[2])
}
// The default table stays compact; admission, network and listeners are
// in --output json only.
for _, hidden := range []string{"ADMISSION", "NETWORK", "LISTENERS", "maintenance", "udp,scion"} {
Expand Down
31 changes: 26 additions & 5 deletions cmd/dbl/query.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"fmt"
"io"
"net/http"
"net/netip"
"runtime/debug"
"strconv"
"strings"
Expand All @@ -27,8 +28,10 @@ Filters return ready matching executors only. Unknown capability reports and an
unknown ISD-AS do not match. Capacity means advertised total bandwidth, not free
admission capacity. NAME is the dispatcher operator's label; LOCATION prefers
operator location and otherwise shows approximate database location with its source;
ISD_AS is the executor's own report. --output json also carries admission, the
operator's network label and the reported listener transports.
ISD_AS is the executor's own report. ADDRESS_V4 and ADDRESS_V6 are the addresses
the dispatcher last observed, "private" when the executor withholds them. --output
json also carries admission, the operator's network label, the reported listener
transports and each address family's prefix, ASN and observation time.

Lists the dispatcher's registered executors. JSON output is always an array.
`
Expand Down Expand Up @@ -89,7 +92,7 @@ func nodesCommand(ctx context.Context, args []string, options globalOptions, std
}
return emit("dbl nodes", options.Output, stdout, stderr, nodes, func(w io.Writer) error {
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
fmt.Fprintln(tw, "ID\tREADY\tNAME\tLOCATION\tISD_AS\tASN\tLOCATION_SOURCE\tLAST_SEEN\tVERSION\tPRICE_PER_BW\tCURRENCY\tPROTOCOLS\tENFORCEMENT\tCAPACITY_BPS\tATTRIBUTION\tIPV4\tIPV6\tTCP_LISTENER\tUDP_LISTENER")
fmt.Fprintln(tw, "ID\tREADY\tNAME\tLOCATION\tISD_AS\tASN\tADDRESS_V4\tADDRESS_V6\tLOCATION_SOURCE\tLAST_SEEN\tVERSION\tPRICE_PER_BW\tCURRENCY\tPROTOCOLS\tENFORCEMENT\tCAPACITY_BPS\tATTRIBUTION\tIPV4\tIPV6\tTCP_LISTENER\tUDP_LISTENER")
for _, n := range nodes {
location := n.Display
location.City, location.Country = n.Location()
Expand All @@ -112,14 +115,32 @@ func nodesCommand(ctx context.Context, args []string, options globalOptions, std
if connectivity == nil {
connectivity = &wire.Connectivity{}
}
fmt.Fprintf(tw, "%s\t%t\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", n.ID, n.Ready,
labelText(n.Display.DisplayName), nodeLocation(location), observedText(n.SCIONISDAS), nodeASN(n.IPMetadata), locationSource(location),
fmt.Fprintf(tw, "%s\t%t\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%d\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n", n.ID, n.Ready,
labelText(n.Display.DisplayName), nodeLocation(location), observedText(n.SCIONISDAS), nodeASN(n.IPMetadata),
addressColumn(n.AddressV4, n.IsPublic), addressColumn(n.AddressV6, n.IsPublic), locationSource(location),
lastSeen, n.Version, n.PricePerBw, n.Currency, protocols, enforcement, capacity, attribution, reachabilityColumn(connectivity.IPv4), reachabilityColumn(connectivity.IPv6), reachabilityColumn(connectivity.TCPListener), reachabilityColumn(connectivity.UDPListener))
}
return tw.Flush()
})
}

// addressColumn shows an observed address, "private" when a private executor's
// address is withheld and "-" when none is known. A value that is not an
// address is not printed.
func addressColumn(address *string, public *bool) string {
if address == nil {
if public != nil && !*public {
return "private"
}
return "-"
}
ip, err := netip.ParseAddr(*address)
if err != nil {
return "invalid"
}
return ip.String()
}

func reachabilityColumn(r wire.Reachability) string {
if r.Stale || r.ExpiresAt != nil && time.Now().Unix() >= *r.ExpiresAt {
return "stale"
Expand Down
2 changes: 2 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ An account's allowance is `granted`, the sum of its grants, less what its `TEST`

API 1.9 also adds `admission`, operator `display` metadata, the executor-reported `scion_isd_as` and `listeners` to `GET /executors`. Each value carries a source label; see [executor discovery](operations/executor-discovery.md#vantage-point-metadata). It also adds the executor's host probes: `capabilities.icmp`, `capabilities.enforcement_reason` and `clock`. Host platform detail is operator-only and is never listed; it appears only in result provenance ([host probes](operations/executor-discovery.md#host-probes)).

API 1.16 adds RIPE Atlas-style addressing to every entry: `is_public`, the dispatcher-observed `address_v4` and `address_v6`, their `prefix_v4`, `prefix_v6`, `asn_v4` and `asn_v6`, and `address_observations` with how and when each address was seen. A private executor's addresses are withheld from everyone but operators; nothing else is ([probe addresses](operations/executor-discovery.md#probe-addresses)).

## Probe attribution

API 1.11 adds two public routes for [probe verification](verification.md). They need no credential and are rate-limited per client address; an excess answers `429 rate_limited` with `Retry-After`. Behind a reverse proxy the address is the proxy's unless the proxy is listed in `[attribution] trusted_proxies` ([configuration](operations/configuration.md#dispatcher-attribution-history)).
Expand Down
Loading