Skip to content

Operational Lifecycle — Init → Execute → Sync

Status & scope

  • Module: lens/runtime/ (planned)
  • Milestone: Phase 3

Purpose

Define how the lens framework operates across three phases: connected initialization (data staging), disconnected crisis execution (off-network compute), and reconnection (evidence sync). This is the full lifecycle from customer problem set to auditable decision.

The Three Phases

PHASE 1: INIT (connected)          PHASE 2: EXECUTE (off-network)       PHASE 3: SYNC (reconnected)
─────────────────────────           ──────────────────────────────       ────────────────────────────
Download + cache data               LLM configures scenario              Evidence blocks sync to DES
Pre-compute baselines               Framework computes (pure Python)     SHA-256 hashes verify integrity
Validate lens specs                 COA comparison                       After-action review
Stage to local dataspace            Human attests decision               Blocks become permanent record
                                    Evidence frozen locally

Phase 1: Initialization (Connected, Pre-Crisis)

What Gets Staged

Data Type Source Storage TTL Size Estimate
Road network OSM Overpass API Pre-built graph (.graph.json.gz, §graph-serialization-format) Static — update quarterly ~5MB per 100km²
TLE catalog CelesTrak Raw TLE text files Live — update every 12h (orbital decay) ~1MB for full catalog
DEM tiles USGS/Copernicus GeoTIFF tiles Static — terrain doesn't change ~10MB per 1° tile
Weather forecast Open-Meteo / GFS JSON grid Live — 6h refresh ~2MB per AO
Threat overlays OSINT / intel feeds GeoJSON polygons Live — event-driven ~500KB per layer
Lens specs Pack configuration YAML files Versioned — update on pack publish ~5KB per spec
Coverage baselines Pre-computed from TLEs Coverage surface cache Derived — recompute on TLE update ~1MB per AO per 24h

TTL and Liveness

STATIC (no TTL, update manually):
  - Road graphs (OSM)
  - DEM tiles
  - Vehicle profiles

LIVE (TTL-based, auto-refresh when connected):
  - TLE catalog          → TTL: 12 hours (orbital elements drift)
  - Weather grids         → TTL: 6 hours (forecast horizon)
  - Threat overlays       → TTL: event-driven (new intel = new layer)
  - Coverage baselines    → TTL: derived (recompute when TLEs refresh)

VERSIONED (pack lifecycle):
  - Lens specs            → Published version, never mutated
  - Accountability packs  → Same

Staleness Contract

TTL is a manifest fact checked on read, not a background daemon — Phase 2 runs off-network where nothing can refresh anyway.

  • Each source_configs entry may carry ttl_seconds (omitted = STATIC). StagedSource records it and its own staged_at — a partial refresh moves only the refreshed sources' clocks. ManifestRecord.staged_at anchors legacy manifests whose sources carry no per-source stamp.
  • check_staleness(ao_root, now) -> tuple[StaleSource, ...] (lens/runtime/init.py) reports every staged source whose staged_at + ttl_seconds < now (now is an explicit required argument — callers own the clock). Pure read — no network, no mutation. Propagation through derived_from is transitive (a derived-of-derived chain reports stale end to end; cycles terminate).
  • refresh_dataspace(ao_root, resolver, now) re-stages only stale sources (connected, Phase 1/3): idempotent re-run of the staging step for those sources, previous artifact kept as <name>.prev (keep-1, per §cleanup), manifest rewritten — refreshed sources get new per-source staged_at, untouched entries are preserved verbatim.
  • An uninitialized dataspace (no manifest) yields no staleness facts; execute_lens logs the absence and embeds an empty list.
  • Phase 2 never blocks on staleness. execute_lens records check_staleness findings at the top level of each evidence artifact it writes (lens_run_*.json / coa_comparison_*.json, key stale_sources) and logs a warning — a commander routing on 13h-old TLEs gets the decision and the auditable fact that the data was stale. The envelope, not EvidenceBlock, carries it: the evidence model (component.prism.evidence-model) is complete and frozen-shape; staleness is an execution-context fact, not a computation fact. Refusing to compute off-network is the one unacceptable behavior.
  • Derived artifacts (coverage baselines) declare their input (derived_from: <source name>): stale input ⇒ derived artifact reported stale regardless of its own TTL.

Data Staging Location

Realized by lens/runtime/paths.py (Paths.from_ao_root); init_dataspace (lens/runtime/init.py) stages sources here and writes dataspace/manifest.json (per-source sha256, size_bytes, status: ready|incomplete).

{AO_ROOT}/
├── dataspace/
│   ├── manifest.json                  ← ManifestRecord: staged_at, per-source sha256/ttl
│   ├── graphs/
│   │   └── oregon_cascades.graph.json.gz  ← Pre-built RoadGraph (§graph-serialization-format)
│   ├── tle/
│   │   ├── planet.tle                 ← Planet optical constellation
│   │   ├── weather.tle                ← NOAA weather sats
│   │   ├── maxar.tle                  ← Maxar/WorldView (if available)
│   │   └── catalog_epoch.txt          ← When TLEs were last fetched
│   ├── dem/
│   │   └── srtm_N42W123.tif          ← DEM tiles for AO
│   ├── weather/
│   │   └── gfs_20260317_12z.json      ← Latest GFS forecast grid
│   ├── threat/
│   │   └── osint_overlay.geojson      ← Current threat polygons
│   └── coverage/
│       └── baseline_24h.json          ← Pre-computed coverage surface
├── packs/
│   ├── sat_avoidance_v2.yaml          ← Lens spec
│   ├── domain.yaml                    ← Domain model
│   └── accountability.yaml            ← Who attests
└── evidence/
    └── (empty — populated during Phase 2)

Init Script Contract

def init_dataspace(ao_config: dict) -> Dataspace:
    """Phase 1: Stage all data for an area of operations.

    ao_config:
        bbox: [min_lat, min_lon, max_lat, max_lon]
        road_types: ["motorway", "trunk", "primary", "secondary"]
        tle_sources: ["planet", "weather", "maxar"]
        dem_resolution: 30  # meters
        coverage_hours: 24

    Returns:
        Dataspace with all data staged and indexed.
    """

Graph Pre-Building

The 41MB OSM XML → graph conversion is the expensive step (~minutes). Do it once during init, serialize the graph:

# During init (connected, not time-critical)
graph = build_graph_from_osm_xml("oregon_cascades.osm")
graph.serialize("oregon_cascades.graph.json.gz")

# During crisis (off-network, fast)
graph = RoadGraph.deserialize("oregon_cascades.graph.json.gz")  # ≤1s per 100km² AO

Graph Serialization Format

RoadGraph.serialize()/deserialize() (lens/traversal/graph.py) use a versioned, gzipped JSON node-link envelope — stdlib only (json + gzip), no new dependency:

{
  "format": "prism.roadgraph.node-link",
  "format_version": 1,
  "node_fields": ["node_id", "lat", "lng"],
  "edge_fields": ["from", "to", "distance_m", "...all EdgeData fields"],
  "nodes": [[123, 44.1, -122.3]],
  "edges": [[123, 456, 80.0, "..."]]
}

Rows are columnar (field names once, in the header) — ~2× smaller than self-describing records and parses fast enough to hold the DoD ≤1s deserialize bound; deserialize rejects an envelope whose field layout doesn't match the running NodeData/EdgeData shape.

  • JSON, never pickle/msgpack — component.prism.serialization's DO-NOT bans pickle for cross-process artifacts; a staged .graph file ships across machines (init host → edge device) and a pickle load there is an arbitrary-code-execution surface. msgpack buys speed prism doesn't need at ~5MB but costs a dependency.
  • Deterministic output — nodes sorted by node_id, edges by (from, to) then insertion order; keys sorted. Byte-identical re-serialization of an unchanged graph keeps the manifest sha256 stable across re-stages.
  • Round-trip invariant — deserialize(serialize(g)) preserves node/edge counts, every NodeData/EdgeData field (including metadata), and external-id → index resolution (get_node, nearest_node answers unchanged).
  • Versioned envelope — format_version bumps on shape changes; deserialize rejects unknown format/format_version with a clear error, never guesses.
  • Edge direction — the envelope stores directed edges exactly as the rx.PyDiGraph holds them; bidirectional expansion happened at build time and is not re-applied on load.

Coverage Pre-Computation

Propagate all adversary satellites for 24h, compute coverage surface, cache it:

# During init or on TLE refresh
cov = coverage_surface(adversary_sats, bbox, now, now + 24h)
cache_coverage("baseline_24h.json", cov)

# During crisis — load cached, slice to time window
cov = load_coverage("baseline_24h.json", start=t0, end=t0+4h)

Phase 2: Execution (Off-Network, Crisis)

Trigger

Commander says: "Route Portland to Medford avoiding optical satellites, priority concealment."

LLM Role (Cortex)

The LLM configures, it does not compute:

  1. Select lens spec — picks sat_avoidance_v2.yaml from staged packs
  2. Set parameters — origin, destination, time window, cost weights
  3. Choose adversary sensors — which TLE files represent the threat
  4. Generate COA configs — 2-3 alternatives with different weight profiles:
  5. COA 1: FAST (weight: speed 0.8, exposure 0.1, concealment 0.1)
  6. COA 2: CONCEALED (weight: speed 0.2, exposure 0.6, concealment 0.2)
  7. COA 3: BALANCED (weight: speed 0.4, exposure 0.4, concealment 0.2)

Execution Flow

# 1. Load staged data (<1s)
graph = RoadGraph.deserialize("oregon_cascades.graph")
cov = load_coverage("baseline_24h.json", start=now, end=now+4h)

# 2. Annotate graph with coverage (<1s)
annotate_edges(graph, cov)

# 3. Run lens for each COA (<2s per COA)
for coa_config in coa_configs:
    result = run_lens(spec, inputs, layer_data)

# 4. Compare COAs (<1s)
comparison = compare_coas(spec, coa_configs, base_inputs)

# 5. Freeze evidence (<1ms)
# Evidence blocks created automatically by run_lens() and compare_coas()
# SHA-256 hashes lock the computation forever

# Total: <5 seconds for full COA comparison

What Gets Stored Locally

evidence/
├── lens_run_{uuid1}.json        ← COA 1 evidence block
├── lens_run_{uuid2}.json        ← COA 2 evidence block
├── lens_run_{uuid3}.json        ← COA 3 evidence block
├── coa_comparison_{uuid4}.json  ← Comparison evidence block
└── attestation_{uuid5}.json     ← Commander's decision (which COA, why)

Temporal Integration

The temporal lens answers: "When should we depart?"

layers:
  - name: satellite_gap
    weight: 0.50
    source: "lens://observation/adversary_coverage"
    cost_model: inverse_linear   # Gap = low coverage = low cost
    params: { reference: 100 }

  - name: weather_window
    weight: 0.30
    source: weather_forecast
    cost_model: threshold_gate   # Must have visibility > 1km
    params: { threshold: 1000, direction: above }

  - name: darkness
    weight: 0.20
    source: astronomical_twilight
    cost_model: categorical_match
    transform: extract_field
    transform_params: { field: is_dark }
    params: { field: is_dark, values: [true] }

Output: "Depart at 0230 PST — 47-minute satellite gap, no precipitation, astronomical darkness."

Phase 3: Sync (Reconnected)

Evidence Block → DES Block

When connectivity returns, evidence blocks sync to the platform:

Local EvidenceBlock                    DES Block (objects.yml)
─────────────────                      ────────────────────────
id                        →            block_id
block_kind: "lens_output" →            block_kind: "lens_output"
lens_id                   →            evidence_class
query_hash                →            query_hash (VERIFIED — same hash = same computation)
result_hash               →            result_hash
payload                   →            materialization
frozen: True              →            lifecycle_stage: "frozen"
provenance                →            provenance (embedded)
attestation               →            attestation (human decision)

Beacon Rendering

The evidence blocks become monitor page content:

Block Field Beacon Component What It Shows
payload.route GeoTemporalViewer Route on map (colored by cost)
payload.coverage GeoTemporalViewer Coverage heatmap layer
payload.coa_comparison EditionTab Side-by-side COA cards
attestation EditionTab Commander's decision + signature
query_hash EvidencePanel Proof of computation
provenance EvidencePanel When, where, which lens version

Cleanup

Evidence blocks are append-only (Invariant 2). They don't get deleted. The dataspace cleanup follows TTL:

Cleanup rules:
- Expired weather grids     → Delete after TTL (6h)
- Stale TLE files           → Replace on refresh (keep 1 previous)
- Old coverage baselines    → Replace on recompute (keep 1 previous)
- Road graphs               → Keep until AO changes
- DEM tiles                 → Keep indefinitely (static)
- Evidence blocks            → NEVER DELETE (append-only, sync to DES)
- Lens specs                → Keep all versions (versioned, immutable)

Assessment: Can This Flow to Beacon Today?

Component Ready? Gap
LensResult → JSON Yes to_dict() / to_json() works
EvidenceBlock → DES Block Schema aligned Need TASK-C01 to wire REST endpoint
Route → GeoTemporalMarker[] Shape matches Need layer_id + marker format adapter
Coverage → heat_map layer Shape matches Need MapLayerDef with data_source: "block"
COA comparison → Edition cards Conceptually aligned Need TASK-B01 (multi-edition view)
Attestation Model exists Need human UI in Beacon EditionTab

Bottom line: The data shapes are right. The evidence model aligns with DES blocks. What's missing is the plumbing — TASK-C01 (Cortex endpoint), TASK-B01 (Beacon multi-edition view), and a thin serialization adapter that maps LensResult → GeoTemporalMarker[] for the map layer.

Data Baseline for Demos

Every demo should reference this standard dataset:

Dataset File Source Used By
Oregon roads data/osm/oregon_cascades.osm Overpass API Traversal lens
Planet TLEs data/tle/planet.tle CelesTrak Observation lens
Weather TLEs data/tle/weather.tle CelesTrak Observation lens
Starlink TLEs data/tle/starlink.tle CelesTrak Observation lens
Station TLEs data/tle/stations.tle CelesTrak Observation lens
Demo subset data/tle/demo_subset.tle Curated Quick demos
Seattle OSM data/osm/seattle_metro.osm Overpass API Local traversal

(The VRS POC / entity-lens dataset is parallax's — the entity engine owns semantic resolution. A cost lens consumes that entity output via the correlation source, SPEC-13; it is not a prism dataset.)

Definition of Done

Item Done when
Graph serialization RoadGraph.serialize()/deserialize() round-trip preserves counts, all node/edge fields, and id resolution; unknown format_version rejected; output byte-deterministic; ≤1s deserialize per 100km² AO
Init flow init_dataspace stages a graph source through §graph-serialization-format (no pickle fallback for RoadGraph); manifest sha256 stable across unchanged re-stages
Staleness check_staleness flags an expired-TTL source and a derived artifact with a stale input; refresh_dataspace re-stages only stale sources and keeps one .prev
Phase 2 execute_lens on a stale dataspace computes anyway and writes stale_sources at the top level of each evidence artifact

DO NOT

  • Store raw PII in evidence blocks — use hashes and references
  • Delete evidence blocks — append-only (Invariant 2)
  • Compute during Phase 1 what can be pre-computed — graph building, coverage baselines
  • Require connectivity during Phase 2 — everything runs from cached data
  • Let the LLM compute scores — LLM configures, framework computes (Invariant 6)
  • Skip attestation — "no action" is a decision (Invariant 7)
  • Assume TLEs are static — orbital elements decay, 12h TTL minimum

Depends on: component.prism.cost-primitives, component.prism.evidence-model, component.prism.scoring-engine, component.prism.universal-lens-parser

Required by: component.prism.incremental-updates, component.prism.scenario-runner