Platform Integration & Scale

Status & scope

Scope: how the Prism lens library wires into the ADI platform, the layer-ownership boundaries that keep it pure, and how scale determines architecture. Forward-looking — the library is feature-complete; platform wiring follows the proven fusion path.
Milestone: Phase 3 (integration). The standalone library (Phases 1–2) is complete.

Purpose

A lens is a governed, versioned, federated computation primitive: a YAML spec (auditable, version-pinned), an input hash (reproducible), an output hash (tamper-evident SHA-256 evidence block — component.prism.evidence-model), a governance lifecycle (draft → review → published → versioned → retired), and a sharing model. This spec defines how that primitive plugs into the platform without taking on platform dependencies, following the same 7-step path Engine 1 (fusion) already took.

How a lens crosses an organizational boundary depends on cost, perishability, and sensitivity. Declared in lens YAML under sharing:.

Mode	When	What crosses the boundary	Example
`evidence_only`	Sensitive data, expensive compute	Frozen evidence block (hash + output)	HawkEye 360 coverage surface
`spec_shared`	Cheap compute, consumer has the data	Lens YAML spec (consumer runs locally)	Simple threshold model
`both`	Flexible — consumer chooses	Spec AND cached evidence available	Weather risk surface

lens:
  sharing:
    mode: evidence_only | spec_shared | both
    cache_ttl: 3600           # re-compute after N seconds
    sensitivity: high          # forces evidence_only for cross-org
    compute_cost: expensive    # hint for caching / pre-computation
    publish_evidence: true     # publish output as a reusable block

Layer Ownership

Four layers, clean separation. Prism stays pure compute with zero I/O.

CORTEX (MCP Gateway)  — thin wrapper; execute_lens_block delegates downward
        ↓
TITAN (Orchestrator)  — distributed execution, multi-node composition,
                        resolves lens:// across federates, Dask parallel
        ↓
ATHENA (Ops Layer)    — single-node orchestration; resolve sources → data → run_lens;
                        adapters (uds://, external://, pack://), caching, formatting
        ↓
PRISM (Lens Library)  — pure compute, zero I/O; run_lens(spec, layer_data) → LensResult;
                        frozen dataclasses, deterministic hashing, pip-installable

Layer	Owns	Does NOT own
Prism	Cost models, A* routing, SGP4 propagation, SIR model, threshold eval, evidence hashing	I/O, data fetching, caching, orchestration
Athena	Source resolution (adapters), single-lens execution, result formatting, caching	Multi-node composition, MCP protocol, UI
Titan	Dependency-graph execution, cross-node `lens://` composition, Dask parallel	Cost-model math, data-fetching details
Cortex	MCP tool wrapping, `create_block()`, tool registration	Business logic, data resolution, compute

Data flow

1.  Cortex receives MCP call: execute_lens_block(lens_id, inputs)
2.  Cortex delegates to athena.ops.lens.run(lens_id, inputs, context)
3.  Athena loads lens pack → LensSpec
4.  Athena resolves layer sources via adapters:
      uds://bio_case      → es_adapter.query()   → rows[]
      external://nasa/dem → http_adapter.fetch()  → DEM array (cached)
      pack://bases        → file_adapter.load()    → JSON
      lens://threat/bio   → titan.execute(lens_id) → LensResult
5.  Athena calls prism: run_lens(spec, inputs, layer_data={...})
6.  Prism computes: cost models → composite → threshold → evidence hash
7.  Prism returns LensResult (frozen dataclasses, pure data)
8.  Athena formats result for the cortex contract
9.  Cortex wraps in create_block() → evidence block with projections
10. Block flows to Beacon via the standard MCP result

Two data-source patterns resolve through lens/adapters/resolver.py (component.prism.source-adapters): MCP-tool references (mcp://elevation/srtm) for external reference data, and UDS adapters (uds://entity_name) for operational data already in federation. New URI schemes add without changing the orchestrator.

7-Step Integration Path

Engine 1 (fusion) is already integrated across the full stack (athena/libs/fusion/, athena/ops/fusion/, titan, cortex/tools/fusion.py, REST, ES indices, beacon). Prism (Engine 2) follows the exact same path — the pattern is proven, no architectural unknowns.

Step	What	Where	Effort	Depends on
B1	Copy lens library to athena	`athena/libs/lens/`	1 hour	—
B2	Lens ops wrappers	`athena/ops/lens/`	1 day	B1
B3	ES indices for lens results	`lens_result`, `lens_run` (fusion index pattern)	half day	—
B4	REST registers "lens" operation	`/api/lens/{op}` (fusion route pattern)	half day	B1
B5	Cortex `execute_lens_block` tool	generalize `cortex/tools/lens.py` to call `run_lens()`	1 day	B1, B3, B4
B6	Signal generation from thresholds	wire threshold → DES signal	half day	B5
B7	Beacon lens result component	render layers + COA comparison	1–2 days	B5

Total: ~5 days parallel, ~2 weeks sequential. After B5, cortex stops computing scores directly — execute_lens_block parses the pack's lens YAML, resolves layers via UDS, and calls run_lens(), so any pack's lens YAML executes without new cortex code (same ~15-line wrapper as execute_fusion). Task tickets for each step are tracked operationally (see docs/future/tasks/ in the repo history); the spec mandate is the step table and ownership above.

Scale: From Block to Planet

Scale determines whether a lens runs single-process or distributed. Same op contract throughout (component.prism.scoring-engine, component.prism.mobility-surface).

Tier	Area	Graph nodes	Threat cells	Satellites	Architecture
Block	1 km²	~500	100	—	Single process, prism direct
City	100 km²	~100K	10K	5–10	Single process
State	50K km²	~2M	500K	20–50	Single process + preprocessing (graph contraction); maybe Dask for threat surface
Country	1M km²	~50M	5M	100+	Tiled computation, partition by region; Dask tile-parallel
Region	10M km²	~200M	50M	500+	Fully distributed, multi-node (Titan orchestrates)
Hemisphere	100M km²	~1B	200M	2000+	Fully distributed, pre-compute everything
Planet	510M km²	~5B	1B	All catalogued	Hierarchical tiling + LOD, multi-resolution, Dask cluster

Partitioning strategies: spatial tiling (compute tiles, merge at boundaries), temporal partitioning (chunk long horizons, cache intermediate state), resolution scaling (coarse grid for orientation → fine grid for the route).

Current benchmarks & open scale questions

Validated standalone (representative): A* routing 98K nodes (Seattle OSM) <1s; MobilitySurface 5 layers / 4K cells <3s; ocean routing 9.5K nodes ~2s; SGP4 76 sats / 2h <1s; risk surface 5.4K cells <0.5s; COA comparison (3 runs) <2s; flood assessment (100 gauges + 3861 soils) <5s.

Open (must validate before the corresponding tier ships): - A* at ~2M nodes (State) — does rustworkx hold, or are contraction hierarchies needed? - Threat surface at ~5M cells (Country) — numpy may need Dask tiling. - SGP4 at the full 9,840-sat Starlink constellation — memory footprint of full propagation (76 sats tested).

Platform surfaces — the substrates a lens must touch

The 7-step path above wires the library in: copy lens to athena, wire ops, register the MCP tool, render in Beacon (companion doc: docs/INTEGRATION-PLAN.md). That is necessary but not sufficient. Prism is not integrated until it honours the platform's three governance substrates and one transport substrate. Crawl/walk/run (below) is defined by how many of these surfaces are wired, not by which features are built.

Surface	What it gives the lens	Why it's required
Dataspace (UDS)	ABAC-enforced layer resolution	Lens layers MUST resolve through UDS, never bypass. Invariant 1.
Userspace (Profiles + Packs)	Configuration without deploy, profile-gated visibility	Pack-driven UX. Profile decides which lenses a user may run, which `decision_types` they may attest.
Sentinel	Streaming triggers, incremental recomposition	A lens that only runs on demand is a report. Sentinel makes it a sensor. See component.prism.incremental-updates.
DES (Promotion boundary)	Threshold crossings → governed signals → Investigations → Editions	Without this, lens output is a number, not a decision. Invariant 7.
Xanadu	Cross-federation lens messaging	A lens that only runs in one node is a tool. Xanadu makes it federated.

Maturity: Crawl / Walk / Run

Three substrate-completeness tiers. Each tier's acceptance criterion is the demo that proves it end-to-end.

Crawl — single node, local pack, on-demand

Prove the contract end-to-end inside one federation node.

Surface	Required behaviour
Cortex MCP	`execute_lens_block` registered, delegates to athena.
Athena ops	`athena/ops/lens/run.py` — single-node execution.
UDS	Read-only adapter. Lens calls `uds://<entity>` → existing UDS query API (NO new ABAC code in Prism).
Userspace	Pack loaded from filesystem. No profile gating yet.
Sentinel	Not wired. Lens runs only on explicit MCP call.
DES	Not wired. Threshold crossings logged but do not promote.
Xanadu	Not wired. Single-node only.
Beacon	Renders block via existing geo-temporal viewer.

Acceptance: Houston flood lens runs end-to-end through Cortex → Athena → Prism, returns a frozen block, renders in Beacon. Already ~80% there per docs/INTEGRATION-PLAN.md.

Walk — single federation, full governance

Lens output becomes a governed decision artefact.

Surface	Required behaviour
UDS / Dataspace	(a) `deny_mode: omit` honoured on layer rows. (b) Fan-out + merge across federate nodes within ONE federation. (c) Layer resolver records ABAC scope per layer in the evidence block.
Userspace	(a) Pack loader resolves Profile → allowed lens IDs. (b) Accountability pack drives `generate_draft_editions` (one draft per `decision_type` from `lens.yaml output.decision_types`). (c) Deny lens execution if profile lacks scope.
Sentinel	Streaming triggers wired (component.prism.incremental-updates): `LayerChangeEvent` → `RecompositionEvent` → `ThresholdCrossingEvent`. Per-domain footprint resolvers registered.
DES	Promotion boundary honoured. Threshold crossing emits a `Cue` (parallax `specs/SPEC-32`, external repo) that becomes a Signal → InsightRoot → Investigation. Frozen lens evidence block referenced by InsightRoot.
Beacon	Lens-result panel renders composite + 4-layer breakdown + threshold bar. Investigation Edition tab shows draft Editions generated from lens `decision_types`.
Xanadu	Still not required if data lives in one federation.

Acceptance: A signal arrives, Sentinel triggers a lens recomputation against UDS-fan-out data with ABAC honoured, threshold crosses, an Investigation is created with frozen evidence, the accountability pack generates draft Editions (per Pacific Sentinel), commander attests one. End-to-end OBSERVE→DECIDE without engineer intervention.

Run — cross-federation

A lens spans federation boundaries. Xanadu is required here, not before.

Surface	Required behaviour
Xanadu	Carries wire-message families (parallax `specs/SPEC-32`) between federation peers over RabbitMQ. Three flows must work: (1) `evidence_only` — peer returns frozen block, never raw rows. (2) `spec_shared` — lens YAML ships to peer, peer runs against its own data, returns block. (3) `lens://` composition — Titan resolves cross-federate references via Xanadu.
DES	Cross-federate Cue/Signal promotion governed by per-federation policy. Evidence chain crosses the boundary as hash references, never as raw rows.
UDS	Federate-of-federates fan-out. Each peer enforces its own ABAC; results merge at the requesting node only after each peer has applied `deny_mode`.
Userspace	Pack governance lifecycle (draft → review → published → versioned → retired) replicates across federation peers. A retired lens stops running everywhere.
Sentinel	Triggers may cross federation boundaries via Xanadu (a layer change in peer A triggers recomputation of a lens published in peer B that consumes A's layer).

Acceptance: Scenario A 5-node demo — five Xanadu peers, each with a different data slice, run a composed lens that consumes layers from all five, returns one frozen evidence block at the requesting node. No raw rows cross any boundary; replay determinism verified.

Per-surface contract

UDS / Dataspace

In: layer URI (uds://<entity>), ABAC scope from caller context, query params from the lens layer config.
Out: rows projected by UDS, with deny_mode: omit already applied. Per-layer record of which fields were omitted (for the evidence block).
Prism never sees: ABAC predicates, user identity, federation topology.
New code in axonis-lens: zero. One adapter at lens/adapters/uds.py calling the existing UDS query op (component.prism.source-adapters).
New code in athena: none — athena/ops/uds/ already exists.

Userspace / Packs / Profiles

Pack contract: lens.yaml is a pack contribution, alongside domain.yaml, experiences.yaml, accountability.yaml in the bundle.
Profile contract: a profile's allowed_lens_ids: [...] controls visibility; its accountability_role decides which decision_types it may attest.
Accountability contract: accountability.yaml allowed_decision_types × lens.yaml output.decision_types = the set of draft Editions generated.
Open question: does a profile gate which layers of a lens it can see, or only whether the lens runs at all? Default: lens runs all-or-nothing; UDS gates rows. Layer-level gating is future scope.

Sentinel

No new contract here; see component.prism.incremental-updates. This spec only declares Sentinel is required for Walk.

DES promotion boundary

In: ThresholdCrossingEvent from Prism (lens_id, score, threshold, evidence_block_hash).
Out: Cue → Signal → InsightRoot → Investigation per the existing DES promotion boundary.
Promotion policy: lives in the pack (lens.yaml output.promotion) — auto, manual, suppress. Default manual (creates a Cue, requires triage to promote to Signal).
Frozen evidence: DES references the evidence block by hash. The block lives in the UDS-Server / evidence store, NOT in DES.

Xanadu

NOT a URL. It's the cross-node fabric for federated lens messaging, fronted by RabbitMQ in Scenario A.
Wire format: parallax specs/SPEC-32 message families (Capability / Cue / Fusion / Evidence / Operational), external repo.
What rides Xanadu for lens: (a) execute_lens_remote(lens_id, inputs) → Evidence family return; (b) publish_lens_spec(spec) for spec_shared mode → Capability family; (c) LayerChangeEvent cross-federate → Operational.
What does NOT ride Xanadu: raw entity rows. Ever. UDS rows stay in their home federation.

Open questions

Does Sentinel call Cortex or Athena directly for re-execution? (Affects MCP-tool re-entrancy.)
Should lens:// composition resolve in Athena (single-node) or Titan (multi-node)? Today: both, depending on scope. Codify.
evidence_only cache eviction: who owns TTL enforcement — Athena cache, UDS-Server evidence store, or pack-level cache_ttl?
Pack governance lifecycle replication across federation peers — Forge's job or Userspace's?
Xanadu schema registry (parallax specs/SPEC-32 follow-on) — required for Run, or nice-to-have?

DO NOT

Add platform dependencies (ES, RabbitMQ, Cortex, Beacon) to the Prism library — it stays pure compute, pip-installable.
Put data fetching, caching, or orchestration in run_lens() — those belong to Athena/Titan.
Re-implement the orchestrator in cortex — cortex is a thin wrapper around run_lens().
Create a new MCP tool per lens type — execute_lens_block handles all five.
Ship a scale tier before its open scale question is validated.

Depends on: component.prism.integration-test, component.prism.packaging, component.prism.scoring-engine, component.prism.source-adapters

Realizes: product.lens