Skip to content

Lens

Definition

A Lens (Semantic/Entity Lens) is a declarative, versioned configuration that fully specifies how to resolve the same real-world entity across independent federated parties — without exposing PII. It is the single contract a fusion run executes against: which fields identify an entity, how to normalize/derive them for the wire, which similarity metrics and weights drive matching, the blocking strategy, the evidence/provenance rules, the privacy policy envelope, and the output semantics. It is the "screen" through which each party's raw, heterogeneous local records are projected into one shared matching vocabulary. Because every party blocks and scores against an identical, coordinator-authoritative Lens, the comparison is apples-to-apples while raw data stays home.

Two representations exist: the compute-layer value object LensSpec (parallax/parallax/ops/fusion/models/lens_spec.py:103) used at runtime, and the governance lifecycle wrapper Lens (parallax/parallax/ops/fusion/models/governance.py:38) that tracks state. Authoring and persistence of the lens authority live in prism (prism/lens/userspace/fusion.py, class Lens(PrismModel)); parallax is the matching engine that consumes a parsed lens spec (stated at parallax/parallax/userspace/fusion.py:4).

Lifecycle

Governance state machine — LensStatus (governance.py:24): DRAFT (editable) → SUBMITTED (under review) → APPROVED (spec frozen) → ACTIVE (first execution has occurred) → RETIRED (cannot execute; existing correlations remain valid). The hard rule: once APPROVED, the spec is frozen — any change to a match function, weight, threshold, or policy envelope requires a new version (governance.py:45). Retirement is append-only (a new event, not a mutation). A parallel review lifecycle (LensReview, LensReviewStatus PENDING/APPROVED/REJECTED/ CHANGES_REQUESTED) carries a structured boolean checklist.

Journey through the code

  1. Authoring/persistence (prism) — Lens / LensBinding / LensReview UDS objects persisted to ES; lifecycle in parallax's governance.py. Lenses are authored as YAML (examples parallax/examples/*.yaml).
  2. Wire entry into parallax — POST /fusion/run (server/api/routes.py:112) → server.mcp.commands.start_run (commands.py:249), with the Lens supplied as lens_id + lens_path (YAML) or a pre-parsed spec.
  3. Parse + validate — ops/fusion/lens/parser.py:357 parse_lens builds the LensSpec; validate_lens runs the semantic checks V-01..V-10; a JSON-Schema gate at parallax/schemas/lens_spec.schema.json precedes it.
  4. Runtime application — ops/fusion/lens/runtime.py remaps each party's local fields into the lens vocabulary (_apply_field_map, _auto_field_map_from_synonyms) and builds the match function (_build_match_function).
  5. Binding to a federate's schema — LensBinding (models/binding.py:103) maps lens field paths to one federate's local fields (LLM-assisted discovery), exchanged on the wire via LENS_BINDING_REQUEST / RESPONSE.
  6. Federation — the lens spec travels as a dict in the FUSION_RUN_START payload so every node derives/blocks against the same contract; titan's FusionNodeHandler stores the lens_spec per run.

Data shape

LensSpec (lens_spec.py:103) — seven components. Required: lens_id, version, name, scope (object types, time window, federation), field_groupings (identity_hints[] of FieldHint, optional context/suppressed fields), identity_fusion (match_function[] of MatchField, initial_threshold, confirmation_threshold, optional blocking_strategy, target_model), evidence_rules (min_independent_sources, max_age, provenance_required, confidence_decay), policy_envelope (access, field suppression, audit level, federation boundaries, result classification). Optional: domain, description, output_semantics (correlation type, signal severity/type, display fields, evidence class), relationship_patterns. A FieldHint carries field, transform, label, transform params, and mapping hints; v2 lenses split the federation contract into normalize: / derive: / score:. Storage: governance objects persisted by prism to ES; lens definitions authored as YAML.

Invariants

The validator enforces (parser.py:398): V-01 match-function weights sum to 1.0 (±0.001); V-02 initial_threshold < confirmation_threshold; V-03 thresholds ∈ [0, 1]; V-04 match fields reference declared identity hints; V-05 a field cannot be both an identity hint and suppressed; V-06 min_independent_sources ≥ 2; V-07 blocking keys reference declared hint fields; V-08/V-09 metrics/transforms from supported sets; V-10 lens_id matches ^[a-z][a-z0-9_]*$. Plus: frozen-after-approval (semantic change ⇒ new version), and suppressed fields are defense-in-depth — suppressed at extraction and in the policy envelope.

  • product.fusion — Fusion executes against a Lens; every FusionRun stamps lens_id + lens_version, and the Lens defines the entire matching contract.
  • product.trained-model — identity_fusion.target_model and the v2 scoring path can reference titan-trained ranker/LLM models by registry key.
  • product.pack — lens.yaml is a part of a domain pack.

Open questions

  • Ownership split — prism owns Lens authoring/persistence while parallax holds the compute-side governance value object; the spec should state which is canonical for which purpose.
  • LensConfig / IdentityHint (models/fusion.py:249) is a thin lens shape used only for the extract_vectors op param — distinct from the full LensSpec; confirm intent.
  • relationship_patterns is parsed but its consumer (graph traversal) is not visible in parallax — likely downstream in cortex.

Realized by: component.parallax.feature-extraction, component.parallax.fusion-governance-lifecycle, component.parallax.lens-parser, component.parallax.service-interface, component.prism.bayesian-posterior-engine, component.prism.cost-lens-governance, component.prism.cost-primitives, component.prism.evidence-model, component.prism.incremental-updates, component.prism.lens-families, component.prism.platform-integration, component.prism.rendering-architecture, component.prism.scoring-engine, component.prism.service-interface, component.prism.universal-lens-parser