Fusion Governance Lifecycle & Operational Execution

Status & scope

Stage: DRAFT — Architecture Decision
Milestone: M4 (Platform Integration)

Note: The consent model and FusionRun audit trail are now fully specified in: - component.parallax.three-phase-protocol: Three-Phase Federation Protocol §6 Consent Model, §7 FusionRun Audit Trail - component.parallax.derived-features: Derived Features & Privacy Boundary — privacy guarantees for derived data

This spec covers the broader governance lifecycle (CREATE → ATTEST) which spans cortex/beacon/titan layers. The parallax-level consent and audit implementations are in component.parallax.three-phase-protocol.

Ownership (two-engine model — product.lens / SPEC-LENS + spec-seam-ledger §F1, architect-confirmed 2026-06-03). parallax is the entity-lens engine and owns the entity-lens governance implementation end-to-end: the LensStatus / LensReview / Participation / LensBinding ES persistence and the transition state machines (LENS_TRANSITIONS / PARTICIPATION_TRANSITIONS / BINDING_TRANSITIONS, separation-of-duties, frozen-after-approval semver bump, consent-scope validation, model-first auto-approve). This subsystem is ported into parallax from prism (prism/lens/governance/ + prism/lens/userspace/fusion.py) — prism built it as entity-lens over-reach and retires it under slug 2026-06-03-lens-framework-ownership. Cortex MCP tools and the Beacon Fusion Manager are the management / UI surface over these parallax-owned objects, not a separate governance owner. prism keeps only its own cost-lens governance (full | lightweight | none); the two governance models stay per-engine (dedup via axonis-core only, if ever — parallax SPEC-27).

Problem

parallax solves how to run fusion — the compute layer is complete with 944 passing tests and F1=0.973 on VRS POC (0.967 on Febrl4). component.parallax.fusion-binding defines how data flows into the fusion engine. component.parallax.correlation-persistence defines where results persist.

What's missing is everything before and around execution:

Who creates a lens? A lens defines what data gets compared across organizational boundaries. You can't let anyone create one and run it unchecked.
Who approves it? A lens touching PII across federates needs governance review — are suppressed_fields correct? Is the policy_envelope right? Is the scope appropriate?
What federation does it run in? Which nodes are invited to participate?
Which nodes consent? Each federate must explicitly opt in. "I consent to share soundex(name) + year(dob) for blocking."
How does it run operationally? Not just ad-hoc — streaming on new data, scheduled batch sweeps, reactive to signals from other domains.
What happens with results? Correlations aren't just data points. They're intelligence that drives signals, investigations, and decisions.

The compute is done. The governance and operational layer is the product gap.

Decision

The lens lifecycle has 8 stages, and parallax — the entity-lens engine — owns the implementation of all of them. The LensStatus / LensReview / Participation / LensBinding objects, their ES persistence, and the transition logic that guards each stage live in parallax. ES objects, Cortex MCP tools, and the Beacon Fusion Manager page (driven by experience packs) are the management / UI surface over these parallax-owned objects — they expose and orchestrate the governance, they do not own it.

CREATE → REVIEW → APPROVE → PUBLISH → CONSENT → BIND → EXECUTE → ATTEST
  │         │        │          │          │        │        │         │
  │         │        │          │          │        │     parallax    Invariant 6
  │         │        │          │          │     component.parallax.fusion-binding
  │         │        │          │       Consent model
  │         │        │       Federation topology (UDS)
  │         │     Edition pattern (Invariant 6)
  │      Governance review
  Lens authoring (form or YAML editor)

All eight stages' governance objects — and the transitions between them — are owned and persisted by parallax; the per-stage annotations above name each stage's conceptual driver, not a separate owner. There is no long-running workflow engine and no auto-routing: each transition is an explicit, audited, append-only operation. But it is guarded by a state machine in parallax (LENS_TRANSITIONS / PARTICIPATION_TRANSITIONS / BINDING_TRANSITIONS) that validates the move and enforces separation-of-duties before the new record is written. Cortex MCP tools expose each transition as an independent tool and are the manager; the lens is the configuration; the experience pack decides what shows on the page. The transition logic and ES persistence those tools call live in parallax (ported from prism's lens/governance/ + lens/userspace/fusion.py, slug 2026-06-03-lens-framework-ownership).

The 8 Governance Objects

Object 1: Lens

The lens definition itself. Extends the existing LensSpec with lifecycle metadata.

Index: fusion_lenses
Lifecycle: draft → submitted → approved → active → retired

Field	Type	Description
lens_id	string	Unique identifier (e.g., `vrs_vulnerability_v1`)
version	string	Semantic version (immutable once any result references it)
status	enum	`draft`, `submitted`, `approved`, `active`, `retired`
spec	object	The full LensSpec (component.parallax.lens-parser schema)
created_by	string	Author identity
created_at	datetime	Creation timestamp
submitted_at	datetime	When submitted for review (null if draft)
approved_at	datetime	When governance approved (null if not yet)
activated_at	datetime	When first execution occurred
retired_at	datetime	When retired (null if active)
retirement_reason	string	Why retired (null if active)

Immutability rule: Once a lens reaches approved, the spec field is frozen. Any change to the match function, weights, thresholds, or policy envelope requires a new version. Draft lenses can be edited freely.

Retirement: A retired lens cannot be executed. Existing correlation records produced by it remain valid — they reference the frozen version. Retirement is append-only: a new event, not an update to the original.

Object 2: LensReview

Governance review of a submitted lens. Follows the Edition pattern (Invariant 6: AI assists, humans attest).

Index: fusion_lens_reviews
Lifecycle: pending → approved | rejected | changes_requested

Field	Type	Description
review_id	string	Unique review identifier
lens_id	string	Which lens is being reviewed
lens_version	string	Which version
reviewer	string	Governance officer identity
status	enum	`pending`, `approved`, `rejected`, `changes_requested`
review_notes	string	Reviewer's assessment
checklist	object	Structured review (see below)
created_at	datetime	When review was assigned
completed_at	datetime	When decision was made

Review checklist:

checklist:
  scope_appropriate: true        # Is the scope (object types, time window, federation) right?
  suppression_verified: true     # Are suppressed_fields correctly configured?
  policy_envelope_valid: true    # Access level, audit level, classification correct?
  thresholds_justified: true     # Are initial/confirmation thresholds appropriate?
  metrics_appropriate: true      # Do the metrics match the data types?
  weights_balanced: true         # Are field weights reasonable for the domain?
  evidence_rules_sound: true     # Min sources, max age, provenance requirements?
  output_semantics_safe: true    # Signal severity, correlation type, display fields appropriate?

Why not reuse Edition directly? Editions are for decision outcomes in investigations. A lens review is a governance decision on a configuration artifact, not an investigation finding. Different object, same pattern (human reviews, decides, decision is frozen).

Object 3: Federation

Federations already exist in UDS. No new object needed — this spec references them as-is. The Fusion Manager page queries UDS for federation topology.

Existing UDS Data	Fusion Manager Use
Federation members	Which nodes can be invited to a lens
Federation health	Which nodes are online/offline
Federation ABAC	What data each node can see

Object 4: Participation

A node's consent to participate in a specific lens within a specific federation. This is the consent model from parallax/ops/fusion/models/consent.py persisted to ES.

Index: fusion_participation
Lifecycle: invited → consented → active → suspended → revoked

Field	Type	Description
participation_id	string	Unique identifier
lens_id	string	Which lens
lens_version	string	Which version
federation_id	string	Which federation
federate_id	string	Which node
status	enum	`invited`, `consented`, `active`, `suspended`, `revoked`
consent_scope	object	What the node consented to share (see below)
invited_by	string	Who sent the invitation
invited_at	datetime	When invited
consented_at	datetime	When node administrator consented
activated_at	datetime	When binding was validated and participation became active
suspended_at	datetime	When temporarily paused (null if active)
revoked_at	datetime	When permanently withdrawn (null if active)
revocation_reason	string	Why revoked

Consent scope:

consent_scope:
  blocking_fields:               # What the node shares for blocking
    - field: full_name
      transform: soundex         # "I share soundex of name, not the name itself"
    - field: dob
      transform: year_only       # "I share birth year, not full DOB"
  scoring_fields:                # What the node shares for scoring (feature vectors)
    - field: full_name
      metric: jaro_winkler       # "I allow jaro_winkler comparison of names"
    - field: dob
      metric: exact              # "I allow exact comparison of DOB"
    - field: postcode
      metric: geo_prefix         # "I allow prefix comparison of postcode"
  suppressed_fields:             # What the node will NOT share under any circumstances
    - vulnerability_status
    - case_notes

Consent is granular, not binary. A node doesn't just say "yes, I participate." It says "I consent to share these specific derived values for these specific purposes." This maps directly to ConsentPolicy in parallax's consent model.

Revocation is immediate and append-only. When a node revokes, a new event is created. The node's data is excluded from future fusion runs. Existing correlation records that reference the node's data remain valid (they reference a specific fusion run at a specific point in time) but are flagged as stale (one contributing source revoked).

Object 5: Binding

The per-node schema mapping. Already specified in component.parallax.fusion-binding as LensBinding. The governance lifecycle adds review status tracking.

Index: fusion_bindings (from component.parallax.fusion-binding)
Lifecycle: draft → validated → review → approved → active → invalidated

component.parallax.fusion-binding fully defines the binding format. This spec adds only the governance wrapper:

Additional Field	Type	Description
participation_id	string	Links to the node's consent record
validation_result	object	Schema validation output (field coverage, type mismatches)
review_status	enum	From component.parallax.fusion-binding: `pending`, `approved`, `rejected`, `auto_approved`
reviewed_by	string	Who approved the binding

Auto-approval: When the federate's local_model matches the lens target_model AND all field confidences ≥ 0.90, the binding is auto-approved. This is the model-first resolution from component.parallax.fusion-binding — covers ~80% of nodes at scale.

Object 6: LensActivation

This is the new object. It bridges "lens exists" and "fusion runs." Defines when and how a lens executes, and what happens with results.

Index: fusion_activations
Lifecycle: configured → active → paused → retired

Field	Type	Description
activation_id	string	Unique identifier
lens_id	string	Which lens
lens_version	string	Which version
federation_id	string	Which federation
mode	enum	`streaming`, `adhoc`, `scheduled`, `reactive`
trigger	object	What fires execution (see modes below)
execution_config	object	Scope, threshold overrides, blocking strategy
on_match	object	Signal generation rules
status	enum	`configured`, `active`, `paused`, `retired`
created_by	string	Who configured the activation
created_at	datetime	When configured
last_run_at	datetime	When last executed

Operational counters (run_count, match_count) are not stored on the activation object. They are computed as aggregations over the fusion_runs index filtered by activation_id. This avoids mutable counters on what is otherwise an append-only model. The Fusion Manager UI queries these aggregations for display.

Object 7: FusionRun

Execution instance. One run per activation trigger.

Index: fusion_runs
Lifecycle: queued → running → complete | failed

Field	Type	Description
run_id	string	Unique run identifier
activation_id	string	Which activation triggered this
lens_id	string	Lens used
lens_version	string	Lens version
status	enum	`queued`, `running`, `complete`, `failed`
started_at	datetime	Execution start
completed_at	datetime	Execution end
input_record_count	integer	Total records across all federates
candidate_pair_count	integer	Pairs generated by blocking
match_count	integer	Pairs above threshold
cluster_count	integer	Entity clusters formed
false_positive_rate	float	nullable — only meaningful when ground truth is available (primarily dev/test)
timing	object	Per-phase timing (blocking, scoring, clustering)
error	string	Error message if failed
participating_federates	list	Which nodes contributed data

Object 8: CorrelationRecord

Already specified in component.parallax.correlation-persistence and implemented in parallax/ops/fusion/models/correlation.py. The governance layer adds signal generation.

component.parallax.correlation-persistence fully defines the correlation persistence model. This spec adds:

Additional Field	Type	Description
signal_id	string	If this correlation generated a signal (null if no signal)
signal_severity	enum	From lens `output_semantics.signal_severity`
attested_by	string	Human who confirmed/rejected (null if pending)

The Four Execution Modes

All four modes use the same parallax compute underneath. The difference is orchestration — what triggers the run, what scope of data, and what happens with results.

Mode 1: Streaming (Real-Time Matching)

Trigger: Data ingest event — a new record arrives in an index watched by an active lens.

Use case: Match every new record as it arrives. "New patient registered — instantly check against all federates."

activation:
  lens_id: vrs_vulnerability_v1
  mode: streaming
  trigger:
    type: ingest
    index_pattern: social_care_*
    filter:
      type: person
  execution_config:
    scope: new_record_vs_all         # match new record against existing blocking keys
    blocking: incremental            # compute blocking key for 1 record, query ES for matches
    threshold: 0.70
  on_match:
    signal_type: fusion_vulnerability_match
    severity: high
    route_to: safeguarding_team

Execution flow:

1. New record arrives in social_care_* index
2. ES watcher / titan message fires trigger
3. Extract features from the new record (parallax)
4. Compute blocking keys for the new record (parallax)
5. Query ES for existing records with matching blocking keys (component.parallax.correlation-persistence acceleration)
6. Score candidate pairs (parallax)
7. If match found → create CorrelationRecord + emit signal
8. Signal appears on analyst's Monitor page

Performance: Sub-second for a single record against millions. You don't re-block the whole dataset — you compute the new record's blocking key, query ES for that key, and score only those candidates.

What parallax needs: score_one_vs_many() — score a single record against a set of candidates. Currently run_fusion() does batch. This is the incremental variant.

Mode 2: Ad-Hoc (Investigation)

Trigger: Analyst action — "Find this person across all participating nodes."

Use case: Investigation query. An analyst has a person of interest and wants to check if they appear in other federates.

activation:
  lens_id: vrs_investigation_v1
  mode: adhoc
  trigger:
    type: manual
    input: single_record             # analyst provides the search entity
  execution_config:
    scope: query_vs_all              # search entity against all federates
    blocking: compute_for_query      # blocking keys from the search entity only
    threshold: 0.60                  # lower threshold for investigation (surface more candidates)
  on_match:
    signal_type: fusion_investigation_hit
    severity: medium
    present_to: requesting_analyst   # results go back to the analyst who searched

Governance question: Does every ad-hoc search need individual approval? No — the governance is on the lens, not each individual search. A pre-approved investigation lens (e.g., vrs_investigation_v1) has already been through the REVIEW → APPROVE cycle. The analyst is authorized to use it by the accountability pack. Each search is logged as a FusionRun with full audit trail.

Mode 3: Scheduled (Batch Pattern Detection)

Trigger: Cron schedule — periodic sweep for patterns.

Use case: "Every Monday, find all new duplicate benefits claims across councils."

activation:
  lens_id: benefits_fraud_detection_v2
  mode: scheduled
  trigger:
    type: cron
    schedule: "0 2 * * MON"           # Every Monday at 2am
  execution_config:
    scope: incremental_since_last     # only records modified since last run
    blocking: incremental             # only recompute blocks for changed records
    threshold: 0.65
    lookback: 7d                      # records modified in last 7 days
  on_match:
    signal_type: fusion_fraud_pattern
    severity: medium
    aggregate: true                   # group related matches into one signal
    route_to: fraud_investigation_team

Execution flow:

1. Cron trigger fires (titan scheduler or ES watcher)
2. Query each federate for records modified since last_run_at
3. Compute blocking keys for changed records (parallax)
4. Cross-reference against existing blocking index (component.parallax.correlation-persistence)
5. Score all new candidate pairs (parallax)
6. Cluster matches (parallax)
7. Create CorrelationRecords for new matches
8. Generate aggregated signal: "47 new potential duplicates found this week"
9. Signal routes to fraud_investigation_team Monitor page

Incremental vs full sweep: Most scheduled runs should be incremental (only process changes since last run). Full sweep is available for initial backfill or periodic re-baseline (e.g., monthly full recalculation).

Mode 4: Reactive (Signal-Triggered Fusion)

Trigger: A signal from one domain triggers fusion in another.

Use case: AML system flags suspicious transaction → fusion runs the entity against PEP lists → if match, critical signal.

activation:
  lens_id: pep_screening_v1
  mode: reactive
  trigger:
    type: signal
    signal_type: aml_suspicious_transaction
    extract_entity: signal.entity_ref  # take this entity from the triggering signal
  execution_config:
    scope: entity_vs_dataset           # match extracted entity against PEP dataset
    dataset: politically_exposed_persons
    threshold: 0.80
  on_match:
    signal_type: pep_match
    severity: critical
    escalate: true                     # immediate escalation

Signal chains: Fusion produces signals. Signals can trigger more fusion. This creates intelligence processing chains — but NOT cycles. Each activation has a defined signal_type trigger and a defined output signal_type. The platform prevents cycles at configuration time (a lens cannot be triggered by the signal type it produces).

The Signal Loop

This is the key architectural insight: fusion produces signals. A correlation IS a signal — "We found the same entity appearing in two federates." The signal feeds back into the platform's existing Monitor → Investigate → Decide workflow.

Data ingested into ES
        ↓
Activation trigger fires (ingest / cron / signal / manual)
        ↓
"Records match an active lens scope"
        ↓
Run fusion (parallax compute)
        ↓
Correlations found?
    ├── Yes → Create CorrelationRecord (component.parallax.correlation-persistence)
    │         ↓
    │   Generate signal (from activation.on_match)
    │         ↓
    │   Signal appears on Monitor page
    │         ↓
    │   Analyst investigates → attests correlation → Edition
    │
    └── No match → log FusionRun with match_count=0, move on

on_match configuration:

Field	Type	Description
signal_type	string	Type of signal to generate (maps to experience pack)
severity	enum	`low`, `medium`, `high`, `critical`
route_to	string	Team or role that receives the signal
aggregate	boolean	Group related matches into one signal (for batch modes)
escalate	boolean	Immediate escalation (bypasses normal triage)
present_to	string	Specific user (for ad-hoc mode)
decision_template	string	Which accountability pack template for attestation

The Fusion Manager Page

A Beacon page driven by experience pack configuration. No intelligence in Beacon — it renders what Cortex returns (ADL-002).

Experience Pack Configuration

fusion_manager:
  page_type: fusion
  title: Fusion Manager
  tabs:
    - id: lenses
      label: Lenses
      view: lens_table
      columns: [lens_id, version, status, domain, created_by, created_at]
      actions: [create, edit, submit_for_review, retire]
      filters: [status, domain]

    - id: reviews
      label: Reviews
      view: review_queue
      columns: [lens_id, version, reviewer, status, submitted_at]
      actions: [approve, reject, request_changes]
      filters: [status, reviewer]

    - id: activations
      label: Activations
      view: activation_table
      columns: [lens_id, mode, status, last_run_at]
      computed_columns: [run_count, match_count]  # aggregated from fusion_runs index
      actions: [configure, activate, pause, retire]
      filters: [mode, status, lens_id]

    - id: participation
      label: Participation
      view: participation_matrix
      rows: federates
      columns: active_lenses
      cell: consent_status
      actions: [invite, view_consent]
      filters: [federation, status]

    - id: runs
      label: Runs
      view: run_history
      columns: [run_id, lens_id, mode, status, started_at, input_records, matches, duration]
      actions: [view_details, rerun]
      filters: [lens_id, status, date_range]

    - id: correlations
      label: Correlations
      view: attestation_queue
      columns: [correlation_id, lens_id, confidence, accuracy, status, contributing_records]
      actions: [attest_confirm, attest_reject, attest_defer, investigate]
      filters: [status, lens_id, confidence_range]

    - id: analytics
      label: Analytics
      view: fusion_dashboard
      widgets:
        - type: kpi
          metrics: [total_correlations, pending_attestation, f1_score, avg_confidence]
        - type: timeseries
          metric: matches_over_time
          group_by: lens_id
        - type: bar
          metric: false_positive_rate
          group_by: lens_id

Tab Descriptions

Lenses — All lens definitions with their lifecycle status. Author, submit for review, track approval, retire. The "Create" action opens a form-based lens editor (simple) or YAML editor (advanced).

Reviews — Governance queue. Lenses awaiting review with the structured checklist. Only users with the governance role (from accountability pack) can approve.

Activations — Operational control plane. How each lens runs. Configure streaming triggers, cron schedules, reactive signal chains. Activate, pause, monitor health.

Participation — Consent matrix. Rows = federates, columns = active lenses. Each cell shows consent status (invited/consented/active/revoked). Drill into consent scope details.

Runs — Execution history. Every fusion run with timing, counts, participating nodes. Filter by lens, status, date range. Click through to see per-pair results.

Correlations — Attestation queue. Match results awaiting human decision. Confidence score, STANAG accuracy/credibility, contributing records. Attest (confirm, reject, defer) or open investigation.

Analytics — Operational metrics. Total correlations over time, pending attestation backlog, F1 score (when ground truth available), false positive rate, processing latency. Per-lens breakdown.

MCP Tools

Cortex tools that expose the governance lifecycle as the management layer — no standalone "fusion manager service." The transition logic and ES persistence they invoke are owned by parallax (the entity-lens engine: the LensStatus / LensReview / Participation / LensBinding state machines + persistence ported from prism); these tools are the interface over parallax-owned objects, not the governance owner.

Lens Management

Tool	Action	Input	Output
`create_lens`	Create draft lens	LensSpec YAML or form data	lens_id, version
`update_lens`	Edit draft lens	lens_id, updated fields	updated LensSpec
`submit_lens_for_review`	Submit draft → submitted	lens_id, note	updated lens (submitted)
`review_lens`	Approve/reject lens	lens_id, decision, note, checklist	updated status
`activate_lens`	Submitted → active (after approval)	lens_id	confirmation
`retire_lens`	Active → retired	lens_id, reason	confirmation
`revise_lens`	Revise approved/active lens → new draft version (§Object 1: any change is a new version; parent-linked)	lens_id	new lens_id, version
`list_lenses`	Query lenses	filters (status, domain, author)	Lens[]
`get_lens`	Get single lens with full history	lens_id	Lens + reviews + runs

Participation & Binding

Tool	Action	Input	Output
`invite_node`	Invite federate to lens	lens_id, federate_id, federation_id	participation_id
`record_consent`	Node consents with scope	participation_id, consent_scope	updated status
`revoke_participation`	Node withdraws	participation_id, reason	updated status
`create_binding`	Create schema binding	participation_id, binding config	binding_id
`validate_binding`	Validate field coverage (auto-approves on §Object 5 joint condition)	binding_id	validation_result
`submit_binding_for_review`	Validated → review (non-auto-approval path)	binding_id	updated status
`approve_binding`	Approve binding under review (review → approved)	binding_id	updated status
`reject_binding`	Reject binding under review (review → draft)	binding_id, reason	updated status
`list_participation`	Query participation	filters (lens, federation, status)	Participation[]
`get_participation`	Get single participation record	participation_id	Participation
`list_bindings`	Query bindings (Fusion Manager Bindings tab)	filters (lens, federate, status)	Binding[]
`get_binding`	Get single binding record	binding_id	Binding

Execution

Tool	Action	Input	Output
`configure_activation`	Set up execution mode	lens_id, mode, trigger, on_match	activation_id
`activate`	Start activation	activation_id	confirmation
`pause_activation`	Pause	activation_id	confirmation
`retire_activation`	Retire (terminal)	activation_id, reason	confirmation
`execute_fusion`	Trigger manual/ad-hoc run	activation_id (or lens_id + query)	run_id
`get_run_status`	Check run progress	run_id	FusionRun status
`list_runs`	Query run history	filters (lens, status, date)	FusionRun[]

Correlation & Attestation

Tool	Action	Input	Output
`list_correlations`	Query correlations	filters (lens, status, confidence)	CorrelationRecord[]
`get_correlation`	Get single CR with lineage	correlation_id	CorrelationRecord + lineage
`attest_correlation`	Human decision	correlation_id, decision (confirm/reject/defer)	updated status
`decay_correlations`	Apply time decay	lens_id, decay_config	count of decayed CRs

What parallax Needs

Most of the compute is done. What's needed for operational modes:

Capability	Status	What's Needed
Batch fusion	Done	`run_fusion()`, `run_multi_fusion()` — existing
Score one vs many	Not done	`score_one_vs_many(query_record, candidates, lens)` for streaming/ad-hoc
Incremental blocking	Not done	Compute blocking key for 1 record, match against existing keys
Lookback window	Not done	Filter input records by modification date
Activation config model	Not done	`LensActivation` dataclass for execution mode + trigger + on_match
Signal generation	Not in parallax	Cortex concern — parallax returns correlations, Cortex emits signals
Decay computation	Done	`apply_decay()` in correlation model — already implemented
Consent enforcement	Done	`check_record_consent()`, `filter_by_consent()` — already implemented
Passport TTL	Done	`check_passport_expiry()`, `apply_passport_expiry()` — already implemented

Heavy compute doesn't change. Metrics, scoring, clustering work as-is. It's the input adapter (one record vs batch) and orchestration wrapper (trigger → scope → compute → signal) that need the four execution modes.

New parallax Interfaces

def score_one_vs_many(
    query_record: dict,
    candidates: list[dict],
    lens: LensSpec,
    binding: LensBinding | None = None,
    consent_policies: list[ConsentPolicy] | None = None,
) -> list[ScoredPair]:
    """Score a single query record against a list of candidates.

    Used by streaming and ad-hoc modes. Computes blocking keys for
    the query record, filters candidates by blocking key match,
    then scores remaining pairs.

    If consent_policies are provided, candidates are filtered through
    filter_by_consent() before scoring — same enforcement as batch
    mode's run_fusion(). Consent enforcement is the caller's
    responsibility in batch mode but must be explicit here because
    streaming callers may not have the full pipeline context.

    Returns scored pairs above lens initial_threshold, sorted by
    confidence descending.
    """

def incremental_blocking_keys(
    record: dict,
    lens: LensSpec,
    binding: LensBinding | None = None,
) -> list[str]:
    """Compute blocking keys for a single record.

    Returns the set of blocking keys that can be used to query
    ES for existing candidate records. Used by streaming mode
    to avoid re-blocking the entire dataset.
    """

CorrelationRecord Decay

The CorrelationRecord lifecycle includes decayed — the first time-dependent state transition in the object model.

Decay is NOT automatic. It is triggered by a scheduled activation or a manual MCP tool call. The decay_correlations tool applies time-based rules from the lens's evidence_rules:

evidence_rules:
  confidence_decay:
    strategy: exponential     # default — matches calculate_decayed_confidence() implementation (0.5^(days/half_life))
    half_life: 90d            # confidence halved after 90 days without re-confirmation
    min_confidence: 0.30      # below this, status → decayed
    reconfirmation_resets: true  # a new fusion run that re-confirms resets the decay clock

Note: strategy: exponential is the only currently implemented strategy in parallax's calculate_decayed_confidence(). Future strategies (linear, step) can be added but are not required for M4.

Why not automatic? Invariant 7 ("no action" is a decision). Decay should be observable and auditable. A scheduled activation (e.g., weekly decay sweep) creates a FusionRun record for the decay operation, with full audit trail of which correlations were affected.

Implementation: parallax already has apply_decay() in the correlation model. The platform schedules the call via a mode: scheduled activation with trigger.type: cron.

Correlation Compaction

At production scale with streaming mode, fusion_correlations can accumulate millions of records — most of them DECAYED or REJECTED. These are dead weight in the hot index: DECAYED CRs don't participate in blocking acceleration (component.parallax.correlation-persistence) and can't be attested. The index needs a compaction strategy.

Compaction is NOT deletion. It moves records from the hot index to a cold archive index. Invariant 2 (append-only) is preserved — the record still exists, just in fusion_correlations_archive. The compaction run itself creates a FusionRun record, maintaining full audit trail.

compaction:
  strategy: archive_after
  decayed_archive_days: 180       # DECAYED CRs older than 180d → archive
  rejected_archive_days: 365      # REJECTED CRs kept longer (negative evidence)
  retain_lineage: true            # archive includes full lineage (audit trail)
  retain_blocking_keys: false     # blocking keys not needed in archive
  trigger: scheduled              # cron activation, same pattern as decay

Why REJECTED CRs persist longer: check_negative_correlation() uses rejected CRs to block known false positives (e.g., two different John Smiths that were manually rejected should not be re-proposed). Archiving too early loses this negative evidence. 365 days gives sufficient coverage for most scheduled re-runs.

Archive index: fusion_correlations_archive has the same mapping as fusion_correlations but uses a cold storage tier. Queries that need historical audit can search both indices. Normal operational queries (attestation queue, blocking acceleration) search only the hot index.

MCP tool:

Tool	Action	Input	Output
`compact_correlations`	Archive old DECAYED/REJECTED CRs	lens_id, compaction config	count archived, archive index name

Correlation Reconfirmation

When a scheduled or streaming run re-matches a pair that already has a PROPOSED or CONFIRMED CorrelationRecord, the system must handle reconfirmation — not create a duplicate CR.

Reconfirmation flow:

Fusion run scores pair (A, B) at 0.85
    ↓
Check: does a CR already exist for this pair + lens?
    ├── No existing CR → create new CR (normal path)
    └── Existing CR found
        ├── Status: PROPOSED → reconfirm (update confidence, reset decay clock)
        ├── Status: CONFIRMED → reconfirm (update confidence, reset decay clock)
        ├── Status: REJECTED → skip (human already rejected this pair)
        ├── Status: DECAYED → reconfirm (resurrect — re-observation overrides decay)
        └── Status: DEFERRED → reconfirm (new evidence available)

Reconfirmation is a lineage event, not an update. The CR's lineage gains a new RECONFIRMED entry with the new score, new fusion_run_id, and new timestamp. The CR's time_most_recent_observation is updated. The decay clock resets.

What parallax needs:

def reconfirm_correlation(
    cr: CorrelationRecord,
    new_score: float,
    fusion_run_id: str,
    now_iso: str = "",
) -> CorrelationRecord:
    """Reset decay clock and update confidence from re-observation.

    Adds a RECONFIRMED lineage entry. Updates time_most_recent_observation.
    If CR was DECAYED, status reverts to CONFIRMED (resurrection).
    If CR was DEFERRED, status reverts to PROPOSED (new evidence available).
    REJECTED CRs are never reconfirmed — human decision stands.
    """

LineageAction extension: The existing enum in correlation.py has: CREATED, RECORD_ADDED, ATTESTED, INVALIDATED, DECAYED. This spec adds RECONFIRMED — a re-observation event from a subsequent fusion run.

Lens Version Pinning

Activations are pinned to a specific lens version. When a lens goes from v1 → v2:

Existing v1 activations are not auto-migrated. They continue running against v1 until the lens version is retired.
When lens v1 is retired, all activations referencing v1 are automatically paused. An administrator must create new activations for v2.
An administrator may create v2 activations while v1 activations are still active — this allows gradual rollover (run both versions in parallel, compare results, retire v1 when confident).
CorrelationRecords always reference the lens_version that produced them. A CR from v1 and a CR from v2 for the same pair are distinct records with distinct lineage.

Why not auto-migrate? A new lens version may change weights, thresholds, metrics, or field scope. Auto-migrating activations would silently change operational behavior — violating the frozen principle (Invariant 4) and bypassing governance review of the activation's effective configuration.

Streaming Debounce

Streaming mode fires on every ingest event. If 100 records arrive in a batch, 100 independent score_one_vs_many() calls would create duplicate work — records 5 and 47 might match each other, producing two CRs for the same pair from two separate micro-runs. Additionally, 100 simultaneous triggers create a thundering herd problem.

Solution: micro-batch streaming. The trigger includes a debounce window that collects ingest events before dispatching a single fusion run.

trigger:
  type: ingest
  index_pattern: social_care_*
  filter:
    type: person
  debounce: 5s              # batch ingest events within 5s window
  max_batch_size: 100       # process up to 100 new records per micro-batch

Behavior: - Ingest events accumulate for debounce seconds (default 5s). - When the window closes (or max_batch_size is reached), a single fusion run processes all accumulated records together. - Within the micro-batch, records are cross-compared (catching intra-batch matches) AND compared against the existing index (catching cross-batch matches). - This turns "pure streaming" into "micro-batch streaming" — same parallax compute, amortized trigger overhead, no duplicate CRs.

Latency trade-off: 5s debounce means a new record is matched within ~5-10s of ingest (5s accumulation + scoring time). For most use cases this is acceptable. Critical use cases can set debounce: 0s for true per-record streaming at the cost of higher load and potential duplicates (which the reconfirmation flow handles gracefully).

Invariant Compliance

Invariant	Compliance
1. UDS is sole ABAC authority	Participation queries UDS for federation topology. Consent scope is enforced by parallax's `filter_by_consent()`. Data access governed by UDS at query time.
2. Events are append-only	All 8 objects use append-only state transitions. Revocation creates a new event. Decay creates a new event. No UPDATE, no DELETE.
3. Blocks are evidence	Every FusionRun creates evidence blocks with query hash, lens version, binding version, timing, and provenance.
4. Frozen means frozen	Approved lenses are frozen. Binding spec frozen after approval. CorrelationRecords are frozen at creation.
5. Editions require frozen evidence	Attestation (confirm/reject) on a correlation references a frozen CorrelationRecord backed by a frozen FusionRun.
6. AI assists, humans attest	AI computes matches. Governance officers approve lenses. Node administrators consent. Analysts attest correlations. Every human decision is a distinct object.
7. "No action" is a decision	Unattested correlations remain `proposed` — they don't auto-confirm. `deferred` is an explicit attestation decision. Decay is a scheduled decision, not a silent expiry.

Relationship to Existing Specs

Spec	Relationship
component.parallax.lens-parser (Lens Parser)	Lens authoring uses `parse_lens()` + `validate_lens()`. LensSpec is the `spec` field of the Lens governance object.
component.parallax.fusion-binding (Binding & Project)	Binding format, project DAG, FUSION_ENGINE node — all used as-is. This spec adds governance lifecycle wrapper.
component.parallax.correlation-persistence (Correlation Persistence)	CorrelationRecord persistence, blocking acceleration — used by all four execution modes. Streaming mode relies heavily on component.parallax.correlation-persistence's incremental blocking index.
component.parallax.primitives-framework (Primitives)	All 94 primitives run in parallax compute. This spec doesn't change what computes — it governs when and how compute is triggered.

What Gets Built (In Order)

Phase A: Governance Foundation (Platform)

ES mappings for fusion_lenses, fusion_lens_reviews, fusion_participation, fusion_activations, fusion_runs
Lens management MCP tools: create_lens, submit_lens_for_review, review_lens, activate_lens, retire_lens, list_lenses
Participation MCP tools: invite_node, record_consent, revoke_participation
Experience pack config for Fusion Manager page (Lenses + Reviews + Participation tabs)

Phase B: Operational Execution (parallax + Platform)

LensActivation dataclass in parallax
score_one_vs_many() and incremental_blocking_keys() in parallax
Activation MCP tools: configure_activation, activate, pause_activation
Execution MCP tools: execute_fusion, get_run_status, list_runs
Integration with titan for trigger dispatch (ingest watcher, cron scheduler, signal listener)
Experience pack config for Activations + Runs tabs

Phase C: Intelligence Output (Platform)

Signal generation from correlations (Cortex → Monitor page integration)
Correlation attestation MCP tools: list_correlations, attest_correlation
Experience pack config for Correlations + Analytics tabs
Reactive mode: signal-triggered lens execution
Cycle detection for reactive signal chains

Deferred

Lens authoring UI (Drawflow node builder for visual lens design)
LLM-assisted lens creation ("describe what you want to match, I'll generate the lens")
Multi-lens orchestration (run lens A's output as input to lens B)
Cross-federation lens sharing (lens templates published to a lens marketplace)

Cross-References

Document	Relationship
component.parallax.lens-parser (Lens Parser)	Lens authoring input contract
component.parallax.fusion-binding (Binding & Project)	Binding format and project DAG
component.parallax.correlation-persistence (Correlation Persistence)	Output persistence and blocking acceleration
component.parallax.scoring-engine (Scoring Engine)	Scoring thresholds govern attestation requirements
models/correlation.py	CorrelationRecord, Attestation, decay — already implemented
models/consent.py	ConsentPolicy, consent checking — already implemented
models/binding.py	LensBinding, ReviewStatus — already implemented
models/project.py	FusionProject, ProjectStatus — already implemented
docs/ARCHITECTURE.md	Module structure, clean architecture layering

Depends on: component.parallax.correlation-persistence, component.parallax.fusion-binding, component.parallax.lens-parser

Realizes: product.fusion, product.lens

Required by: component.parallax.wire-message-families