Signal Payload — Structured Upstream Context

Status & scope

Stage: RATIFIED (2026-03-14) — v2
Supersedes: v1 (flat-fields approach)
Author: Chris + Claude
Date: 2026-03-12
Affects: cortex/docs/spec/des-objects.yml (Signal), cortex/docs/spec/objects/signal.md, cortex/docs/spec/DES_Signal_Lifecycle_Specification.md
Grounded in: Axonis_Object_Taxonomy.md v0.2 — The Stack, The Boundary, The Taxonomy

1. Problem

The Signal schema has metadata (additionalProperties: true) as its only extensibility mechanism. Every pilot stuffs different things in there — fusion IDs, policy references, threshold conditions, composite scores — with no schema validation, no discoverability, and no contract between producer and consumer.

SPEC-14 v1 proposed adding flat fields (correlation_id, lens_id, policy_id, computation_type, etc.) directly to Signal. This works but has a design problem: it makes DES aware of upstream concepts. DES shouldn't know what a "lens_id" is or what "computation_type: state_transition" means. Those are implementation details of the systems that produce signals.

2. Design Principle: DES Starts at Signal

From the Object Taxonomy:

DES begins at Signal. DES does not define how signals are generated, how data is fused, or how schemas are mapped. Signal arrives with enough payload that DES can run a full investigation without reaching back upstream.

The v1 approach (flat fields with enums like lens_match, state_transition) leaks upstream concepts into the DES core schema. If a new signal origin appears (say, a third-party risk engine), someone has to extend the computation_type enum in DES.

The v2 approach: one payload field — a structured, typed, optional container. DES stores it, displays it, includes it in evidence blocks. DES never interprets it. Upstream systems define what goes in it. The payload schema is documented (so consumers can pattern-match) but not enforced by DES.

3. What Changes

v1 approach (SUPERSEDED)

# 7 flat fields added directly to Signal
correlation_id: ...
lens_id: ...
fusion_run_id: ...
matched_subjects: [...]
policy_id: ...
computation_type: { enum: [...] }
composite_score: ...

v2 approach (THIS SPEC)

# 1 structured field added to Signal
payload:
  type: object
  additionalProperties: true
  description: >
    Structured context from the system that generated this signal.
    DES stores, displays, and includes the payload in evidence blocks.
    DES does not interpret or validate payload contents beyond basic
    type checking. The payload schema is defined by convention
    between signal producers and consumers, not by DES.

4. Proposed Schema Change

In `des-objects.yml` — add after `routing` block (line 690), before `additionalProperties: true`

    # ─────────────────────────────────────────────────────────────
    # PAYLOAD — Structured upstream context
    # Added by SPEC-14 v2.
    # DES stores this, displays it, and includes it in evidence.
    # DES does NOT interpret payload contents. The sections below
    # are DOCUMENTED CONVENTIONS, not enforced sub-schemas.
    # Any upstream system can put any structured data here.
    # ─────────────────────────────────────────────────────────────
    payload:
      type: object
      additionalProperties: true
      description: >
        Structured context from the system that generated this signal.
        DES stores, displays, and includes the payload in evidence blocks
        but does not interpret it. Documented sections include correlation
        (from fusion engines), policy (from signal policy evaluation),
        observations (source data references), display (rendering hints),
        and context (domain-specific data). All sections are optional.

That's it. One field added to des-objects.yml. No enums. No upstream-specific types.

The internal structure of payload is documented below as conventions for signal producers, not as DES schema enforcement.

5. Payload Conventions (Producer Documentation)

These are the documented sections that signal producers SHOULD use when populating payload. DES does not enforce these — they exist so that experience packs and investigation UIs can pattern-match on well-known keys.

5.1 `payload.correlation` — Fusion Engine Origin

Present when a signal originates from an entity resolution match (Parallax, any fusion engine).

payload:
  correlation:
    correlation_id: "cor_abc123"       # Reference to CorrelationRecord
    correlation_type: "identity"       # identity | co-location | co-occurrence | temporal | convergence | anomaly
    confidence: 0.91                   # Match confidence 0.0-1.0
    matched_subjects:                  # The entities that were matched
      - type: "customer"
        id: "ent_001"
        name: "John Smith"
        role: "source"                 # source | target
        federate: "node_a"
      - type: "customer"
        id: "ent_047"
        name: "J. Smith"
        role: "target"
        federate: "node_b"
    lens_id: "lens_vrs_v1"            # Which Lens configuration
    run_id: "run_20260312_001"         # Fusion run for audit trail

Evidence chain: Signal → payload.correlation.correlation_id → CorrelationRecord → lens_id → LensSpec → SchemaBinding → source Observations

5.2 `payload.policy` — Signal Policy Origin

Present when a signal originates from a signal policy evaluation (threshold, state transition, staleness, correlation rule, composite scoring).

payload:
  policy:
    policy_id: "sensor_high"           # Which policy from signal_policies.yaml
    computation_type: "state_transition"  # state_transition | threshold | staleness | correlation | aggregate_staleness
    composite_score: 0.85              # Weighted risk score (if composite)
    conditions_met:                    # What conditions triggered this
      - field: "water_level"
        operator: ">"
        threshold: 7.5
        actual_value: 8.2
      - field: "rate_of_change"
        operator: ">"
        threshold: 0.5
        actual_value: 0.8

Evidence chain: Signal → payload.policy.policy_id → signal_policies.yaml → computation details → source Observations

5.3 `payload.observations` — Source Data References

References to the upstream observations that contributed to this signal. DES does not fetch these — they exist for evidence traceability and for MCP tools that can retrieve observation details on demand.

payload:
  observations:
    - observation_id: "obs_001"
      source_system: "norbert-robot-04"
      observed_at: "2026-03-12T14:33:00Z"
      summary: "HR 142bpm, SpO2 94%, Temp 37.8°C"
    - observation_id: "obs_002"
      source_system: "weather-api"
      observed_at: "2026-03-12T14:30:00Z"
      summary: "Heavy rain forecast, 40mm next 6h"

5.4 `payload.display` — Rendering Hints

Tells the rendering layer (experience pack / Beacon) how to visually present this signal. Generated by the signal producer based on the display block in signal_policies.yaml.

payload:
  display:
    subject_class: "geo"               # geo | entity | aggregate
    geo:                               # present when subject_class includes "geo"
      lat: 29.7604
      lon: -95.3698
      label: "North Pond"
    color: "#ef4444"                   # resolved from severity → color scale
    icon: "warning"                    # Material icon name

Three subject classes:

subject_class	Meaning	Primary Display	Examples
`geo`	Subject has a location	Map marker	Sensor alert, unit position, facility status
`entity`	Subject is an individual/account/case	List row or card	Patient vital, customer match, financial risk
`aggregate`	Signal is about a group or rollup	KPI count or summary card	Site offline, unit readiness, composite risk

Where display data comes from: - Signal policy defines the template (display block in signal_policies.yaml) - Signal producer resolves the template against live data at signal creation time - Resolved values are written to payload.display - Experience pack defines which views consume which subject classes (signal_display config) - DES is unaware of all of this

5.5 `payload.context` — Domain-Specific Extension

Catch-all for domain-specific data that doesn't fit the documented sections above. Any upstream system can put anything here.

payload:
  context:
    # VRS example
    vrs_codes: ["V-101", "V-204"]
    permission_type: "Type B"
    screening_batch_id: "batch_20260312"

    # Or sensor example
    site_id: "north_pond"
    maintenance_due: true
    last_calibration: "2026-02-15"

6. Signal Examples Per Pilot

VRS (Lens origin)

{
  "signal_type": "potential_match",
  "source": { "type": "computed", "system_name": "Semantic Lens Engine" },
  "severity": "high",
  "subject": { "type": "customer", "id": "ent_001", "name": "John Smith" },
  "title": "Potential VRS match (91% confidence)",
  "confidence": 0.91,
  "payload": {
    "correlation": {
      "correlation_id": "cor_abc123",
      "correlation_type": "identity",
      "confidence": 0.91,
      "matched_subjects": [
        { "type": "customer", "id": "ent_001", "name": "John Smith", "role": "source", "federate": "node_a" },
        { "type": "customer", "id": "ent_047", "name": "J. Smith", "role": "target", "federate": "node_b" }
      ],
      "lens_id": "lens_vrs_v1",
      "run_id": "run_456"
    },
    "display": {
      "subject_class": "entity"
    }
  }
}

Water Sensor (Policy origin, geo display)

{
  "signal_type": "state_transition",
  "source": { "type": "computed", "system_name": "Signal Policy Engine" },
  "severity": "critical",
  "subject": { "type": "sensor", "id": "sen_north_pond", "name": "North Pond Level Sensor" },
  "title": "Flood sensor HIGH — 8.2ft",
  "confidence": 1.0,
  "payload": {
    "policy": {
      "policy_id": "sensor_high",
      "computation_type": "state_transition",
      "conditions_met": [
        { "field": "water_level", "operator": ">", "threshold": 7.5, "actual_value": 8.2 }
      ]
    },
    "display": {
      "subject_class": "geo",
      "geo": { "lat": 29.7604, "lon": -95.3698, "label": "North Pond" },
      "color": "#ef4444",
      "icon": "warning"
    }
  }
}

Norbert Health (Policy origin, entity display, with observation references)

{
  "signal_type": "vital_anomaly",
  "source": { "type": "computed", "system_name": "Signal Policy Engine" },
  "severity": "high",
  "subject": { "type": "patient", "id": "pat_042", "name": "Patient 042" },
  "title": "Elevated heart rate — nurse review needed",
  "payload": {
    "policy": {
      "policy_id": "hr_anomaly",
      "computation_type": "threshold",
      "conditions_met": [
        { "field": "heart_rate", "operator": ">", "threshold": 120, "actual_value": 142 }
      ]
    },
    "observations": [
      {
        "observation_id": "obs_nr_001",
        "source_system": "norbert-robot-04",
        "observed_at": "2026-03-12T14:33:00Z",
        "summary": "HR 142bpm, SpO2 94%, Temp 37.8°C"
      }
    ],
    "display": {
      "subject_class": "entity"
    }
  }
}

LUF Phase 2 (Lens origin, multi-federate)

{
  "signal_type": "pricing_correlation",
  "source": { "type": "computed", "system_name": "Semantic Lens Engine" },
  "severity": "medium",
  "subject": { "type": "pricing_entity", "id": "px_cruise_001", "name": "Caribbean Cruise Q2" },
  "title": "Pricing signal from 3-source fusion (78% confidence)",
  "confidence": 0.78,
  "payload": {
    "correlation": {
      "correlation_id": "cor_luf_789",
      "correlation_type": "co-occurrence",
      "confidence": 0.78,
      "matched_subjects": [
        { "type": "demand_signal", "id": "dem_001", "role": "source", "federate": "demand_node" },
        { "type": "supplier_rate", "id": "sup_001", "role": "source", "federate": "supplier_node" },
        { "type": "sentiment", "id": "sen_001", "role": "source", "federate": "sentiment_node" }
      ],
      "lens_id": "lens_cruise_pricing_v1",
      "run_id": "run_luf_20260312"
    },
    "display": {
      "subject_class": "entity"
    }
  }
}

AIDP (Lens origin, multi-INT, geo display)

{
  "signal_type": "potential_match",
  "source": { "type": "computed", "system_name": "Semantic Lens Engine" },
  "severity": "high",
  "subject": { "type": "person", "id": "ent_int_001", "name": "[REDACTED]" },
  "title": "Multi-INT entity match (87% confidence)",
  "confidence": 0.87,
  "payload": {
    "correlation": {
      "correlation_id": "cor_aidp_001",
      "correlation_type": "identity",
      "confidence": 0.87,
      "matched_subjects": [
        { "type": "person", "id": "sig_ent_001", "role": "source", "federate": "sigint_node" },
        { "type": "person", "id": "hum_ent_001", "role": "target", "federate": "humint_node" }
      ],
      "lens_id": "multi_int_person_v2",
      "run_id": "fr_20260312_001"
    },
    "display": {
      "subject_class": "geo",
      "geo": { "lat": 33.5138, "lon": 36.2765, "label": "[Location]" },
      "icon": "person_pin"
    }
  }
}

Composite / Aggregate (Multi-policy, aggregate display)

{
  "signal_type": "site_offline",
  "source": { "type": "computed", "system_name": "Signal Policy Engine" },
  "severity": "critical",
  "subject": { "type": "site", "id": "site_north", "name": "North District" },
  "title": "Site offline — 4/4 sensors silent >2h",
  "payload": {
    "policy": {
      "policy_id": "site_offline",
      "computation_type": "aggregate_staleness",
      "composite_score": 0.95,
      "conditions_met": [
        { "field": "active_sensor_count", "operator": "=", "threshold": 0, "actual_value": 0 }
      ]
    },
    "display": {
      "subject_class": "aggregate",
      "geo": { "lat": 29.76, "lon": -95.37, "label": "North District" },
      "color": "#dc2626",
      "icon": "error"
    }
  }
}

7. Signal Policy `display` Block — Upstream Config

Signal policies SHOULD include an optional display block that templates how signals of this type should be displayed. The signal generator resolves this template at creation time and writes the result to payload.display.

Addition to signal_policies.yaml format

policies:
  - policy_id: sensor_high
    name: Sensor High State
    source_model: sensor
    # ... existing fields (triggers, computation, severity_map) ...

    # NEW — optional display template
    display:
      subject_class: geo              # geo | entity | aggregate
      geo_field: sensor.location      # where to resolve lat/lon from source data
      label_field: sensor.site        # what to use as the map/list label
      color_by: severity              # resolve color from severity via color_scale
      icon: warning                   # Material icon

  - policy_id: hr_anomaly
    name: Heart Rate Anomaly
    source_model: bio_vitals
    display:
      subject_class: entity
      label_field: bio_vitals.subject_name
      group_by: bio_vitals.unit

  - policy_id: site_offline
    name: Site Offline
    source_model: sensor
    display:
      subject_class: aggregate
      count_label: "Sites offline"
      geo_field: sensor.location      # also plottable on map

Default behavior

Policies without a display block default to:

display:
  subject_class: entity               # show in signal list — current behavior

This is backward-compatible. Existing policies produce signals that appear in the signal list, exactly as they do today.

8. Experience Pack Signal Display Config

Experience packs SHOULD include a signal_display section that tells the rendering layer how to present signals by subject class.

# experiences.yaml → monitor section
signal_display:
  geo:
    primary_view: map_layer           # add to map as overlay layer
    secondary_view: signal_list       # also show in signal list
    cluster: true                     # cluster nearby signals
    color_by: severity
  entity:
    primary_view: signal_list         # show in signal list
    secondary_view: null
    group_by: subject.type
    sort_by: severity
  aggregate:
    primary_view: kpi_banner          # show as KPI count
    secondary_view: signal_list

This is a new section. Experience packs without it use default behavior (all signals → signal list).

9. Why v2 Over v1

Concern	v1 (flat fields)	v2 (payload)
DES purity	Leaks upstream concepts (lens_id, computation_type enum) into DES core	DES sees one opaque `payload` — no upstream concepts
Schema evolution	Adding new origin types means extending DES enums	Adding new origin types means documenting new payload conventions — DES unchanged
Validation	DES validates upstream-specific fields	DES validates Signal shape; payload is additionalProperties
Discoverability	First-class fields are discoverable in schema	Documented conventions are discoverable in this spec
ES querying	Flat fields auto-map to ES keywords	Nested payload fields require ES object mapping (minor config)
Display hints	Would need MORE flat fields	Natural fit — `payload.display` sits alongside `payload.correlation` and `payload.policy`
Open-source DES	Would need to strip upstream-specific fields before publishing	Publish as-is — `payload` is intentionally opaque

The trade-off: v1 gives you ES query performance on origin fields (keyword lookup). v2 gives you DES purity and schema stability. For the "query signals by correlation_id" use case, ES nested object queries work — slightly more verbose but functionally equivalent.

Recommendation: v2. The strategic value of keeping DES clean outweighs the minor ES query convenience of flat fields. And when DES is published as an open spec, the payload approach means third-party systems can produce DES-compliant signals without knowing about Axonis internals.

10. Impact

Component	Change	Effort
`des-objects.yml` Signal schema	Add 1 `payload` field (object, additionalProperties: true)	15 min
`objects/signal.md`	Add Payload section documenting conventions	30 min
`DES_Signal_Lifecycle_Specification.md`	Add payload to §1.2 Cortex Additions	15 min
Signal policy YAML (per domain)	Add optional `display` block	15 min × 5 domains
Experience pack YAML (per domain)	Add `signal_display` section	15 min × N packs
Cortex signal generation code	Write payload instead of metadata	30 min
ES mapping	Add `payload` as nested/object type	15 min
Beacon signal rendering	Read `payload.display.subject_class` for routing	1-2h
Existing signals	Unchanged — payload is optional, metadata still works	Zero

Total: ~4 hours including all domains and Beacon work.

11. Migration

Existing signals use metadata for upstream context. No migration required — metadata continues to work. New signals SHOULD use payload instead. The metadata field remains for truly ad-hoc data.

Convention: if both metadata and payload exist on a signal, consumers read payload first, fall back to metadata for fields not present in payload.

12. DO NOT

Make payload required — manual and webhook signals may have no payload
Add payload sub-schema validation in DES — DES stores it as-is
Put per-field match scores in payload — that belongs in CorrelationRecord (accessed via correlation_id)
Put raw PII in payload — use entity references (id + display name only)
Remove metadata — it's still useful for truly unstructured data
Change subject semantics — keep singular for routing; payload.correlation.matched_subjects is supplementary
Add DES-specific enums to payload conventions — keep conventions upstream-agnostic

13. Approval Checklist

[x] Chris approves adding payload to Signal in des-objects.yml
[x] Chris approves payload conventions (correlation, policy, observations, display, context)
[x] Chris approves signal_policies.yaml display block format
[x] Chris approves experience pack signal_display config format
[ ] Changes applied to cortex/docs/spec/ (design authority)
[ ] Changes applied to REST objects.yml (runtime authority) — separate PR
[ ] sensor_iot signal_policies updated as reference implementation
[x] SPEC-14 v1 marked as superseded (archived: specs/_archive_v1/SPEC-14-SIGNAL-SCHEMA-EXTENSION.md)

Appendix: Cross-Reference

Document	Relationship
`Axonis_Object_Taxonomy.md` v0.2	Defines the stack and boundary that motivates this spec
`PILOT-SIGNAL-MAPPING.md`	Maps all 5 pilots to signal origins — validates payload covers all cases
`specs/_archive_v1/SPEC-14-SIGNAL-SCHEMA-EXTENSION.md` (SPEC-14 v1, SUPERSEDED)	Original flat-fields approach — replaced by this spec
`SPEC-14-RATIFICATION-PROPOSAL.md`	Ratification proposal for v1 — superseded
`DES_Signal_Lifecycle_Specification.md`	Signal lifecycle — §1.2 needs payload addition
`signal_policies.yaml` (sensor_iot)	Most mature policy config — reference for `display` block
`signal_policies.yaml` (pacific_sentinel)	17 policies — validates entity + geo display classes
`experiences.yaml` (sensor_iot)	Reference for `signal_display` config addition

Realizes: product.signal

Required by: component.parallax.signal-queue-contract, component.parallax.wire-message-families