Skip to content

Scoring Engine (Orchestrator)

Status & scope

  • Module: lens/orchestrator.py
  • Milestone: Phase 1 (complete)

Purpose

Execute a parsed LensSpec against input data: resolve sources → compute cost per layer → weighted composite → threshold evaluation → evidence block + optional signal. This is the function that becomes execute_lens_block in Cortex.

Public API

run_lens(spec, inputs, layer_data, cost_registry, layer_registry) → LensResult

  • Core orchestrator. Executes all layers, produces composite score.
  • Algorithm:
  • For each layer: resolve source (from layer_data dict or layer_registry)
  • Resolve cost model via resolve_cost_model() (binding.py)
  • Compute cost_fn(data, layer.params) → layer score
  • Weighted composite: sum(weight_i * score_i) / sum(weight_i)
  • Evaluate thresholds → optional Signal
  • Create frozen EvidenceBlock
  • Return LensResult

compare_coas(spec, coa_configs, base_inputs, ...) → COAComparison

  • Run lens N times with different layer_data overrides.
  • Rank results by configurable metric.
  • Mark top-ranked as recommended.
  • Create comparison evidence block.

Dataclasses (all frozen=True)

LensResult

Field Type Default Notes
lens_id str — From spec
lens_type str — From spec
status str — "success" | "partial" | "error"
outcome str — Human-readable summary
artifacts dict {} Domain-specific output
metrics dict {} Layer name → raw score
composite_score float 0.0 Weighted composite, 0–1
layer_contributions dict {} Layer name → weighted contribution
evidence_block EvidenceBlock | None None Frozen evidence
signal Signal | None None If threshold crossed
execution_time_ms float 0.0 Wall clock
errors tuple[str, ...] () Per-layer errors

COAResult

Field Type Default Notes
coa_id str — "coa_1", "coa_2", ...
coa_name str — Human-readable
decision_type str — "action" | "no_action" | "escalation"
description str "" COA description
lens_result LensResult | None None Full lens run output
metrics dict {} Composite + layer + summary metrics
rank int 0 1 = best
recommended bool False True for rank 1

COAComparison

Field Type Default Notes
comparison_id str — UUID prefix
scenario_name str — From spec.name
coas tuple[COAResult, ...] () Ranked results
recommended_coa_id str "" Top-ranked COA ID
comparison_metrics tuple[str, ...] () Which metrics compared
evidence_block EvidenceBlock | None None Comparison evidence

Cost Model Dispatch

The orchestrator resolves cost models through binding.py:

Layer.cost_model → binding.resolve_cost_model()
  1. Legacy cost_registry (domain cost_models.py)
  2. LENS_SCORING_OPS (Command subclasses, after component.prism.cost-primitives)
  3. COST_PRIMITIVES (pure functions)
  4. Raise ValueError

When component.prism.cost-primitives is implemented, the orchestrator gains Command-based dispatch automatically — no changes to orchestrator.py needed (binding.py handles the resolution).

Test Fixtures

Test Input Expected File
Run simple 3-layer lens layer_data with floats composite_score = weighted avg tests/test_orchestrator.py
Run with threshold crossing score > confirm Signal with severity="high" tests/test_orchestrator.py
Run with missing layer data one layer missing status="partial", error logged tests/test_orchestrator.py
Compare 3 COAs 3 configs with different data Ranked 1-3, recommended set tests/test_orchestrator.py
Evidence block created evidence=True in output Non-null, frozen, hashes set tests/test_orchestrator.py
Traversal lens with primitives cost_model="linear_scale" Uses primitive, correct score tests/test_integration.py

Current test count: 22 orchestrator tests (all passing)

File Layout

lens/
  orchestrator.py          ← run_lens(), compare_coas(), LensResult, COAResult, COAComparison

Integration Points

  • Cortex (TASK-C01): run_lens() becomes the body of execute_lens_block MCP tool. Cortex wraps it with UDS source resolution and block creation.
  • Athena: No direct integration — orchestrator calls ops through binding.py.
  • Beacon: Receives LensResult serialized as JSON via REST. Renders artifacts in monitor page.

DO NOT

  • Add domain-specific logic to the orchestrator — it's generic across all 5 lens types
  • Import domain submodules (traversal, threat, etc.) — layer_data comes from callers
  • Build workflow or state machines — run_lens is a single pure computation
  • Auto-route between COAs — human attests the selection (Invariant 6)

Depends on: component.prism.cost-primitives, component.prism.universal-lens-parser

Realizes: product.lens

Required by: component.prism.composition, component.prism.lens-families, component.prism.operational-lifecycle, component.prism.platform-integration, component.prism.rendering-architecture, component.prism.scenario-runner, component.prism.semantic-adapter