Example deliverable
What a pilot report looks like.
A report separates what was observed, how it was judged, what changed, and what remains unknown. It never turns a seeded fault into a customer defect.
Executive result
| Path | Synthetic U.S. equity paper-order input |
|---|---|
| Window | 20 expected sessions |
| Primary source | Synthetic source A |
| Reference source | Synthetic source B |
| Independence | Assumed for demonstration only |
| Evaluated observations | 5 |
| Decisions | 2 permit, 2 block, 1 escalate |
| Production actions changed | 0 |
Decision summary
| ID | Evidence | Decision | Reason |
|---|---|---|---|
| SYN-001 | PASS | PERMIT | Twenty sessions, monotonic dates, internally consistent OHLC, and closing-price deviation below 10 basis points. |
| SYN-002 | FAIL | BLOCK | The primary series returned 12 of 20 expected sessions. |
| SYN-003 | FAIL | BLOCK | The closing price on session 14 differed from the reference by 2.00%, above the fixed 0.10% threshold. |
| SYN-004 | INDETERMINATE | ESCALATE | Both sources were readable, but their upstream lineage was unknown. |
| SYN-005 | PASS | PERMIT | One extra response field did not change the evaluated values. |
Example discrepancy
decision_id SYN-003
instrument SYNTHETIC-XYZ
session 2026-09-14
primary_close 102.00
reference_close 100.00
deviation 2.00%
threshold 0.10%
evidence_status FAIL
observe_decision would BLOCK
The customer technical owner classifies the result as a confirmed issue, expected difference, false positive, or unresolved. A confirmed discrepancy becomes a deterministic regression fixture with the same instrument, date, values, and threshold.
What this report would not claim
- It would not say the primary source lied.
- It would not call two sources independent without lineage evidence.
- It would not treat a receipt signature as proof that the market fact was true.
- It would not call a seeded fault a customer defect.
- It would not extrapolate five observations into a reliability score.
The final recommendation
- Redundant on the measured path: no confirmed discrepancy and no new control information.
- Continue Observe Mode: useful signal exists, but false positives, lineage, or operational limits need more evidence.
- Test bounded enforcement: confirmed discrepancies exist, false-positive behavior is acceptable, and the customer chooses one low-risk path for a separate test.
Read the method →
