Evidence · OutFigure

§ 1 Levels

How strong the evidence is

A passing software test supports only the second level. Stronger levels need real data, frozen predictions, controlled comparisons or independent replication.

Hypothesis Stated, not yet tested.
Implemented The software behaves as described under automated tests. Highest level reached by a published claim
Retrospective Checked against historical data after the fact.
Prospective Predictions frozen before outcomes, then compared.
Controlled intervention An intervention compared against a control.
External replication Reproduced independently by others.
Production monitoring Tracked continuously in real use.

§ 2 Claims

Approved claims

These describe software behavior, verified by automated tests on synthetic data. None is a result about learners or learning.

C-001 · 2026-09-14 Implemented

Retrying or duplicating an event submission does not change learner-concept counts.
Limitations
- Local Postgres only; no hosted deployment has been tested.
- Network delivery is at least once; the guarantee applies to stored effects.
Evidence: tests/integration/ingestion.test.ts (A04, A05, A07)
C-002 · 2026-09-14 Implemented

One project cannot read or modify another project's learner data through the API, the console or the runtime database role.
Limitations
- Does not protect against a stolen runtime database credential.
- No independent security review has been performed.
Evidence: tests/isolation/tenant-isolation.test.ts (A10, A11)
C-003 · 2026-09-14 Implemented

Given the same stored events and rule versions, observed-v0 produces the same summary regardless of arrival order.
Limitations
- Determinism is not evidence that the summary is useful or predictive.
Evidence: packages/engine/src/observed-v0.test.ts; tests/replay/replay-and-recovery.test.ts (A12)
C-004 · 2026-09-14 Implemented

Responses without reported assistance are stored but excluded from observed summaries.
Limitations
- Depends on partners reporting assistance accurately, which is not verified.
Evidence: packages/engine/src/observed-v0.test.ts (eligibility-v1)
C-005 · 2026-09-14 Implemented

The console's key screens have no serious or critical findings in automated accessibility scans at 1440 px and 360 px.
Limitations
- Automated scans cover part of WCAG 2.2 AA. No manual audit or assistive-technology testing has been done, and no conformance is claimed.
Evidence: tests/e2e/console.spec.ts; tests/e2e/mobile.spec.ts
C-010 · 2026-09-14 Implemented

On the synthetic export, the readiness report's counts of learner-concept pairs, statuses, flags and eligible responses match what API ingestion and processing produce from the same events.
Limitations
- Verified on the synthetic export only; real exports may contain structures the fixture lacks.
- The test compares totals, not each learner-concept state individually.
Evidence: tests/integration/audit-parity.test.ts
C-011 · 2026-09-14 Implemented

The readiness report makes no network calls, and its output contains no learner identifiers. Validation problems are reported as codes and line numbers, never as submitted values.
Limitations
- The output names item versions with their counts and, when at least 10 learners answered them, their statistics, and it names the input files.
- Aggregate counts and item identifiers can still identify people in small cohorts.
Evidence: packages/audit/src/analyze.test.ts; packages/audit/src/items.test.ts; apps/cli/src/cli.test.ts; eslint.config.js rules for packages/audit and apps/cli

§ 3 Not yet

What has not been shown

No learning-outcome, prediction or efficacy results exist yet. Evaluation against later outcomes is in development.

When results exist, they will appear here with their design, sample, uncertainty and limitations, including results in which a model does not beat a simple baseline.

Updated 15 September 2026