# Task 1 Baseline: 2026-07-16

## Headline Retrieval Results

| Corpus | Queries | Recall@1 | Recall@3 | Recall@5 | Recall@10 | MRR | p50 | p95 | Average |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| Current generated known-item corpus (`current-generated-v2`) | 116 | 95.69% | 100% | 100% | 100% | 0.9756 | 4.4ms | 7.9ms | 5.2ms |
| Frozen legacy query/gold-path gate (`legacy-60-v1`) | 60 | 93.33% | 100% | 100% | 100% | 0.9611 | 4.4ms | 7.2ms | 5.5ms |

This is the measurement-only baseline for Task 1 of the unified evidence
retrieval roadmap. The complete retrieval result is preserved in
[`baseline-2026-07-16-retrieval.json`](baseline-2026-07-16-retrieval.json),
including `k_values`, all raw `latencies_ms`, and every `per_query` record.

Artifact SHA-256: `684e2f909e42732bbda0bbc0dd96c65f8de6f39e56610a3f1d00e98a2d94a2a0`.

## Source And Environment

Measured source state and environment:

```text
Branch: v4.0-unified-knowledge
Commit: 227b8004a094676bbd9ccc5510767b353feee5ea
Python: 3.10.20
SQLite: 3.50.4
Initial git status:
 M docs/ROADMAP-v5.md
?? docs/superpowers/plans/2026-07-16-unified-evidence-retrieval.md
```

Commands used to collect the source facts:

```powershell
git status --short --branch
git branch --show-current
git rev-parse HEAD
uv run python --version
uv run python -c "import sqlite3; print(sqlite3.sqlite_version)"
```

Installed optional extras were derived from `pyproject.toml` and
`uv pip list --format json`:

| Extra | Installed direct requirements |
|---|---|
| `semantic` | `sentence-transformers==5.6.0` |
| `hybrid` | `lancedb==0.34.0`, `sentence-transformers==5.6.0` |
| `cognee` | not installed |
| `code-graph` | `jedi==0.20.0`, `tree-sitter==0.26.0`, `tree-sitter-python==0.25.0`, `tree-sitter-javascript==0.25.0`, `tree-sitter-typescript==0.23.2`, `tree-sitter-go==0.25.0`, `tree-sitter-rust==0.24.2`, `tree-sitter-java==0.23.5`, `tree-sitter-c==0.24.2`, `tree-sitter-cpp==0.23.4`, `tree-sitter-ruby==0.23.1`, `tree-sitter-php==0.24.1`, `tree-sitter-c-sharp==0.23.5`, `tree-sitter-bash==0.25.1` |
| `mcp-server` | `mcp==1.28.1` |
| `reranker` | `onnxruntime==1.23.2`, `optimum==1.27.0`, `tokenizers==0.22.2` |
| `full` | all requirements of `semantic`, `hybrid`, `code-graph`, `mcp-server`, and `reranker` are installed |

Two inventory helper attempts failed: the first with a PowerShell quoting
`SyntaxError`, and the corrected helper with `ModuleNotFoundError: No module
named 'tomllib'` because the baseline uses Python 3.10. The inventory above is
from the successful `uv pip list --format json` command.

## Tests And Lint

Portable command form:

```powershell
$env:LLM_WIKI_ROOT=(Get-Location).Path
$env:LLM_WIKI_STATE_ROOT=Join-Path $env:TEMP 'llm-wiki-stage3-baseline-pytest-20260716-a91f'
uv run pytest -q
# 1781 passed, 23 skipped, 4 warnings in 1044.71s (0:17:24); exit 0

uv run ruff check scripts/ tests/ benchmark/
# All checks passed!; exit 0
```

The four warnings were `DeprecationWarning` reports for LanceDB
`table_names()` in `scripts/lance_store.py:59` and the corresponding asyncio
callback context.

## Retrieval Benchmark

> Historical command warning: the command below records the command used for
> this baseline and is intentionally unchanged. The benchmark entry point now
> defaults to retrieval-v2; use `--legacy-only` to run the old gate today.

Portable command form for the final raw artifact run:

```powershell
$env:LLM_WIKI_ROOT=(Get-Location).Path
$env:LLM_WIKI_STATE_ROOT=Join-Path $env:TEMP 'llm-wiki-stage3-baseline-retrieval-review-20260716-e65f'
uv run python benchmark/run_benchmark.py --json
# exit 0
```

The runner printed three progress lines before the JSON object. The raw evidence
artifact contains the complete JSON object, with 116 current records and 60
legacy records.

## Contradiction Benchmark

The roadmap command failed because this runner does not define `--json`:

```powershell
$env:LLM_WIKI_ROOT=(Get-Location).Path
$env:LLM_WIKI_STATE_ROOT=Join-Path $env:TEMP 'llm-wiki-stage3-baseline-contradiction-20260716-c43d'
uv run python benchmark/run_contradiction_benchmark.py --json
# error: unrecognized arguments: --json; exit 2
```

The supported command emits JSON without an output flag and passed:

```powershell
$env:LLM_WIKI_ROOT=(Get-Location).Path
$env:LLM_WIKI_STATE_ROOT=Join-Path $env:TEMP 'llm-wiki-stage3-baseline-contradiction-supported-20260716-d54e'
uv run python benchmark/run_contradiction_benchmark.py
# exit 0
```

```json
{
  "candidate_recall": 1.0,
  "class_macro_f1": 1.0,
  "extraction_f1": 1.0,
  "false_supersession": 0.0,
  "lifecycle_macro_f1": 1.0,
  "provenance_correctness": 1.0,
  "quarantine_candidates": 80,
  "quarantine_coverage": 0.6666666666666666,
  "quarantine_notes_published": 0,
  "quarantine_risk": 0.0,
  "semantic_benchmark_gate": false,
  "semantic_critique_calls": 40,
  "semantic_evaluators_independent": true,
  "semantic_fallback_probes": 80,
  "semantic_primary_calls": 40
}
```

## Agent Contract Integrity

```powershell
git diff --no-index -- AGENTS.md CLAUDE.md
# no differences; exit 0 (byte-equal)

Get-FileHash -Algorithm SHA256 -LiteralPath 'AGENTS.md','CLAUDE.md'
# AGENTS.md  F5890746F121387FFD76603406FBA6A72B59B0EEBEF8287739059170E70C3E47
# CLAUDE.md  F5890746F121387FFD76603406FBA6A72B59B0EEBEF8287739059170E70C3E47
```
