← Back to Reliability Explorer
Measured selection evidence
Playwright MCP
v28-playwright-mcp
Measured historical tool-selection reliability from frozen benchmark evidence.
Single-batchObserved missesVersion-bound
Version-bound source snapshot. Historical evidence does not establish current endpoint availability or current production behavior. Check observation freshness and schema changes.
1independent benchmark batches
10measured frozen cases
S1evidence level(s)
1observed semantic misses
First measurement: 2026-09-13 00:05:45.290567 UTC · Last measurement: 2026-09-13 00:05:45.290567 UTC · Observed strict accuracy range: 90.0%–90.0%
Batch history
Each immutable benchmark batch is shown separately. No pooled accuracy is calculated here.
| Benchmark / batch | Type | Frozen cases | Strict score | Wilson 95% CI | Provider-valid coverage | Infrastructure failures | Evidence / measured |
|---|---|---|---|---|---|---|---|
agent-trust-lab-v28-batch1agent-trust-lab-v28-batch1-playwright |
CONSTRUCTED_FROZEN_SELECTION_BENCHMARK | 10 | 9/10 90.0% |
59.6%–98.2% | 10 / 10 (100.0%) | 0 | S1 2026-09-13 00:05:45.290567 UTC |
Observed semantic misses
These are preserved adjudicated historical observations, attributable to a benchmark, batch, and case.
| Benchmark / batch | Case ID | Expected tool | Selected tool | Boundary family | Adjudication |
|---|---|---|---|---|---|
agent-trust-lab-v28-batch1-playwrightagent-trust-lab-v28-batch1 |
PW-006 | browser_fill_form |
browser_snapshot | OVERLAPPING_TOOL_SURFACE | model_selection_weakness |
Evidence provenance
Technical identities are secondary detail for auditability.
Playwright MCP
- Canonical subject ID
v28-playwright-mcp- Provenance type
- PINNED_PUBLIC_SCHEMA_WITH_INDEPENDENTLY_AUTHORED_CASES
- Identity mode
- VERSION_BOUND_SNAPSHOT
- Identity source
VERSION_BOUND_SNAPSHOT
agent-trust-lab-v28-batch1 immutable identities
- candidate_corpus_sha256
b75d4cd1bfc451bcd0ca4664183649d4b1ffa12403b5243c58de006e88563c1f- case_freeze_sha256
728d3de11c7f193716e2941965b548fe04dd29ccb89d202e320ce1a382bae84d- external_gold
False- measurement_artifact_sha256
45556674a1f420f0570c0e3a983427c0bd4ed2b98e98c208134d186fea3cfcfd- model_lock_sha256
deb767675074bd3760fc2e871825d802ce8d84975516f6debed2433139d73853