← Back to Reliability Explorer
Measured selection evidence
Microsoft Learn
microsoft-learn
Measured historical tool-selection reliability from frozen benchmark evidence.
Multi-batchS2-backedMixed provenance
Mixed live and version-bound evidence. Historical evidence does not establish current endpoint availability or current production behavior. Check observation freshness and schema changes.
5independent benchmark batches
31measured frozen cases
S2evidence level(s)
0observed semantic misses
First measurement: 2026-09-10 15:38:03.968297 UTC · Last measurement: 2026-09-11 00:07:39.419700 UTC · Observed strict accuracy range: 100.0%–100.0%
Batch history
Each immutable benchmark batch is shown separately. No pooled accuracy is calculated here.
| Benchmark / batch | Type | Frozen cases | Strict score | Wilson 95% CI | Provider-valid coverage | Infrastructure failures | Evidence / measured |
|---|---|---|---|---|---|---|---|
external-gold-expansion-v2-batch1external-gold-expansion-v2-batch1 |
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK | 5 | 5/5 100.0% |
56.6%–100.0% | 5 / 5 (100.0%) | 0 | S2 2026-09-10 15:38:03.968297 UTC |
external-gold-expansion-v2-batch2external-gold-expansion-v2-batch2 |
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK | 5 | 5/5 100.0% |
56.6%–100.0% | 5 / 5 (100.0%) | 0 | S2 2026-09-10 16:49:53.346446 UTC |
external-gold-expansion-v2-batch4external-gold-expansion-v2-batch4 |
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK | 6 | 6/6 100.0% |
61.0%–100.0% | 6 / 6 (100.0%) | 0 | S2 2026-09-10 18:47:01.455700 UTC |
external-gold-expansion-v2-batch5external-gold-expansion-v2-batch5 |
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK | 7 | 7/7 100.0% |
64.6%–100.0% | 7 / 7 (100.0%) | 0 | S2 2026-09-10 19:13:49.842728 UTC |
external-gold-expansion-v2-batch6external-gold-expansion-v2-batch6 |
EXTERNAL_INDEPENDENT_GOLD_BENCHMARK | 8 | 8/8 100.0% |
67.6%–100.0% | 8 / 8 (100.0%) | 0 | S2 2026-09-11 00:07:39.419700 UTC |
Observed semantic misses
These are preserved adjudicated historical observations, attributable to a benchmark, batch, and case.
| Benchmark / batch | Case ID | Expected tool | Selected tool | Boundary family | Adjudication |
|---|---|---|---|---|---|
| No observed semantic misses in current evidence. This is not proof that no failures occurred. | |||||
Evidence provenance
Technical identities are secondary detail for auditability.
Microsoft Learn
- Canonical subject ID
microsoft-learn- Provenance type
- EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
- Identity mode
- LIVE_TOOLS_LIST_SNAPSHOT
- Identity source
CURRENT_LIVE_TOOLS_LIST_FROZEN_BY_SNAPSHOT
Microsoft Learn
- Canonical subject ID
microsoft-learn- Provenance type
- EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
- Identity mode
- LIVE_TOOLS_LIST_SNAPSHOT
- Identity source
CURRENT_LIVE_TOOLS_LIST_FROZEN_BY_EXISTING_IMMUTABLE_SNAPSHOT
Microsoft Learn
- Canonical subject ID
microsoft-learn- Provenance type
- EXTERNAL_INDEPENDENT_GOLD_BENCHMARK
- Identity mode
- VERSION_BOUND_SNAPSHOT
- Identity source
CURRENT_OR_VERSION_BOUND_SNAPSHOT_REUSED_FROM_QUALIFIED_CORPUS
external-gold-expansion-v2-batch1 immutable identities
- case_freeze_sha256
17777d2c3983cd444353b8d69dcc98836a1909ee6fde6f5c8cf18cf9b059427e- final_freeze_receipt_commit
b691170- freeze_input_commit
c614ae51750d12ce113b5e3d51429f52e529f08c- measurement_artifact_sha256
01243422aa946296f235810aaee6f065ec21d9e78a01017ee8161e5e61a8b5e8- measurement_commit
b9f358b1a65aba348fa1c0e625eac6da1f31b651- model_lock_sha256
c4330165deb3d8587e224a2ca1b3d886db60801ca436710a0edace55525923d2- policy_evaluation_commit
3e560e5767f181831c65569b893ca789cd3bdff0- policy_evaluation_sha256
4e710a9c5205660cc07603d2fd7538aa063eda4c2574c29308f9f255e4706f82- promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d- raw_predictions_sha256
ab085cedf13eef5724f4e160b0f49f8402d62bc5b10203de0f8d81e41f2a017e
external-gold-expansion-v2-batch2 immutable identities
- case_freeze_sha256
ce21fc085bd9a0185c9e37d0d795221f286397cbc477ee941e876be24f7ba1b4- final_freeze_receipt_commit
d3c55d2f96a4e45bf2577225ea16929877f2ded1- freeze_input_commit
61f6511c41562c24019cd441b37bc6ee07283951- measurement_artifact_sha256
dd13aec8a90b2b5490c934c9ad9650c37733c65f18d824e2ae6066202176ced9- measurement_commit
f0b6cd33be044d8bad800c20b4f2684e6d8431ce- model_lock_sha256
153647f94eed0ee3d3069229545846eef864fd43fa821f2d311dc3fb5d265ba9- policy_evaluation_commit
f0b6cd33be044d8bad800c20b4f2684e6d8431ce- policy_evaluation_sha256
736f154a5d5c3b8f0d77cd58fa0644ca97ad94c7a6b46af98d394c1dc1578f9a- promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d- raw_predictions_sha256
8c4364ab598241ddbe4adb16453ba2e12b0917a1d1852caedb07884d1a54570b
external-gold-expansion-v2-batch4 immutable identities
- case_freeze_sha256
a9513c1cb9b800b75d822d75fa8c6e47041e551d86a75699f1858b6c3785204d- final_freeze_receipt_commit
224aeab80db539383018d77cc7adb849976f1db5- freeze_input_commit
003985017fbd8c1e207fb146329b97b38582e606- immutable_measurement_receipt_sha256
7d6d0a1ac4b1d501aa4c5d0044b4a4e50bb319b272bd33eb783aec6499d594c6- measurement_artifact_commit
9b10dc0c99c4eed15a3d370743e8f46439302bae- measurement_artifact_sha256
7d9a5f57e0897f2ca8b2734ac0f38dc5afd099b2edf2495dbfa25e41f088cee9- model_lock_sha256
ffc1ec351c1a62c3692cdaedba0c5a6ec2c6285923eb065e0cd31953f978e3a1- policy_evaluation_sha256
0e9b11360db41731ce139584060b3f1417ce50b31dcf86ef88180e391e56e0d5- promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d- raw_predictions_sha256
869812a69814f58e73a1c6c3f94a28b6dabc2cdb729285d3a532dd528ad36c89- scanner_freeze_sha256
e31caf3131b4e03f54ff411a386bb72ac70c35fbe9027e3383b7026019c22ae1
external-gold-expansion-v2-batch5 immutable identities
- case_freeze_sha256
40847d041af701492505676ae027f824248a94dba8dad19c4c764baec224a2bc- final_freeze_receipt_commit
4c11daed10b52027a8a31130703335e4835873ec- freeze_input_commit
935e16c4cc073d72b1c64c48206196feaf28c170- immutable_measurement_receipt_commit
b064dea6a0ea488826d11fc0fc88c5b4de3d5c26- measurement_artifact_commit
87a78aba4797f79f3c99c22e202eddd1da70f034- measurement_artifact_sha256
9b5f0c7f94783f2b25d28d8ecdaebe3a5a11b9db00dd84f677f95ac008f32eeb- model_lock_sha256
60c5653e461180e44e9621a54bb77f3f474e6889ddc29d1ff30d3cb9398013fa- policy_evaluation_sha256
2ef31f2a44a722fb862425dc11755b118c3ffb712672471d10d4cd0bbcee9af1- promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d- raw_predictions_sha256
612c995839a9ad1a32d87320055ada3201c511b626a0f3e8778a8bbe5201bc68- scanner_freeze_sha256
80b3b0d60a5ec2e3b5019d2c45838dd06517453ce64c9a32ed0dbda6be718795- validation_criteria_sha256
0cd5821683b9f85ae37140bfc73d25064012d69aabbbbb4363e56b058cb240a6
external-gold-expansion-v2-batch6 immutable identities
- case_freeze_sha256
4e7500bb9cc1bb4c2b1bbdc757814a7eb2dd5eac48f131665328c1d886f6960b- final_freeze_receipt_commit
6b1957bf7d556dbf023ac24cf22e50bcd64b0136- freeze_input_commit
95d1a1017ab4b0fde8dfb3dbab3140997867a07b- immutable_measurement_receipt_binding_commit
df7c3c0a04b91e4d113e445f1ed4d78a212fe6dd- measurement_artifact_commit
8854ae3a9e69abfc09cd63fb0daaf9fe5b34fc63- measurement_artifact_sha256
e662d13721d11587fb102fe4699b16906c566d1342286a1c946424653aacd912- model_lock_sha256
c020241636261d34903e5155cb6177402ce658c0f6693f3efaf75bd3946829ee- policy_evaluation_sha256
94bbf82474a26461f303406b30a671ef776c31cadbd229e627e114c538acffd1- promotion_policy_sha256
6714ca504b2c5679c0f1ba0c7da2e5ee52f61b70b0e814ecaf9071f080f9f61d- raw_predictions_sha256
ed1eac7db87bf2be0dbc9f94acf79a2f72cc9a5dc9dbb33e5935a0a5c8971c51- scanner_freeze_sha256
b21d5f9693cecf1d29554e4064de814c9b3d64a2c41ea9b446430cd0e68a0a67- scanner_validation_result_sha256
c182332ae5ec6829b68054f1575066c35bf89b8d088b65322d5798e1d8042192- validation_criteria_sha256
6c081fe020106093c25f85b40eb2960faee1bd88337ca3688f231573f06b36b5