Research Round 41 · 2026-10-30
Hard by Construction or Hard by Selection? Disagreement Mined Item Pools, Isomorphic Twin Holdouts, and the Consensus Errors That Disagreement Cannot Find
Research question
Is selecting items on which witnesses disagree a sound way to build the hard pool that separates composition topologies, or does it overstate how much composition helps and leave consensus errors exactly where it cannot look?
Summary
Tests whether building a hard pool by selecting items on which witnesses disagree, as the Round 40 handoff proposed, is sound. Each authored item is written as two isomorphic twins with an identical rule trace and verdict, one used to screen and one held out. An exact diagnostic (E41-A, 0 model calls) shows disagreement selection keeps 42.2 percent of items, raises the forced majority's wrong release rate from 0.139 on the full pool to 0.285 in sample, and more than doubles its measured gap over one witness (0.055 against 0.023), a gap that falls to 0.028 on a fresh twin; 13.4 percent of forced majority errors sit on items disagreement never selects. Live, 200 calls with Witness A (Max reasoning), Witness B, Witness C, and Witness D over 10 hard by construction families (E41-B) and 20 fixed real SEC EDGAR exhibits (E41-C) found 0 of 10 families with any disagreement and 40/40 stable witness twin pairs. The only authored error was unanimous (F07, a missing delivery fact) and only the disagreement routed hybrid held it; the only exhibit disagreements (Y11 and Y13, both effective only on execution and delivery or receipt of all counterparts) were released by the forced majority on a tie and held by the quorum and the resolver. Adds proposition P6, disagreement is a biased detector; no production system changed.
Major findings
- F41.1 (deterministic, E41-A): disagreement selection keeps 42.2 percent of items and doubles the share of hard and trap items; the forced majority's wrong release rate is 0.285 in sample against 0.139 on the full pool.
- F41.2 (deterministic, E41-A): the measured forced majority advantage over one witness is 0.055 in sample against 0.023 on the full pool and 0.028 on a perfect twin holdout.
- F41.3 (deterministic, E41-A): 13.4 percent of forced majority errors lie on items disagreement never selects, and a 3-of-4 quorum releases all of them.
- F41.4 (live, E41-B): 0 of 10 hard by construction families produced any screening disagreement; each witness scored 9/10 on both twins and 40/40 witness twin pairs were stable.
- F41.5 (live, E41-B): all four witnesses released PASS on both F07 twins when the decisive delivery fact was absent; only the disagreement routed hybrid held it, at the price of one false hold on F03E caused by a joined quote.
- F41.6 (live, E41-C): on 20 real exhibits two disagreements occurred (Y11 and Y13, both execution only, each split Witness A and Witness C against Witness B and Witness D); the forced majority released both on a tie and the quorum and resolver held both. The other 18 exhibits were unanimous and correct.
Experiments
- E41-A Exact disagreement selection enumeration with twin holdouts Executed (diagnostic)
- E41-B Live isomorphic twin screen and holdout on authored families Executed (diagnostic)
- E41-C Live screen on real SEC exhibits chosen for surface traps Executed
Hypotheses
- H41.1 Items selected for disagreement show lower accuracy in sample than on held out twins, for every topology.
- H41.2 Selection for disagreement inflates the measured gap between topologies.
- H41.3 Families authored to be hard by construction produce screening disagreement on at least some families.
- H41.4 Each witness gives the same verdict on both twins of a family.
- H41.5 Disagreement selection misses consensus errors that fall on items where all witnesses agree.
Architecture implications
Research diagnostic and live benchmark only. Recommends building Noetis evaluation pools by construction with held out twins and treating unanimous items as the place to look for consensus errors. No production system, release gate, or Noetis behavior changed.
Open questions
- On a pool built by construction, with undetermined fact items labelled HOLD and PASS and scored on held out twins, is the joint error of one witness in direct and extraction roles lower than that of two different witnesses in the same role?
Connected research
Full experimental notes, prompts, internal architecture details and implementation procedures are kept by Encevra SI Research Labs.
