Test case tournament for a module that must not break

The decision tree

6 typed decisions, each with an action for every answer

At fixed moments in a run, the engine puts one narrow question to a decision model. The decision model never writes text: it answers yes or no with a probability, picks from listed options, or gives a score, about a small slice of the material. The engine then does exactly what this tree says, which is what makes the run auditable. The thresholds are the template's design values, not measured results.

1. 1

   Planner, while planning

   Scope checkYes or no, with a probability

   Before work starts on a unit

   Does this candidate test exercise a rule or boundary that the specification sentence it cites actually states?

   Sees only: One candidate test and the specification sentence it cites

   Why: Removes tests of invented requirements before any judge or pipeline time is spent.

   00.30 and 0.601

   • Yes: 0.60 or higher→thenAccept
   • Unsure: 0.30 up to 0.60→thenAccept
   • No: below 0.30→thenSkip this unit
2. 2

   Reconciler, while merging

   Conflict checkYes or no, with a probability

   While reconciling

   Do these two candidate tests check the same rule with the same boundary values, differing only in naming or setup?

   Sees only: Two candidate tests side by side

   Why: Generated piles are mostly repeats; collapsing them early keeps later rounds cheap.

   00.40 and 0.801

   • Yes: 0.80 or higher→thenSkip this unit
   • Unsure: 0.40 up to 0.80→thenContinue
   • No: below 0.40→thenContinue
3. 3

   After workers, the judge checks

   Evidence checkA choice among options

   After a worker answers

   According to the pipeline output, how does this candidate behave on the current code and on the deliberately broken variant?

   Sees only: The pipeline output for one candidate on current code and on the broken variant

   Why: A test that cannot fail protects nothing, and a test failing today may be a real defect.

   • Passes now and fails on the broken variant→thenAccept
   • Fails on current code→thenAsk a person
   • Passes on both, so it cannot fail→thenSkip this unit
   • Run errored or timed out→thenReject and retry
4. 4

   Evidence checkA score

   After a worker answers

   How strongly does the cited specification sentence require the exact behaviour this test asserts?

   Sees only: The test's assertion and the cited specification sentence

   Why: Keeps tests that encode a judge's guess about the rules out of a suite people will trust.

   00.50 and 0.851

   • High: 0.85 or higher→thenAccept
   • Middle: 0.50 up to 0.85→thenEscalate to a strong model
   • Low: below 0.50→thenSkip this unit
5. 5

   Run control, between rounds

   Another round?Yes or no, with a probability

   Between rounds

   Did the last round add a surviving case for any rule or boundary that had none before?

   Sees only: The table of rules against surviving cases, before and after the round

   Why: Ends the tournament when more candidates only repeat what is already covered.

   00.30 and 0.601

   • Yes: 0.60 or higher→thenContinue
   • Unsure: 0.30 up to 0.60→thenStop
   • No: below 0.30→thenStop
6. 6

   Accountable person, before anything is settled

   Person decidesYes or no, with a probability

   Before anything is reported as settled

   Does this surviving case fail on the current code, or assert behaviour that the specification sentence leaves open to two readings?

   Sees only: One surviving case, its run result and the specification sentence

   Why: Only the module owner can say whether that is a defect or a wrong reading of the rules.

   Accountable: The module owner decides which cases enter the suite and whether a failing case is a defect or a wrong reading of the specification.

   00.15 and 0.401

   • Yes: 0.40 or higher→thenAsk a person
   • Unsure: 0.15 up to 0.40→thenAsk a person
   • No: below 0.15→thenAccept