Root-cause hypotheses tested against log windows

For: Site reliability engineer or incident commander writing the review after an outage

Pattern: Cross-examinationNeeds scaleDesigned for 20 to 500 agents

The pain today

After an outage the first plausible story wins. Logs from many services are too long to read in full, so the conclusion rests on the few windows someone happened to open.

The ask

I attached the exported logs and deploy history from yesterday's outage and our draft timeline. The explanations people have suggested are listed in the notes file. Test each one against the logs, show me what supports and what contradicts it, and tell me honestly if none of them holds.

Plain words, as you would say it to a colleague. Edit it to fit your case before you send it.

What you attach or connect

  • Exported logs per service, as text
  • Deploy and configuration change history
  • Draft timeline and alert history
  • List of candidate explanations

The unit of work

One worker task per one hypothesis against one service log window.

Why a swarm fits

Each window is a small slice, and each hypothesis makes a checkable prediction about it. Two families reading the same window independently stop one persuasive story from steering every reader.

Not for

A crash with a stack trace pointing at one line: one engineer with the trace is faster.

The decision tree

6 typed decisions, each with an action for every answer

At fixed moments in a run, the engine puts one narrow question to a decision model. The decision model never writes text: it answers yes or no with a probability, picks from listed options, or gives a score, about a small slice of the material. The engine then does exactly what this tree says, which is what makes the run auditable. The thresholds are the template's design values, not measured results.

  1. Planner, while planning

    Scope checkYes or no, with a probability

    Before work starts on a unit

    Does this log window cover the service and the minutes in which the hypothesis predicts something observable?

    Sees only: One prediction and the window's service name and time range

    Why: Pairs with nothing to observe are dropped before two readers are paid to read them.

    • Yes: 0.60 or higherthenAccept
    • Unsure: 0.25 up to 0.60thenAccept
    • No: below 0.25thenSkip this unit
  2. After workers, the judge checks

    Evidence checkA choice among options

    After a worker answers

    Do the quoted log lines show the predicted event, such as the connection pool running out, before the first user-facing error rather than after it?

    Sees only: The quoted lines with timestamps and the time of the first user-facing error

    Why: Order in time is what separates a cause from a symptom.

    • Before it: supports the hypothesisthenAccept
    • Only after it: contradicts the hypothesisthenAccept
    • Absent from a complete windowthenMark unresolved
    • Window has gaps at the critical timethenMark unresolved
  3. Reconciler, while merging

    Conflict checkYes or no, with a probability

    While reconciling

    Do the two readers give opposite verdicts on the same prediction for the same window?

    Sees only: Two verdicts with their quoted lines for one prediction and window

    Why: Disagreement between families is the signal that one persuasive story is steering a reader.

    • Yes: 0.60 or higherthenEscalate to a strong model
    • Unsure: 0.30 up to 0.60thenEscalate to a strong model
    • No: below 0.30thenAccept
  4. Run control, between rounds

    Retry or stopA choice among options

    After a rejection or low confidence

    Reading both readers' quoted lines together, which verdict do the timestamps support?

    Sees only: Both readers' quoted lines, merged in time order

    Why: Settles only what the log lines settle and leaves the rest visibly open.

    • The supporting reader's lines holdthenAccept
    • The contradicting reader's lines holdthenAccept
    • The lines are consistent with boththenMark unresolved
    • Quoted lines are not in the windowthenReject and retry
  5. Another round?Yes or no, with a probability

    Between rounds

    Does any hypothesis still have untested predictions in windows where logs exist?

    Sees only: The table of hypotheses, predictions and tested windows

    Why: Stops the run when further reading cannot change how the hypotheses rank.

    • Yes: 0.50 or higherthenContinue
    • Unsure: 0.20 up to 0.50thenContinue
    • No: below 0.20thenStop
  6. Accountable person, before anything is settled

    Person decidesYes or no, with a probability

    Before anything is reported as settled

    Is this hypothesis about to be reported as the best supported cause while any of its predictions is contradicted or unresolved?

    Sees only: One hypothesis with the verdict for each of its predictions

    Why: The incident commander owns the stated root cause and must see the weak points first.

    Accountable: The incident commander owns the stated root cause and the follow-up actions; the swarm reports evidence, not blame.

    • Yes: 0.30 or higherthenAsk a person
    • Unsure: 0.10 up to 0.30thenAsk a person
    • No: below 0.10thenAccept

The fleet: who does what

Model tiers by role, not brands: you choose the models. Strong reasoning models plan and reconcile, small fast models do the wide work, and the judge is a decision model from a different family, so it does not share the workers' blind spots.

  1. Planner

    A strong reasoning model turns each hypothesis into predictions per service and time window.

    Decisions here:1. Scope check

  2. Workers

    Paired small workers from two different families test the same prediction on the same window, blind to each other.

    Designed for 20 to 500 agents, one worker task per one hypothesis against one service log window. Each worker receives only its own unit.

  3. Judge, from a different model family

    A decision model from a third family settles disagreements from the quoted lines, or marks them unresolved.

    Decisions here:2. Evidence check

  4. Reconciler

    A strong reasoning model weighs supported and contradicted predictions per hypothesis and states what stays unexplained.

    Decisions here:3. Conflict check4. Retry or stop5. Another round?

  5. Accountable person

    The incident commander owns the stated root cause and the follow-up actions; the swarm reports evidence, not blame.

    Decisions here:6. Person decides

Checked before anything is accepted

  • Supporting and contradicting verdicts both quote log lines with timestamps
  • Pairs that disagree go to the judge; unresolved stays unresolved
  • Windows with missing logs are reported as gaps
  • A hypothesis with a firm contradiction is never reported as the cause

What comes back

  • Each hypothesis with supporting and contradicting evidence
  • Timeline corrections with cited lines
  • Events no hypothesis explains
  • Log gaps that limit the conclusion

What to measure

  • Review conclusions later revised
  • Contradicting evidence found that the draft missed
  • Hours to a defensible review
  • Disagreement rate between worker families

Names of measures only. No result is claimed for this template.

Templates open in the workspace chat with the ask filled in. Nothing runs until you send it.

Get early accessSign in to use

A night of alerts triaged into incidents with evidence

For: Security operations lead or on-call analyst starting the morning shift

Overnight the queue fills with alerts, most of them duplicates or noise.

Pattern: Map, verify, reduceNeeds a connector6 decisionsDesigned for 30 to 1,000 agents

Threat model attacked before the design is built

For: Security architect or product security engineer reviewing a new system design

Threat models are written by the people who designed the system, so they list the threats the design already handles.

Pattern: Adversarial reviewNeeds live models5 decisionsDesigned for 6 to 60 agents

Access recertification across every account and role

For: IT security manager or identity owner running the periodic access review

Managers receive a spreadsheet of entitlements and approve all of it, because nobody can judge that many rows.

Pattern: Map, verify, reduceNeeds scale5 decisionsDesigned for 40 to 1,000 agents