Root-cause hypotheses tested against log windows
For: Site reliability engineer or incident commander writing the review after an outage
The pain today
After an outage the first plausible story wins. Logs from many services are too long to read in full, so the conclusion rests on the few windows someone happened to open.
The ask
“I attached the exported logs and deploy history from yesterday's outage and our draft timeline. The explanations people have suggested are listed in the notes file. Test each one against the logs, show me what supports and what contradicts it, and tell me honestly if none of them holds.”
Plain words, as you would say it to a colleague. Edit it to fit your case before you send it.
What you attach or connect
- Exported logs per service, as text
- Deploy and configuration change history
- Draft timeline and alert history
- List of candidate explanations
The unit of work
One worker task per one hypothesis against one service log window.
Why a swarm fits
Each window is a small slice, and each hypothesis makes a checkable prediction about it. Two families reading the same window independently stop one persuasive story from steering every reader.
Not for
A crash with a stack trace pointing at one line: one engineer with the trace is faster.
The decision tree
6 typed decisions, each with an action for every answer
At fixed moments in a run, the engine puts one narrow question to a decision model. The decision model never writes text: it answers yes or no with a probability, picks from listed options, or gives a score, about a small slice of the material. The engine then does exactly what this tree says, which is what makes the run auditable. The thresholds are the template's design values, not measured results.
Planner, while planning
Scope checkYes or no, with a probability
Before work starts on a unit
Does this log window cover the service and the minutes in which the hypothesis predicts something observable?
Sees only: One prediction and the window's service name and time range
Why: Pairs with nothing to observe are dropped before two readers are paid to read them.
- Yes: 0.60 or higherthenAccept
- Unsure: 0.25 up to 0.60thenAccept
- No: below 0.25thenSkip this unit
After workers, the judge checks
Evidence checkA choice among options
After a worker answers
Do the quoted log lines show the predicted event, such as the connection pool running out, before the first user-facing error rather than after it?
Sees only: The quoted lines with timestamps and the time of the first user-facing error
Why: Order in time is what separates a cause from a symptom.
- Before it: supports the hypothesisthenAccept
- Only after it: contradicts the hypothesisthenAccept
- Absent from a complete windowthenMark unresolved
- Window has gaps at the critical timethenMark unresolved
Reconciler, while merging
Conflict checkYes or no, with a probability
While reconciling
Do the two readers give opposite verdicts on the same prediction for the same window?
Sees only: Two verdicts with their quoted lines for one prediction and window
Why: Disagreement between families is the signal that one persuasive story is steering a reader.
- Yes: 0.60 or higherthenEscalate to a strong model
- Unsure: 0.30 up to 0.60thenEscalate to a strong model
- No: below 0.30thenAccept
Run control, between rounds
Retry or stopA choice among options
After a rejection or low confidence
Reading both readers' quoted lines together, which verdict do the timestamps support?
Sees only: Both readers' quoted lines, merged in time order
Why: Settles only what the log lines settle and leaves the rest visibly open.
- The supporting reader's lines holdthenAccept
- The contradicting reader's lines holdthenAccept
- The lines are consistent with boththenMark unresolved
- Quoted lines are not in the windowthenReject and retry
Another round?Yes or no, with a probability
Between rounds
Does any hypothesis still have untested predictions in windows where logs exist?
Sees only: The table of hypotheses, predictions and tested windows
Why: Stops the run when further reading cannot change how the hypotheses rank.
- Yes: 0.50 or higherthenContinue
- Unsure: 0.20 up to 0.50thenContinue
- No: below 0.20thenStop
Accountable person, before anything is settled
Person decidesYes or no, with a probability
Before anything is reported as settled
Is this hypothesis about to be reported as the best supported cause while any of its predictions is contradicted or unresolved?
Sees only: One hypothesis with the verdict for each of its predictions
Why: The incident commander owns the stated root cause and must see the weak points first.
Accountable: The incident commander owns the stated root cause and the follow-up actions; the swarm reports evidence, not blame.
- Yes: 0.30 or higherthenAsk a person
- Unsure: 0.10 up to 0.30thenAsk a person
- No: below 0.10thenAccept
The fleet: who does what
Model tiers by role, not brands: you choose the models. Strong reasoning models plan and reconcile, small fast models do the wide work, and the judge is a decision model from a different family, so it does not share the workers' blind spots.
Planner
A strong reasoning model turns each hypothesis into predictions per service and time window.
Decisions here:1. Scope check
Workers
Paired small workers from two different families test the same prediction on the same window, blind to each other.
Designed for 20 to 500 agents, one worker task per one hypothesis against one service log window. Each worker receives only its own unit.
Judge, from a different model family
A decision model from a third family settles disagreements from the quoted lines, or marks them unresolved.
Decisions here:2. Evidence check
Reconciler
A strong reasoning model weighs supported and contradicted predictions per hypothesis and states what stays unexplained.
Decisions here:3. Conflict check4. Retry or stop5. Another round?
Accountable person
The incident commander owns the stated root cause and the follow-up actions; the swarm reports evidence, not blame.
Decisions here:6. Person decides
Checked before anything is accepted
- Supporting and contradicting verdicts both quote log lines with timestamps
- Pairs that disagree go to the judge; unresolved stays unresolved
- Windows with missing logs are reported as gaps
- A hypothesis with a firm contradiction is never reported as the cause
What comes back
- Each hypothesis with supporting and contradicting evidence
- Timeline corrections with cited lines
- Events no hypothesis explains
- Log gaps that limit the conclusion
What to measure
- Review conclusions later revised
- Contradicting evidence found that the draft missed
- Hours to a defensible review
- Disagreement rate between worker families
Names of measures only. No result is claimed for this template.
Templates open in the workspace chat with the ask filled in. Nothing runs until you send it.
Get early accessSign in to useMore in Security and IT operations
A night of alerts triaged into incidents with evidence
For: Security operations lead or on-call analyst starting the morning shift
Overnight the queue fills with alerts, most of them duplicates or noise.
Threat model attacked before the design is built
For: Security architect or product security engineer reviewing a new system design
Threat models are written by the people who designed the system, so they list the threats the design already handles.
Access recertification across every account and role
For: IT security manager or identity owner running the periodic access review
Managers receive a spreadsheet of entitlements and approve all of it, because nobody can judge that many rows.