CONTEXT(CLONE)
Sign in

Challenge the root cause of a pipeline incident

For: Data engineering lead writing the review after a failed or wrong pipeline run

Pattern: Adversarial reviewRuns todayDesigned for 4 to 40 agents
Get early accessSign in to use

The pain today

The first plausible cause gets written into the incident review. The run logs, change history and upstream notices that would contradict it are long, so nobody reads them all.

The ask

“I attached the pipeline run logs, the change log for that week, the upstream notices and my draft root cause. Build the evidence for my root cause, then try to break it with anything in these documents that points elsewhere. Show what survives.”

Plain words, as you would say it to a colleague. Edit it to fit your case before you send it.

What you attach or connect

  • Pipeline run logs as text
  • Change log for the period
  • Upstream provider notices
  • Draft root cause statement

The unit of work

One worker task per one log, change record or notice.

Why a swarm fits

Each document is read alone against one stated cause. A second team with the opposite brief reads the same documents, so contradicting evidence gets the same attention as confirming evidence.

Not for

Live incident response. It reviews supplied text after the event and cannot query systems or rerun jobs.

The decision tree

6 typed decisions, each with an action for every answer

At fixed moments in a run, the engine puts one narrow question to a decision model. The decision model never writes text: it answers yes or no with a probability, picks from listed options, or gives a score, about a small slice of the material. The engine then does exactly what this tree says, which is what makes the run auditable. The thresholds are the template's design values, not measured results.

  1. Planner, while planning

    Scope checkYes or no, with a probability

    Before work starts on a unit

    Does this log, change record or notice fall within the time window and systems the draft root cause refers to?

    Sees only: The draft root cause's window and systems, and one document's header and timestamps

    Why: Keeps both teams on records that can bear on the incident.

    • Yes: 0.60 or higherthenAccept
    • Unsure: 0.30 up to 0.60thenAccept
    • No: below 0.30thenSkip this unit

  2. After workers, the judge checks

    Evidence checkYes or no, with a probability

    After a worker answers

    Does the quoted log line or change record show the event the supporting team says happened, at the time they say it happened?

    Sees only: One statement of the root cause and the quoted record with its timestamp

    Why: A root cause written into a review should rest on records, not on the first plausible story.

    • Yes: 0.85 or higherthenAccept
    • Unsure: 0.50 up to 0.85thenEscalate to a strong model
    • No: below 0.50thenReject and retry

  3. Evidence checkYes or no, with a probability

    After a worker answers

    Does the quoted record show something that could not be true if this statement of the root cause were correct?

    Sees only: One statement of the root cause and the record the breaking team quoted against it

    Why: Only a genuine contradiction may break the cause; loosely related records may not.

    • Yes: 0.85 or higherthenAccept
    • Unsure: 0.50 up to 0.85thenMark unresolved
    • No: below 0.50thenReject and retry

  4. Reconciler, while merging

    Conflict checkYes or no, with a probability

    While reconciling

    Do the timestamps show the claimed cause occurring after the first sign of the failure?

    Sees only: The timestamped record for the claimed cause and the first failing log line

    Why: A cause that follows its effect is the simplest disproof and is checked explicitly.

    • Yes: 0.70 or higherthenMark unresolved
    • Unsure: 0.30 up to 0.70thenEscalate to a strong model
    • No: below 0.30thenAccept

  5. Run control, between rounds

    Another round?Yes or no, with a probability

    Between rounds

    Did the last round produce any accepted record, for or against, that had not been cited before?

    Sees only: Accepted records per round for each statement

    Why: Ends the challenge when the documents have nothing new to say.

    • Yes: 0.60 or higherthenContinue
    • Unsure: 0.30 up to 0.60thenStop
    • No: below 0.30thenStop

  6. Accountable person, before anything is settled

    Person decidesYes or no, with a probability

    Before anything is reported as settled

    Does the result break or leave undecided any statement of the draft root cause, or point to a rival cause?

    Sees only: The list of surviving, broken and untested statements and rival causes

    Why: The data engineering lead signs the review and chooses corrective actions.

    Accountable: The data engineering lead signs the incident review and decides the corrective actions.

    • Yes: 0.30 or higherthenAsk a person
    • Unsure: 0.10 up to 0.30thenAsk a person
    • No: below 0.10thenAccept

The fleet: who does what

Model tiers by role, not brands: you choose the models. Strong reasoning models plan and reconcile, small fast models do the wide work, and the judge is a decision model from a different family, so it does not share the workers' blind spots.

  1. Planner

    A strong reasoning model turns the draft root cause into statements the documents can confirm or contradict.

    Decisions here:1. Scope check

  2. Workers

    Small fast workers in two teams: one cites evidence for each statement, the other cites evidence against.

    Designed for 4 to 40 agents, one worker task per one log, change record or notice. Each worker receives only its own unit.

  3. Judge, from a different model family

    A decision model from a different family decides whether each statement survives on the quoted evidence.

    Decisions here:2. Evidence check3. Evidence check

  4. Reconciler

    A strong reasoning model reports what survives, what breaks and which rival cause the evidence favours.

    Decisions here:4. Conflict check5. Another round?

  5. Accountable person

    The data engineering lead signs the incident review and decides the corrective actions.

    Decisions here:6. Person decides

Checked before anything is accepted

  • Every point for or against quotes a log line, change record or notice with its timestamp
  • Timeline order is checked: a cause cannot follow its effect
  • The breaking team never sees the supporting team's reasoning
  • Statements no document speaks to are reported as untested

What comes back

  • Statements of the root cause that survive, with evidence
  • Statements broken by contradicting evidence
  • Rival causes the documents point to
  • Evidence that was missing and would settle the question

What to measure

  • Root causes revised after the challenge
  • Repeat incidents traced to a wrong earlier root cause
  • Engineer hours per incident review
  • Cost per review compared with one strong model alone

Names of measures only. No result is claimed for this template.

Templates open in the workspace chat with the ask filled in. Nothing runs until you send it.

Get early accessSign in to use

More in Data and analytics

Find conflicting definitions across the data dictionary

For: Data governance lead or analytics engineering manager

Active customer, net revenue and churn are each defined several ways across tables, models and dashboards.

Pattern: Map, verify, reduceNeeds scale6 decisionsDesigned for 40 to 1,500 agents