CONTEXT(CLONE)
Sign in

Review a new certified dataset before release

For: Head of analytics or data product owner approving a new certified dataset

Pattern: Specialist panelNeeds live modelsDesigned for 5 to 50 agents
Get early accessSign in to use

The pain today

A certified dataset needs sign-off on definition, statistical soundness, privacy classification, lineage and access. Reviews happen in sequence, and each reviewer sees only a summary.

The ask

“I attached the specification, SQL, data classification policy and access policy for our new customer lifetime value dataset. Review it for business definition, statistical soundness, privacy classification, lineage and freshness, and access rules. Quote the text behind each concern.”

Plain words, as you would say it to a colleague. Edit it to fit your case before you send it.

What you attach or connect

  • Dataset specification
  • SQL and transformation code
  • Data classification policy
  • Access policy
  • Lineage description

The unit of work

One worker task per one specification or code section per specialist brief.

Why a swarm fits

Five briefs read the same material against different policies, and none needs the others' checklist. The panel runs in parallel and the reconciler shows where two briefs conflict.

Not for

A small change to an existing dataset. One reviewer reading the diff is enough.

The decision tree

5 typed decisions, each with an action for every answer

At fixed moments in a run, the engine puts one narrow question to a decision model. The decision model never writes text: it answers yes or no with a probability, picks from listed options, or gives a score, about a small slice of the material. The engine then does exactly what this tree says, which is what makes the run auditable. The thresholds are the template's design values, not measured results.

  1. Before workers, before a task runs

    Small worker or strong modelA choice among options

    Before a task runs

    Which treatment does this specification or code section need from the privacy, statistics, lineage, definition or access brief it is paired with?

    Sees only: One section's heading and first lines, and one brief's scope

    Why: Statistical and privacy reasoning over code goes to the stronger model; the rest stays cheap.

    • In scope, small workerthenAccept
    • In scope, needs strong modelthenEscalate to a strong model
    • Out of scope for this briefthenSkip this unit

  2. After workers, the judge checks

    Evidence checkYes or no, with a probability

    After a worker answers

    Does the quoted specification or SQL text contain the assumption, column or rule that the specialist's concern says it contains?

    Sees only: One specialist concern and the specification or SQL text it quotes

    Why: Release reviews stall when a concern turns out to misread the code.

    • Yes: 0.85 or higherthenAccept
    • Unsure: 0.50 up to 0.85thenEscalate to a strong model
    • No: below 0.50thenReject and retry

  3. Reconciler, while merging

    Conflict checkA choice among options

    While reconciling

    Does the access rule for this column grant access wider than the classification policy allows for the column's stated classification?

    Sees only: One column's classification, the policy rule for that class and its access rule

    Why: Mismatched classification and access is the release defect with the highest cost.

    • Access wider than policythenMark unresolved
    • Access within policythenAccept
    • Column has no classificationthenAsk a person

  4. Conflict checkYes or no, with a probability

    While reconciling

    Do these two specialist concerns about the same section call for changes that work against each other?

    Sees only: Two accepted concerns from different briefs on one section

    Why: Pairs opposing advice so the owner sees the trade-off.

    • Yes: 0.70 or higherthenMark unresolved
    • Unsure: 0.30 up to 0.70thenMark unresolved
    • No: below 0.30thenAccept

  5. Accountable person, before anything is settled

    Person decidesYes or no, with a probability

    Before anything is reported as settled

    Does any accepted concern involve personal data, access rights or a statistical assumption that changes what the dataset's figures mean?

    Sees only: The accepted concerns for the dataset, grouped by brief

    Why: The data product owner and privacy officer approve release; the panel approves nothing.

    Accountable: The data product owner and the privacy officer approve release; the panel approves nothing.

    • Yes: 0.30 or higherthenAsk a person
    • Unsure: 0.10 up to 0.30thenAsk a person
    • No: below 0.10thenAccept

The fleet: who does what

Model tiers by role, not brands: you choose the models. Strong reasoning models plan and reconcile, small fast models do the wide work, and the judge is a decision model from a different family, so it does not share the workers' blind spots.

  1. Planner

    A strong reasoning model writes the five briefs and assigns the sections and policies each must read.

  2. Workers

    Small fast workers from mixed families, one per section and brief, reporting only against their own checklist.

    Designed for 5 to 50 agents, one worker task per one specification or code section per specialist brief. Each worker receives only its own unit.

    Decisions here:1. Small worker or strong model

  3. Judge, from a different model family

    A decision model from a different family checks that each concern is anchored in quoted specification, code or policy.

    Decisions here:2. Evidence check

  4. Reconciler

    A strong reasoning model merges concerns per section and pairs the ones that contradict each other.

    Decisions here:3. Conflict check4. Conflict check

  5. Accountable person

    The data product owner and the privacy officer approve release; the panel approves nothing.

    Decisions here:5. Person decides

Checked before anything is accepted

  • Each concern quotes the specification, SQL or policy text it rests on
  • Privacy concerns name the column and the classification rule together
  • Statistical concerns state the assumption in the specification they question
  • Concerns without a text anchor are dropped and counted

What comes back

  • Concerns per section, grouped by specialist area
  • Columns whose classification and access rule do not match
  • Contradicting concerns, paired
  • Questions for the dataset owner before release

What to measure

  • Share of concerns the human reviewers keep
  • Issues found after release that the panel missed
  • Elapsed time of the release review
  • Cost per dataset reviewed

Names of measures only. No result is claimed for this template.

Templates open in the workspace chat with the ask filled in. Nothing runs until you send it.

Get early accessSign in to use

More in Data and analytics

Find conflicting definitions across the data dictionary

For: Data governance lead or analytics engineering manager

Active customer, net revenue and churn are each defined several ways across tables, models and dashboards.

Pattern: Map, verify, reduceNeeds scale6 decisionsDesigned for 40 to 1,500 agents