Distilling knowledge for Don Nelson

There's one idea running through everything I've built: an agent's reliability doesn't live in the model, it lives in the harness. The model brings fluency. The harness brings knowledge, contracts and guardrails.

This post is about a key piece of that harness: the rules I distilled from the CEN's review history. Probably the most boring to explain, and the hardest to get. And along the way I'll take the chance to talk about the term that gives this post its name: distilling.

For Don Nelson to review a Power Flow Study, I tell it what a reviewer actually checks: what they look at, what they verify, what they contrast it against.

The checklist isn't the criterion

Chile's grid operator publishes review checklists for power flow and short-circuit studies, and the PGP user guide describes the procedure. I read them cover to cover. They tell you what gets delivered — "Report delivered", "PowerFactory database delivered" — and almost nothing about what gets flagged.

Between "the study shall correctly model the equipment" and "the line's zero-sequence impedance doesn't match the manufacturer datasheet" there is an enormous gap. That gap is the reviewer's job.

The criterion does exist, but it lives in two places: in the heads of people who have been reviewing for years, and — this is what saved me — in the review documents the operator issues study by study. Every observation is a rule applied, with the concrete case attached. I don't know what gets reviewed because someone wrote it down: I know it because I know what gets flagged.

Where I got it

I built a corpus from the CEN's reviews of power flow studies: 223 studies, 571 review iterations and 1,797 items — 1,180 observations, 640 considerations and 11 recommendations — snapshotted on May 20, 2026.

And something that wasn't in the original plan: the third-party observations. The companies that own the connection point review in parallel, and turned out to be as tough a source as the operator itself.

The pipeline is unglamorous: download the review PDFs, extract the text, split it item by item, and classify each item with a model along four axes — type (observation / consideration / recommendation), category, severity and required action (fix, clarify, verify, add case, update data, redo). Then group the items that say the same thing in different words and keep the rule underneath.

That's the whole funnel: 2,168 real observations reducing to 1,468 codable checks and from there to 48 unique rules. Forty-eight. That's what you need to know to review a power flow study in Chile.

What the corpus says

Before the rules, three things that changed how I think about this.

First: only 15.9% of studies pass clean. The rest come back with observations or considerations. The median review takes 24 days.

Second: 82.8% of studies get third-party observations. The connection point owner reviews with a magnifying glass, and flags different things than the operator. They know their own installation better than anyone, and they notice when the model doesn't represent it.

Third, and the one that surprised me most: what gets flagged most isn't hard engineering. It's incorrect input data, with 297 items. More than line overload, unmet N-1 criterion and out-of-range voltage drop combined.

Almost none of what gets flagged is the power system behaving badly. It's the number in the report not matching the number in the official source.

A reviewer doesn't review "the study". They review consistency across sources.

The rules that carry the weight

Of the 48, these eight explain most of the volume. I'm writing them the way an engineer would say them, not the way the standard does:

Notice the pattern. Six of the eight are verifiable against an external source: InfoTécnica, the manufacturer datasheet, the Scenario Letter, the works schedule, the simulation database itself. Almost none of them need engineering judgment to detect — judgment comes afterwards, to decide how serious it is.

From rules to harness capabilities

The 48 rules aren't implemented one by one. They cluster into six capabilities, and each capability carries a weight: how many real observations from the corpus it would have caught.

That weight is what turns the corpus into an engineering tool: the agent's coverage stops being declared and starts being measured. I can say what share of the CEN's real observations Don Nelson would have caught with what's already built — validating parameters against InfoTécnica, cross-checking the Scenario Letter — and what share depends on re-simulating in PowerFactory, which is Spark's job.

What's missing gets measured too. Document consistency and database version are still out of Don Nelson's reach, and now I know exactly what they cost: 274 real observations that would slip by today.

The 48 rules

Here they all are, grouped by the capability that covers them. It's public knowledge distilled from public documents, so there's not much point in keeping it to myself:

Being honest about it

The 48 rules are my synthesis, not an official document: nobody published this and nobody is going to validate it. They're built from the public checklists, the applicable regulation and the observation history; the corpus runs through May 2026 and covers power flow — short-circuit is another pass. Each item was classified by a model, with manual spot-checking, so the categories carry noise.

Even so, it's the best I have, and it's infinitely better than the alternative — which was asking the agent to review a study using instructions I wrote from memory. The harness now carries an answer key that can be checked against reality: every time Don Nelson raises a finding, there's a corpus of 2,168 real observations to ask whether a human reviewer would have raised it too.

That, in the end, is all a good harness does: turn knowledge that lived in someone's head into something you can run, measure and correct.