Hardening our Claude skills: from prose to determinism

Joe McCormick · May 28, 2026

Most of the skills we write for Claude start life as a few lines of prose. That's the point — a SKILL.md is just a markdown recipe with a description that tells Claude when to use it. The barrier is intentionally low: "I keep telling Claude to do X" becomes "Claude does X" with almost no ceremony.

This post is about what happens next — when a quick personal skill earns its keep and you want it reliable enough for the whole team to depend on. The short version: leave a skill as prose while you're still figuring out whether it's worth keeping, and harden the mechanical parts into deterministic scripts once it goes stable. Here's the reasoning, with numbers.

A running example

Throughout, I'll use a skill I built for myself called slack-catch-up. It lives in my personal skills directory; nobody else runs it. Every weekday morning it pulls all my unread Slack, applies a few filters, sorts what's left into Action Required / Decisions / FYI, and drops a markdown digest into my workspace — with drafted replies for the messages that need one.

1. Early on, prose is the right shape

When a skill is new and personal — an audience of one, where the only cost of failure is your own time — all-prose is the natural shape. Open the file, write what you want done, save. That's the appeal.

Extracting scripts early isn't wrong, but it can cost you twice: once to write the script, and again when the skill's shape changes and you have to inline the logic back into prose so you can keep iterating quickly.

slack-catch-up started this way. The first version made inline calls to the Slack API in prose, had the model paginate the responses, parsed the raw JSON in context, then classified messages and drafted replies — all prose. It was slow, expensive per run, and occasionally inconsistent. But it ran end to end after a single evening of work, and it answered the only question that mattered at that stage: is this skill worth keeping?

2. Once it's stable, harden with determinism

When a skill graduates — promoted out of its work-in-progress state, or authored straight into a shared repo — the prose-only shape stops paying for itself. Two reasons:

Correctness. Asking the model to call an API in prose every run rolls the dice on argument shape, pagination, and edge-case parsing. Asking it to invoke a script that knows how to talk to the API gives the same result every run. That extraction pays off even when the data is small, purely because the work happens identically each time.

Cost. The runtime cost doesn't live in the SKILL.md — it lives in what the model carries across reasoning steps. A prose-only skill makes the model re-read raw API responses, intermediate JSON, and partial computations on every step. A skill that pushes the loud data path into a script reads one structured summary once and never sees the raw responses again. The instruction file barely changes size; the savings are in what isn't being re-carried.

So the hardened shape is two layers: a script that does the fragile mechanical work deterministically, and a prose SKILL.md that orchestrates and applies judgment. Concretely, an instruction like:

Read all my unread Slack messages from the last 24 hours. Group into DMs vs
channels. Sort oldest first, and prioritize ones where I was @-mentioned.

becomes:

Use `bin/fetch-slack-unreads` to load my unread messages.

Same model, same end-to-end task — very different runtime shape, and a guarantee that the API half happens the same way every time.

The proof

I measured two versions of slack-catch-up against the same inbox. V0 is all prose. V1 extracts exactly one script — the Slack ingest. Same digest format, same classification rules.

Version	Wall clock (median)	Cost per run
V0 — all prose	7m 26s	$2.71
V1 — Slack ingest extracted	2m 48s	$1.44

2.65× faster, 47% cheaper — about $1.27 saved every run, forever, for this one skill.

How much you'll save scales with how much intermediate data your skill carries across steps. A skill whose loud path is a single gh pr list sees a modest win. One paginating a Slack inbox or fanning out across many PRs sees something closer to 2–3×. If your skill reads tiny inputs and writes tiny outputs, the cost win is small — but the correctness win still applies.

3. If you're authoring straight to stable, skip the prose phase

The "iterate in prose, harden when it earns it" pattern is calibrated to the work-in-progress state, which exists to generate a signal you don't have yet: is this worth keeping? When you already have that signal — a team-wide setup helper, an oncall workflow, a shared review tool — the prose-only phase is just a tax. Write it hardened from the start: judgment in the SKILL.md, mechanics in code.

Most skills don't start at either extreme. They start personal, with one user, and either stay there or eventually graduate. That path is right when you're not yet sure. The "skip the prose phase" rule applies only when the audience is named up front.

4. Where the line goes

A two-question test: does this need judgment? → prose. Is it mechanical? → script.

Belongs in prose	Belongs in a script
Drafting (replies, summaries, plans)	API calls, pagination, fan-out
Bucketing and classification with judgment	Deterministic filtering and de-duplication
Recognizing edge cases (a "punt" reply, an ambiguous ask)	Timestamp formatting, permalink generation, ID resolution
Tone and voice matching	Parsing structured input
Choosing among multiple acceptable outputs	Repeated computation over large data

Most skills get a mix. The hardened slack-catch-up keeps drafting, punt detection, ambiguous bucketing, and channel-ignore suggestions in prose; everything else is in scripts.

5. What's next

A review pass across the skills we've already promoted to stable, looking for procedural prose that's load-bearing enough to deserve a script — and extracting it where the extraction pays. Skills still in their work-in-progress state aren't the target; the whole point of that stage is room to iterate.

To do that at scale I built a reviewer skill — determinism-review — that reads a SKILL.md, spots the candidate extraction, and explains why: each finding with a severity, supporting evidence, and a suggested extraction. It identifies; it doesn't rewrite. It's also wired in as a non-blocking advisory check on skill changes, softened to a heads-up where iteration is still the point.

Anthropic already ships a skill-reviewer agent that grades description quality, length, voice, and progressive-disclosure compliance — but it doesn't look at the script-vs-prose axis. determinism-review composes with it rather than replacing it. The expected output for most skills is small: one extracted script — the one wrapping the loud data path — and a SKILL.md that's measurably tighter.