The problem​

I had a working plan for distributing small content packages (call them "skills" — self-contained bundles of docs + reference files an AI coding assistant can load) three ways at once: an existing symlink-based install, a plugin marketplace, and versioned npm packages. The plan went through several rounds of review before any code got written — a planner drafts, an architect and a critic both try to find holes, and the planner revises until nobody can find another one.

Three of those review rounds each found a real bug. Not a style nit, not a "have you considered" — an actual, reproducible defect that would have shipped broken artifacts to a public package registry. What made them worth trusting wasn't that the reviewers were thorough on paper. It's that they stopped taking the plan's word for anything and started running it.

What I tried​

The review discipline was simple to state and expensive to follow: every claim about how a third-party tool behaves gets verified against the real tool, in a scratch environment, before it's allowed into the plan.

Round one: the plan assumed a version-bump-and-publish tool would shell out to one specific packaging command regardless of context. A reviewer built a real workspace, ran the real publish path, and watched it call a different command than the plan assumed — one with different file-exclusion rules. The plan's own validation step would have silently passed while the real artifact shipped broken.

Round two: the plan piped a multi-command shell chain into a CI action's input string. A reviewer traced the action's source, found it spawns processes without a shell, and reproduced the exact failure — the chain silently truncated after the first command and the job reported success anyway.

# what the plan assumed (works in a terminal):
"version && sync-metadata && install --lockfile-only"

# what actually happens with no shell — everything after && is dropped,
# and the tool that received it still exits 0

Round three: a version-consistency rule was supposed to stop private packages from silently drifting ahead of what's actually published. It worked — until a reviewer noticed the fix itself had a timing bug: it wrote the corrected version to source control one step after the artifact was already packed and pushed to the registry. Structurally correct, sequenced wrong.

None of these were things you'd catch by reading the diff carefully. All three needed someone to actually spin up the dependency, feed it the exact input the plan specified, and watch what came back.

What I learned​

• "I read the source and I'm confident" is a different claim from "I ran it and confirmed." The gap between them is exactly where these three bugs lived.
• A review that only critiques the plan's logic will pass a plan whose logic is sound and whose premises are wrong. All three bugs were premise bugs — "this tool behaves like X" — not reasoning bugs.
• Give reviewers permission to be adversarial, and give them a scratch environment to be adversarial in. A review that can only read text argues about what's plausible. A review that can spin up the real dependency argues about what's true.
• The fix for a caught bug deserves the same scrutiny as the original plan. The version-timing bug above was found on the second review pass, in a fix for something the first pass had already caught — verification has to be recursive, not one-and-done.
• Rolling the same pattern out to a second, third, and fourth real environment surfaces the assumptions a single environment let you get away with. The first repo this pattern shipped to was clean. The next several each had at least one thing the first one never exercised — a different tool version, an existing pipeline with conflicting conventions, content that had never been validated before. None of those were bugs in the pattern; they were gaps in how much of the real world the first pass had touched.

Takeaways​

• Treat "I'm confident this is how the tool works" as a hypothesis, not a fact — verify it against the real tool before it becomes a load-bearing assumption in a plan.
• When you can afford it, give a review pass a disposable environment and explicit permission to try to break the plan for real, not just to critique it on paper.
• A bug found in round N doesn't retire the need for round N+1 to re-verify — fixes have premises too.
• The cheapest time to find a wrong assumption about a third-party tool is before you've built five things on top of it, not after.

The problem​

We had a proven pattern: AWS email infrastructure, provisioned by Terraform, synced into a hosted auth provider's config, with branded transactional emails compiled from templates and pushed the same way. It worked, in production, across more than a dozen internal projects. It had just never been turned into something a new project could pick up and use — it was tribal knowledge, copy-pasted and drifting.

The task: package it as a reusable, documented pattern. Simple enough to describe in a sentence. Nowhere near simple enough to get right on the first pass — because the pattern touches Terraform-provisioned AWS credentials, a hosted auth provider's SMTP/captcha config, and an HMAC-signed internal email-sending endpoint. Get any of those subtly wrong and you don't get a build failure. You get a working-looking scaffold that quietly mishandles a secret.

What I tried​

The approach that mattered wasn't the architecture — it was the review loop around it. A plan went through four rounds of an adversarial AI critic before implementation even started, each round explicitly told to verify claims against real files on disk, not trust the plan's prose. Then, after a team of parallel agents implemented it, the implementation went through four more rounds of the same critic, re-reading source files and re-running the test suite independently each time.

The rounds narrowed in a way worth noticing:

• Round 1 found an architectural flaw: the plan split the system along the wrong axis. Two "clean" halves shared no real connection — one produced data the other needed, but nothing wired them together. The fix wasn't a patch, it was re-cutting the boundary around the actual data flow between them.
• Round 2, after the re-cut, found execution-level gaps: a drift-check test that would always fail against its own supporting files, and generated documentation that pointed at the wrong tool for one specific step.
• Round 3 found something sharper: an idempotency rule meant to keep a secret-sync script from re-sending credentials on every run was too effective — it also silently swallowed a legitimate credential rotation. The script would report success while quietly not rotating a secret that had actually changed upstream.
• Round 4 found one narrow leftover: a comment in generated output still named the internal tool the new script had replaced.

By round 4, the reviewer's own words were "no architectural replanning is otherwise required." That's the signal that the loop had converged — not a fixed number of rounds, but the point where findings stop being about the design and start being about a single stale string.

Round 1: wrong system boundary          (architecture)
Round 2: test scope, doc accuracy       (contract completeness)
Round 3: idempotency vs. secret rotation (correctness)
Round 4: one stale reference             (polish)
→ APPROVE

The same shape repeated after implementation: four more rounds, same narrowing pattern, same stop condition.

What I learned​

• A single review pass is not enough for secret-handling code. The rotation bug in round 3 is exactly the kind of thing that passes every functional test — the script behaves correctly on every input except "the secret changed and nothing else did," which is precisely the input a real credential rotation produces. It took a reviewer specifically looking for it.
• Split systems along the data dependency, not along the technology. The instinct to divide "infrastructure code" from "application code" felt clean and was wrong — it ignored that infrastructure output was the literal input the application half needed. The dependency should decide the boundary, and the boundary should carry a concrete, versioned artifact across it — not an assumption.
• Track when a review loop is actually converging. Round-over-round, each finding got narrower and less architectural. That's the signal to keep going one more round, not stop — and the signal to actually stop is an explicit "no architectural replanning needed," not a fixed budget.
• Multi-agent implementation surfaces its own race conditions. With several agents editing a shared codebase in parallel, a test failure during active concurrent work isn't automatically a bug — it can be a read caught mid-write by another agent. Re-run before diagnosing; don't let a transient read become a false story about what broke.

Takeaways​

• For anything touching credentials, secrets, or auth config: budget for 3+ independent review rounds, not 1. The rounds get cheaper and narrower as they go — that's the tell they're working, not a sign to stop early.
• When splitting a system into independently-usable pieces, ask what data actually has to cross the boundary before deciding where the boundary goes.
• The single most valuable instruction to give a review pass isn't "check this" — it's "verify this against the real files, don't trust the description." Every round that mattered here found something because it re-read source, not because it re-read the plan.

The problem​

I gave an AI coding agent a fairly open brief: read this tutorial about wiring up MCP servers (the protocol coding agents use to call tools) for an app, and turn it into a reusable pattern I could stamp into any of my projects — a CLI, a local server for my coding agent, a remote server for hosted AI tools, and a "skill" file teaching an agent how to use all of it, all sharing one service- account auth story.

The agent did what I asked. It designed a new shared package, a three-way pluggable auth adapter, and a distribution plan, all cleanly argued from the tutorial. It looked good. It was also about to duplicate a system I'd already built, tested, and shipped to production in a completely different project — a fact the agent's own plan should have surfaced and didn't, because it only searched the one repository the request landed in.

What I tried​

Before letting execution start, I ran the plan through an adversarial review step — a second AI pass whose entire job is to try to break the plan, not approve it. That review did something the design pass hadn't: it searched across my whole project family, not just the one repo, for anything that already did what the plan proposed.

It found three independent, hand-rolled implementations of the same "mint a secret token, hash it, verify it later" pattern — in three different projects, written at different times, with entropy quality that had already drifted between them. One project even had a working admin UI for managing these tokens. The plan got scrapped and rebuilt from scratch around harvesting the best of that existing code instead of reinventing it.

Before: design from the tutorial
  tutorial → new abstraction → 3-way adapter matrix → build

After: search for prior art first
  tutorial → grep the whole project family → found 3 drifting
  hand-rolled versions of the same thing → harvest the best one →
  generalize it → build

That reframing changed everything downstream: instead of inventing a new auth-provider abstraction speculatively, the work became "take the pattern that's already proven in production and make it reusable" — a much narrower, much safer scope.

What I learned​

The harvest-first plan still wasn't done. It went through four rounds of adversarial review before the implementation was approved, and the findings in round one were not nitpicks:

• A module that was supposed to be "off" by default only hid its UI — the underlying API routes it managed stayed fully reachable. A "disabled" feature that isn't actually disabled is a real security gap, and it's exactly the kind of bug that passes every unit test (each route worked correctly in isolation) while failing the property that actually matters (does every surface this feature owns respect the same flag?).
• A "this operation is safe to retry" guarantee was implemented as an in-memory flag on one process — which means it silently stopped being true the moment there were two processes, or one restart. The fix moved the guarantee to the one place it could actually hold: the database write itself, made idempotent at that layer instead of tracked in memory above it.
• A function returned an object that shared memory with what had just been saved to the database — meaning a caller could mutate the result and retroactively change what was persisted, without going through any authorized path.

None of these are the kind of bug a single careful author tends to catch in their own code, because the same mental model that wrote the implementation also tends to review it, and it doesn't reliably interrogate its own assumptions. What actually caught them was a genuinely adversarial second pass — instructed explicitly to find reasons to reject, not to confirm the work looks reasonable — repeated until it ran out of real findings, with every fix re-verified against a fresh test run rather than trusted on the fixing agent's own word.

Takeaways​

• Before designing anything from an external tutorial or article, search your whole project family for prior art — not just the repository the request landed in. The right question often isn't "how do I build this" but "have I already built this, and is it good enough to generalize?"
• "It passed all its tests" and "it's actually safe" are different claims. Property-level checks (does every surface a flag governs actually respect it?) catch a different class of bug than unit tests do.
• An idempotence or safety guarantee needs to live at the layer where the state actually is. If two people can each think they're "the first" to do something, the guarantee is in the wrong place.
• Adversarial review earns its cost in proportion to how security-adjacent the change is. For anything touching auth, access control, or "this should be disabled," budget for a real adversarial pass, and re-verify each fix independently before calling it done — not just once, but until the reviewer runs out of real findings.

Sanitized write-up. Names/hosts/keys generalized. Not for a specific client.

I had a dozen Next.js + Supabase apps that each reinvented environment configuration — the same layered-dotenv loader forked under a handful of package scopes, inlined into a preload script in some repos, absent in others. Config that varies by environment leaked into the build as NEXT_PUBLIC_ constants, so rotating a URL meant a rebuild. I set out to make one pattern and migrate the first app end-to-end as the template.

The model​

Environment variables are runtime config, not build constants. A single variable — ENV — selects the stage (dev/test/staging/prod), and a small loader merges dotenv files in precedence order:

.env < .env.local < .env.${ENV} < secretsPath < .env.${ENV}.local

The server reads process.env at request time and forwards only the explicitly-public values to the client through a provider — no NEXT_PUBLIC_, nothing baked into the bundle. Change a value, restart, done. One image promotes across every stage.

Two shared packages carry it: a framework-agnostic loader and a typed registry where each variable declares its type (public/secret/connection) — which also drives what's safe to commit.

The gotchas that actually cost time​

ESM-only broke CommonJS consumers. I shipped the loader ESM-only. Wrong: tracing agents, --require hooks, and a CJS preload all need to require() it — and Next compiles next.config.ts to CJS and require()s externals, throwing ERR_PACKAGE_PATH_NOT_EXPORTED. Dual ESM+CJS, with CJS as a first-class citizen, not a shim.

A build-time env preload clobbered NODE_ENV. Wiring the preload into the build script overwrote the framework's NODE_ENV=production with development, producing a dev/prod React mismatch that crashed prerendering. Rule: the env preload belongs on dev/start, never on build.

Local database bring-up ran migrations before the schema existed. The local Supabase stack runs its own migrations at boot — but the ORM owns the schema, so any migration enabling row-level security or an auth hook that reads app tables failed. The fix: relocate that table-dependent SQL into a post-schema step that runs after the ORM creates tables — the same ordered step locally and in prod.

Row-level security with zero policies silently broke every login. The auth token hook runs as a role that does not bypass RLS, so it read zero rows and every token came back with no org/role. A GRANT isn't enough under RLS — you need an explicit permissive policy for that role. Caught only by decoding a JWT.

Seeded users couldn't log in. Identity is the auth provider's user id; a plain database seed writes app rows with random ids that map to no auth user. The fix: create real auth users via the admin API with email-confirm on (skips signup, captcha, and the email round-trip), then key the app rows to the returned id. Now automated tests log in with known credentials.

.env.prod never reaches the platform. The container build ignores .env.*, and the managed host doesn't run the preload — so on both platforms every variable, secret and non-secret, has to be entered in the platform's settings. A push does not magically configure prod; secrets, migrations, and the auth dashboard config are one-time manual (or scripted) setup.

What shipped​

The pattern is now three composable skills — the app-side loader, the local stack (containerized DB, ordered process orchestration, per-repo port blocks so several stacks coexist), and the first-prod-deploy runbook — plus the two shared packages. The first app migrated cleanly and verified end-to-end: local stack up, seeded accounts logging in with correct roles, no captcha friction for tests. It becomes the diff-template for the remaining eleven.

The meta-lesson: a shared dependency beats a shared document. Every one of those forks started as "we all agree on the pattern." Agreement drifts; a published package doesn't.