Skip to main content

I asked an AI agent to build a feature. It found I'd already built it — badly, three times.

· 5 min read
Maintainer, next-starters

A tutorial-shaped feature request turned into a lesson about checking for prior art before designing, and about what adversarial code review actually catches that a careful first pass doesn't.

The problem

I gave an AI coding agent a fairly open brief: read this tutorial about wiring up MCP servers (the protocol coding agents use to call tools) for an app, and turn it into a reusable pattern I could stamp into any of my projects — a CLI, a local server for my coding agent, a remote server for hosted AI tools, and a "skill" file teaching an agent how to use all of it, all sharing one service- account auth story.

The agent did what I asked. It designed a new shared package, a three-way pluggable auth adapter, and a distribution plan, all cleanly argued from the tutorial. It looked good. It was also about to duplicate a system I'd already built, tested, and shipped to production in a completely different project — a fact the agent's own plan should have surfaced and didn't, because it only searched the one repository the request landed in.

What I tried

Before letting execution start, I ran the plan through an adversarial review step — a second AI pass whose entire job is to try to break the plan, not approve it. That review did something the design pass hadn't: it searched across my whole project family, not just the one repo, for anything that already did what the plan proposed.

It found three independent, hand-rolled implementations of the same "mint a secret token, hash it, verify it later" pattern — in three different projects, written at different times, with entropy quality that had already drifted between them. One project even had a working admin UI for managing these tokens. The plan got scrapped and rebuilt from scratch around harvesting the best of that existing code instead of reinventing it.

Before: design from the tutorial
tutorial → new abstraction → 3-way adapter matrix → build

After: search for prior art first
tutorial → grep the whole project family → found 3 drifting
hand-rolled versions of the same thing → harvest the best one →
generalize it → build

That reframing changed everything downstream: instead of inventing a new auth-provider abstraction speculatively, the work became "take the pattern that's already proven in production and make it reusable" — a much narrower, much safer scope.

What I learned

The harvest-first plan still wasn't done. It went through four rounds of adversarial review before the implementation was approved, and the findings in round one were not nitpicks:

  • A module that was supposed to be "off" by default only hid its UI — the underlying API routes it managed stayed fully reachable. A "disabled" feature that isn't actually disabled is a real security gap, and it's exactly the kind of bug that passes every unit test (each route worked correctly in isolation) while failing the property that actually matters (does every surface this feature owns respect the same flag?).
  • A "this operation is safe to retry" guarantee was implemented as an in-memory flag on one process — which means it silently stopped being true the moment there were two processes, or one restart. The fix moved the guarantee to the one place it could actually hold: the database write itself, made idempotent at that layer instead of tracked in memory above it.
  • A function returned an object that shared memory with what had just been saved to the database — meaning a caller could mutate the result and retroactively change what was persisted, without going through any authorized path.

None of these are the kind of bug a single careful author tends to catch in their own code, because the same mental model that wrote the implementation also tends to review it, and it doesn't reliably interrogate its own assumptions. What actually caught them was a genuinely adversarial second pass — instructed explicitly to find reasons to reject, not to confirm the work looks reasonable — repeated until it ran out of real findings, with every fix re-verified against a fresh test run rather than trusted on the fixing agent's own word.

Takeaways

  • Before designing anything from an external tutorial or article, search your whole project family for prior art — not just the repository the request landed in. The right question often isn't "how do I build this" but "have I already built this, and is it good enough to generalize?"
  • "It passed all its tests" and "it's actually safe" are different claims. Property-level checks (does every surface a flag governs actually respect it?) catch a different class of bug than unit tests do.
  • An idempotence or safety guarantee needs to live at the layer where the state actually is. If two people can each think they're "the first" to do something, the guarantee is in the wrong place.
  • Adversarial review earns its cost in proportion to how security-adjacent the change is. For anything touching auth, access control, or "this should be disabled," budget for a real adversarial pass, and re-verify each fix independently before calling it done — not just once, but until the reviewer runs out of real findings.