Most of what you read about AI coding tools is a hot take from one of two camps: it’s going to replace every developer by next year, or it’s a glorified autocomplete that hallucinates garbage. The reality is it’s both at the same time. Personally I think AI is overhyped and is currently ruining the economy, but it does some things really well and should be exploited by any self-respecting systems or application engineer. Here’s how I use it safely.
Key Takeaways#
- “Vibe coding” (prompt, accept, run, repeat) and engineering with AI assistance use the exact same tool but land in wildly different risk profiles. The difference is entirely in the process the human wraps around it.
- AI coding assistants are only as safe as the process wrapped around them. The model doesn’t know when it’s wrong, so the process has to catch that instead.
- Persistent instruction files (
CLAUDE.mdand similar) turn “remember to tell it that every time” into a standing default the tool just reads on its own. - With AI, tests-first development is the mechanism that forces you to define “correct” before the model is allowed to generate anything.
- YAGNI (You Aren’t Gonna Need It, the old XP/agile principle against building for hypothetical future needs) has to be a spelled-out, standing instruction. Left alone, AI defaults to more code, more abstraction, more “just in case” scaffolding than the task needs.
- The hard boundary: only let AI act autonomously in stuff you’re competent enough to personally catch it screwing up. Outside that, it’s a liability with good grammar.
- Token spend compounds because your full conversation history gets re-billed every turn. Scoped instructions, clearing context between tasks, and matching model cost to task difficulty cut that spend a lot without touching output quality.
- Real documentation (READMEs, architecture write-ups, a searchable session log) is what lets you pick a project back up after months away without re-deriving everything from cold code
- Asking for a “critiqued draft” in the same breath as the draft itself doesn’t work. The fact-check has to be its own separate instruction, fired after the output already exists, or you just get performed skepticism instead of a real re-check
The rule everything else hangs on#
Trusting AI in a domain you can’t personally verify is how confident-sounding garbage gets shipped.
Here’s the one that matters most, so it goes first: I don’t use AI for things I don’t understand well enough to catch it lying to me.
I use Claude Code CLI, an AI agent, constantly for software development, infrastructure, and security work, because I’ve spent years doing that work and I can tell, fast, when the output is wrong. A hallucinated function signature, a subtly broken auth check, a config that looks right but isn’t. I catch those because I already know what right looks like.
I don’t use it to draft or interpret anything legal. Not contracts, not compliance language, not “is this clause enforceable.” AI probably isn’t any worse at law than it is at code, but I have no independent way to tell when it’s confidently wrong there. A hallucinated legal citation reads exactly as authoritative as a real one to someone who doesn’t already know the case law. In code, a hallucinated function call throws an error. In a contract, a hallucinated obligation just sits there quietly until it costs you.
That’s the actual test, and it generalizes past law: if you can’t independently verify the output, you can’t safely let AI generate it unsupervised. Everything below is the process I use inside the domain where that test passes.
Vibe coding is not what this post is about#
You can’t catch yourself sliding into a failure mode you haven’t named first.
There’s a term for the other approach: vibe coding, popularized by Andrej Karpathy’s 2025 description of prompting an AI, accepting whatever it hands back, running it, and pasting the next error message in until something works, without ever really reading the code in between. You’re steering by vibes, not by understanding what got built.
Vibe coding isn’t inherently wrong. For a weekend prototype, a throwaway script, a proof-of-concept nobody’s life depends on, it’s the fastest way to get from idea to working thing, and honestly I do it too when the stakes are that low. The failure mode is using it on anything with real users, real money, customer data, or a security boundary, and not noticing you’ve slid into it, because the AI’s output looks just as confident either way. Confidence just means the model finished generating text. It says nothing about whether the text is correct.
The rest of this post is the other mode: an engineer using AI as a very fast pair programmer inside an engineering process that already existed (specs, tests, review, security gates, reversibility) instead of letting the AI replace that process. Same tool, same model, wildly different risk profile. The difference is entirely in what the human insists on around it.
The tool has to read the instructions, not just get told them once#
Repeating yourself every session doesn’t scale, and forgetting to repeat yourself is how guardrails quietly disappear.
I run Claude Code from the CLI, and every project gets a CLAUDE.md file the agent reads automatically at the start of every session. That’s the difference between “I told it to write tests first” and “it writes tests first every single time, on every project, without me repeating myself.” The instructions file carries:
- Coding style and conventions for that specific codebase
- The testing and review requirements below, as standing defaults, not one-off requests
- Domain-specific context it would otherwise have to be re-taught every session (deployment setup, architecture decisions, known gotchas)
If you’re new to this: the instructions file is the single highest-leverage thing you can set up before writing any code with an assistant. It’s the difference between managing a tool and managing a very fast, very literal new hire who forgets everything overnight unless you write it down.
Spec first, tests first, then code#
A model has no internal sense of “correct” until a test defines it for that specific case.
I require Red-Green-Refactor test-driven development on anything that matters: write the failing test that defines correct behavior, watch it fail, then let the AI write the minimum code to pass it, then clean up.
This matters more with AI than it did before AI, not less. A model will happily generate code that looks plausible and runs without error while doing the wrong thing. It has no internal sense of “wait, is this actually right,” only “does this match the pattern of correct-looking code.” A test written before the implementation exists is a spec the AI can’t talk its way around. It either passes or it doesn’t. Much harder to fake than a code review skim.
Straight from my instructions file, on any project I’ve flagged as production-grade:
Test-driven: applications are to be written using Red Green Refactor
TDD. Write tests first, before code, not as an afterthought.YAGNI, said out loud, every time#
Left alone, AI defaults to more code than the problem needs, and a human has to maintain whatever it built.
Left to its own devices, an AI coding assistant tends to over-build: extra abstraction layers, config options for values that will never change, error handling for cases that can’t happen, “just in case” flexibility nobody asked for. There’s no malice in it, just an optimizer chasing “looks thorough” instead of “is minimal.”
I counter that with an explicit, standing instruction: build the smallest thing that correctly solves the actual problem, reuse what already exists in the codebase before writing something new, and don’t design for hypothetical future requirements. Every unrequested abstraction is a thing a human now has to understand, maintain, and eventually debug at 3am, and the person debugging it might not be the one who “wrote” it.
Here’s the actual paragraph from the standing instructions file I keep loaded in every session, lightly trimmed of an internal cross-reference so it reads standalone:
Apply YAGNI: don't build for hypothetical future requirements. Before reaching
for a library, framework, or custom abstraction, check whether the standard
library or a native platform feature already covers it, and prefer the
shortest correct solution, a one-liner beats fifty lines of scaffolding. This
applies by default; production code still needs real security, testing, and
architecture, so don't let "shortest solution" skip those.Nothing fancy. Just a plain paragraph telling the tool what “done” is supposed to look like, read fresh at the start of every session instead of something I have to remember to say.
But the instruction alone isn’t what makes this work. I know what over-engineering looks like because I’m great at it myself, plenty of hard-earned experience shipping a config option nobody asked for. That’s exactly the kind of domain competence from earlier in this post: instructions catch most of it, and I catch the rest by actually reading the diff, not by trusting the instruction to have worked.
Small scopes, reviewed as you go#
A bigger unreviewed diff is more surface area for something wrong to hide in.
I don’t ask for “build me the whole feature.” I work in small, agile-sized increments (one behavior, one bug fix, one component at a time) and review the actual diff before accepting it. Plan mode, or the equivalent step in whatever tool you’re using, comes before big changes, so the approach gets agreed on before code gets written, not after.
The bigger the chunk of unreviewed code you accept in one go, the more surface area for something wrong to hide in. Small diffs are reviewable diffs.
Security review and code review are not optional steps#
AI-authored code is still code, and the tooling that wrote it is itself part of your threat model.
Every non-trivial change gets a security-focused pass, the same threat-modeling questions I’d ask of a junior developer’s pull request: does this validate input at the boundary, does this leak anything in logs or error messages, does this introduce an injection path, is this the least-privilege way to do it. I’ve written before about how AI agents themselves are a live prompt-injection and supply-chain risk. The tooling that writes your code is itself part of your threat model, not just the code it produces.
Treat AI output like a capable but unvetted contributor’s PR: probably fine, still needs a real review, no exceptions for “the AI usually gets this right.”
Two more lines from the same production-grade standard as the TDD one above:
Security by design: threat model from the start, never bolt it on
later.
No shortcuts on input validation, auth, error handling, or data
integrity at system boundaries.Make it check its own work#
A confidently wrong answer looks exactly like a confidently right one, and a second pass catches what the first pass couldn’t see about itself.
This will surprise a lot of people, but I explicitly ask the AI to fact-check and critique its own output before I trust it. Re-verify the specific numbers, dates, and claims it just generated against the actual source. Check for internal contradictions. Flag anything stated with more confidence than the evidence supports.
It’s not foolproof, but this will give you a lot of bang for your token buck by catching obvious bugs and let you spend time on the ones it didn’t catch. A model critiquing its own output is still the same model, with the same blind spots. But it catches a real, non-trivial slice of errors that a single confident first pass misses. One extra step, and it’s caught real bugs before they shipped. Cheap insurance.
The critique has to be its own separate instruction, issued after the output already exists, not folded into the same request that generated it. Asking for “a critiqued draft” or “write this and then critique it” in one shot doesn’t get you a real critique. The model is still in generation mode when it writes the caveat, so the “critique” comes out as a token gesture toward caution rather than an actual re-read. Making it a distinct second pass, pointed at output that’s already sitting there as a finished artifact, is what gets it to actually re-check numbers against sources instead of just performing skepticism.
This one lives inside the skills that actually produce a written deliverable, not the global instructions file. Here’s the fact-check step from one of my project-specific skills, trimmed of the domain-specific details:
Fact-check and critique the draft before presenting it. This is not
optional and not skippable for time. Before reporting anything:
- Re-check the specific figures and dates you wrote against the
sources you gathered them from. Catch transcription errors and
stale numbers.
- Check for internal contradictions between sections.
- Flag anything stated with more confidence than the source actually
supports. Soften or cut it rather than leaving it sounding
authoritative.
- If the critique changes anything material, revise the draft. Don't
silently fix it and stay quiet if you reverse a claim, say so.Verify by running it, not by reading it#
Passing tests prove the code is structurally correct, not that the feature actually works.
Type-checking and a passing test suite tell you the code is structurally correct. They don’t tell you the feature actually works. For anything with a runtime surface (a UI, a CLI, an API) I actually run it and drive the real flow before calling it done. “It compiles and the tests are green” and “I watched it do the thing” are different claims, and only one of them is actually verification.
Part of what makes that practical is that I build with Docker Compose by default, so the entire stack (app, database, whatever else it depends on) comes up locally with one command, the same way it’ll come up anywhere else it gets deployed. There’s no “well it works on my machine, but the real environment is different enough that I can’t easily check” excuse. If it runs locally, it runs. That’s a standing default in my instructions file too, not something I decide project by project.
Reversibility is the safety net under all of it#
None of the above makes AI infallible, it just makes mistakes cheap to catch and cheap to undo.
None of the above makes the AI infallible, it just makes mistakes cheap to catch and cheap to undo. So the process assumes things can be undone: everything lives in git, changes land in small reviewable commits, and anything hard to reverse (force-pushing, dropping a database table, skipping a pre-commit hook, handing the agent broad filesystem or credential access) requires an explicit stop-and-confirm instead of happening automatically. An AI agent with unrestricted, unreviewed write access to production is a bigger risk than the bad code it might write. Scope what it can touch the same way you’d scope any other automated process with write access to your systems.
In practice that means I don’t let it run unattended, full stop, except for file edits inside a small set of directories I’ve explicitly scoped as low-risk. Everything else (installing a package, running a command outside that scope, touching git history, anything network-facing) stops and asks. While a session is running I’m actually watching: answering those permission prompts as they come up, and reading what scrolls by in the terminal for anything that looks like a prompt injection artifact, not just trusting the model to flag it itself.
Documentation is for the version of me that comes back in six months#
Future-you won’t remember the decisions, only the code, and code doesn’t explain itself.
I have Claude write real documentation as part of the work, not as an afterthought once something’s already shipped. A plain, easy-to-read README for anything with setup steps or a non-obvious way to run it. For anything bigger, a full architecture write-up: what the pieces are, how they talk to each other, why it’s built this way instead of the more obvious way.
The reason is selfish and specific: I have a lot of projects, and I don’t touch most of them daily. Some sit for months. When I come back to one, I’m not the person who built it anymore, not really, I’ve forgotten the specifics. Good documentation is what lets me pick it back up in twenty minutes instead of an afternoon of re-reading code cold and trying to reconstruct decisions I already made once.
I also keep a running, searchable log of what happened in each working session (what got built, what got decided and why, what got tried and rejected, what state things were left in). I’ve written a separate post on that skill specifically, since it’s a big enough piece of the workflow to stand on its own. Here’s the line from that skill’s own instructions that sets the tone for the whole thing:
This is a narrative/reference doc for future sessions and the user
to quickly recall what was built and why, not a changelog and not
marketing copy. Write it like an engineer leaving notes for their
future self.Short version: a README tells you what a finished thing does. A session log tells you the story of how it got that way, which matters just as much when you’re trying to remember why you didn’t just do the obvious thing.
The token bill is a discipline, not an afterthought#
Full conversation history gets re-billed every turn, so unmanaged usage compounds invisibly until the bill shows up.
I’ve written before about the mechanic that makes AI usage expensive by default: the model has no memory between calls, so your entire conversation history gets re-sent and re-billed on every single turn. A five-turn conversation runs closer to six and a half times its raw token count, because turn five re-pays for everything said in turns one through four. Left unmanaged, that compounds fast, and it’s invisible until the bill shows up.
I’ve taken a few concrete measures against that, on top of everything above:
Scoped, on-demand instructions instead of one giant standing prompt. Rather than a mega-file of instructions loaded into every conversation regardless of task, I keep task-specific workflows as separate, invokable skills that only load when actually used. A research routine’s instructions don’t get re-sent on a day I’m debugging infrastructure.
Clearing context between unrelated tasks. Once a task is done, I start the next one fresh instead of dragging a growing, unrelated conversation history forward. That history was never free. Every prior message in it gets paid for again on every new turn, per the mechanic above.
Matching the model to the task. Not every step needs the most capable, most expensive model running. Grunt work, like a research skill that runs a batch of web searches and summarizes results, gets pinned to a cheaper, faster model in its own config. Judgment calls that actually need the sharpest reasoning still get the top-tier one. I’m not paying premium-model rates for tasks that don’t need premium-model judgment. In practice that’s a one-line addition to a skill’s frontmatter:
--- name: pretrade-research description: Run the market research routine and save a dated brief. model: claude-haiku-4-5 ---Cutting unconditional heavy steps down to conditional ones. A workflow step that always runs an extra research pass “just in case,” whether or not the earlier steps turned up anything worth chasing, gets rewritten to only fire when there’s an actual lead to follow. Same output quality on the nights that matter, no wasted spend on the nights that don’t.
Running a YAGNI-enforcer plugin on the tool’s own output, not just the codebase’s. I keep ponytail loaded, a plugin that pushes back on the model’s own instinct to generate more: more abstraction, more speculative flexibility, more code than the task needed. Less generated output is fewer output tokens, on top of being the smaller diff described earlier in this post.
None of this is about being cheap for its own sake. It’s the same instinct as YAGNI applied to the tool’s own footprint: don’t spend on generation you don’t need, same as you shouldn’t ship code nobody asked for.
The short version#
- Instructions file, read every session, not repeated by hand
- Tests before code, always
- YAGNI stated explicitly, or it’ll over-build
- Small scopes, reviewed diffs, plan before big changes
- Security and code review as a mandatory gate, no exceptions for AI-authored code
- Ask it to fact-check and critique its own output before you trust it
- Verify by running the thing, not by reading the diff
- Git as the safety net; anything hard to reverse gets a human confirmation
- Scoped instructions, cleared context, and the right model for the task, so the token bill matches the work actually done
- READMEs, architecture docs, and a searchable session log, written as part of the work, not after it
- The boundary that makes all of it safe: only let it act autonomously where you’re competent enough to catch it being wrong.
That last one is the whole point. Every practice above is a way of catching AI being confidently wrong inside a domain I understand well enough to know what right looks like. Outside that domain, none of these guardrails help, because I wouldn’t know which alarm bells to listen for. That’s not a limitation of AI tooling. It’s a limitation of me, and knowing where it sits is the actual skill.
This is the same discipline I bring to client engagements, helping teams adopt AI-assisted development without turning it into an unreviewed attack surface or a liability nobody signed off on. If you’re standing up AI tooling on your team and want a second set of eyes on the guardrails before it’s load-bearing, reach out.