AI Follows Your Instructions 70% of the Time: How to Enforce the Rest

Part 6 of a 6-part series on configuring AI coding assistants for large codebases

Imagine you’ve written a clean instruction file for your Dropwizard service. It says “all DAOs must return Optional<T> for single-row lookups, never return null.” Clear rule. No ambiguity.

You ask the AI to write a new DAO method. It returns null.

You fix it. Ask for another method. This time it returns Optional<T>. Ask again. Back to null.

This isn’t a bug. This is the base rate. Instruction files are followed about 70% of the time. Even well-written ones. The model is probabilistic. It will occasionally ignore any rule you give it.

For style preferences, 70% is fine. Fix it in review and move on. For security boundaries (“never log PII,” “never query production from local”), 70% is dangerous. The 30% that slips through is where incidents come from.

This post is about closing that gap.

Intent vs enforcement

Instruction files are suggestions to a probabilistic system. They make the AI more likely to do the right thing. They don’t guarantee it.

Hooks are deterministic. They run after every AI action. They pass or fail. No probability involved.

Here’s a concrete example. Your instruction file says “use JDBI3 @SqlQuery annotations, never raw JDBC.” The AI ignores this and writes a method using Connection.prepareStatement(). With just the instruction file, this reaches your PR and you catch it in review. Maybe.

With a post-edit hook that runs mvn compile -pl <module> after every file write, the build catches type mismatches immediately. The AI sees the compilation error. It rewrites the code using JDBI3. No human involved.

Another example. Your instruction file says “never import from javax.persistence.” A hook that greps for banned imports after every file write catches violations in seconds:

.claude/settings.json:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "Write",
        "command": "grep -rn 'javax.persistence\\|org.hibernate' --include='*.java' src/ && echo 'BANNED IMPORT DETECTED' && exit 1 || exit 0"
      }
    ]
  }
}

The instruction file handles the 70%. The hook handles the 30%. Together, they get you to 100%.

Hooks vs instructions

Not everything needs a hook. Hooks add build time. They interrupt flow. Use them for rules where violations cause real damage.

Use hooks for:

• Banned imports (javax.persistence, Spring annotations, deprecated packages)
• Compilation checks after file writes (mvn compile)
• PII patterns in log statements (grep for field names that should never be logged)
• Checkstyle or SpotBugs after edits

Leave as instructions:

• Code organization preferences (where to put a new class)
• Naming conventions (these are hard to validate programmatically)
• Architectural boundaries (call service layer, not DAO directly)
• Pattern preferences (use builder pattern for complex objects)

The rule of thumb: if you can write a grep or a compiler check for it, make it a hook. If it requires judgment, leave it as an instruction.

Sandboxing

Hooks catch code-level violations. But AI coding assistants do more than write code. They run shell commands. They read files. They make network calls.

Your instruction file might say “never read .env files” or “never make external HTTP calls.” But instruction files can’t enforce filesystem or network boundaries. The AI can ignore the instruction and run cat .env in a shell command.

Sandboxing enforces these boundaries at the OS level. The AI physically cannot access files or networks outside the allowed scope.

Claude Code offers an OS-level sandbox using macOS Seatbelt or Linux bubblewrap. It restricts filesystem access to the project directory, limits network to an allowlist of domains, and blocks process-level escape hatches. Anthropic reports 84% fewer permission prompts with sandboxing enabled.

Cursor shipped native sandboxing that blocks network access and limits file access to the workspace and /tmp/ by default. About a third of Cursor requests now run sandboxed.

GitHub Copilot’s coding agent runs in a GitHub Actions environment with scoped tokens. Network and filesystem access is constrained by the Actions runner configuration.

For any team handling credentials, PII, or payment data: sandboxing is not a nice-to-have. It’s the only way to guarantee that an AI assistant can’t accidentally exfiltrate sensitive data, regardless of what the instruction file says.

Production config discipline

A change to your root instruction file affects every AI interaction for every developer on your team. That’s a bigger blast radius than most config changes you’ll ship this quarter.

Version control. This is obvious. Instruction files already live in your repo. But “lives in the repo” and “is managed like production config” are different things. Track changes in your changelog. Write commit messages that explain why a rule was added, not just what.

Code review. Every change to a shared instruction file should go through a PR. A poorly worded rule degrades everyone’s AI output. A contradictory rule causes unpredictable behavior. A second pair of eyes catches both.

CODEOWNERS. Assign ownership of instruction files explicitly. The person who owns the billing module should own billing/CLAUDE.md. The platform team should own the root file. Without ownership, instruction files rot like any other unowned documentation.

Changelog discipline. When you add a rule, note what failure it prevents. Six months from now, someone will ask “why is this rule here?” If the answer is documented, the rule survives. If it’s not, someone deletes it, and the original failure comes back.

Three-tier governance

Individual developers, team leads, and platform teams all need different levels of control.

Shared files (version-controlled). Team standards. Architectural boundaries. Build commands. Module boundaries. These are the same for every developer. They live in the repo. Changes go through PR review.

Personal files (gitignored). Individual preferences. Output verbosity. Explanation style. Test suggestion behavior. These live in gitignored files like CLAUDE.local.md or Cursor User Rules. They don’t affect the team.

Enterprise policies (centrally managed). Security constraints. Compliance rules. Banned operations. These are pushed by platform teams and cannot be overridden by individual developers or even team leads. In Claude Code, managed settings are deployed via MDM tools like Jamf or Intune. In Cursor, Team Rules can be marked as required. In Copilot, organization-level policies are set via GitHub.com.

The separation matters. Shared files standardize. Personal files customize. Enterprise policies enforce. Mixing these layers (putting personal preferences in shared files, or putting security rules where developers can override them) creates friction and risk.

Self-evolving instructions

The most sustainable maintenance model: the instruction file grows from observed failures, not from speculation.

The process is simple. The AI makes a mistake. An engineer identifies why. The correction gets added to the instruction file.

Week 1: AI generates a @Controller class. Add: “NOT Spring Boot. Use Jersey @Path resources.”

Week 3: AI uses EntityManager. Add: “Use JDBI3 DAOs. Never JPA/Hibernate.”

Week 5: AI puts validation logic in a DAO. Add: “DAOs contain only SQL queries. Validation belongs in the service layer.”

Week 8: AI merges two resources into a base class. Add: “OrderResource and InvoiceResource serve different auth flows. Never merge them.”

After a quarter, you have a curated list of 15–20 rules. Every rule traces to a real incident. No bloat. No speculation. High signal.

Some teams formalize this. When an engineer rejects an AI suggestion for a structural reason, they tag the rejection. Tagged rejections get reviewed weekly. Patterns get promoted to the instruction file.

Other teams use the AI itself. In Claude Code, you can tell the AI to add the correction to CLAUDE.md as part of the fix. The file evolves in the same PR as the code change.

Either way, the feedback loop is the same: failure → diagnosis → instruction → prevention.

Measuring impact

You can’t improve what you don’t measure. But instruction file impact is hard to isolate. You can’t A/B test “with instruction file” vs “without” across the same team.

Proxy metrics that work:

Style-related PR comments. Count how many PR comments are about style, naming, or pattern violations in AI-generated code. Before instruction files, this might be 5–10 per PR. After, it should drop. One team reported a 50% reduction.

Build failures from AI-generated code. Track how often AI-generated code fails to compile. This should decrease over time as version pins and DO NOT lists mature.

Time to first working suggestion. Informally, how many back-and-forth rounds does it take before the AI produces code you’d accept? Better instruction files reduce this.

Hallucinated import frequency. Track phantom imports caught by hooks or CI. This should trend downward as your DO NOT list grows.

Qualitative signal. Ask engineers in retros: “Is the AI getting better at understanding our codebase?” Subjective but useful for spotting trends.

Don’t over-instrument. Pick two or three metrics. Track them monthly. Look for trends, not absolutes.

Quarterly audit

Instruction files accumulate entropy. Rules get added but never removed. Referenced files get renamed. Module boundaries shift. After six months, the instruction file describes a codebase that no longer exists.

Once a quarter, spend an hour:

• Read every rule. Does it still apply? Delete what’s obsolete.
• Check referenced files. Do the pointers still resolve? Update broken paths.
• Verify the package map. Did new modules get added? Are any modules gone?
• Look for contradictions. Did a migration complete, making both the “current” and “legacy” rules stale?
• Check instruction count. Are you still under 200 lines for the root file? If not, what can move to a scoped file?

An hour per quarter. Cheap insurance.

The maturity ladder

Most teams are at Level 0 or 1. That’s fine. Jumping straight to Level 4 isn’t the goal. Each level compounds on the previous one.

Level 0: No instruction file. Or a copy-pasted template that doesn’t match the project. The AI works from generic training data. Every developer fights the same battles independently.

Level 1: A curated root file. Tech stack, build commands, basic conventions. 50–100 lines. Already a meaningful improvement. This is where most teams should start.

Level 2: Hierarchical scoping. Service-specific or module-specific instruction files. Progressive disclosure. The AI sees only relevant context. This is where large codebases see the biggest single jump in quality.

Level 3: Hooks and CI enforcement. Post-edit hooks for compilation and banned imports. CI checks for hallucinated dependencies. The 70% compliance gap closes. Self-evolving instruction files from observed failures.

Level 4: Enterprise governance. Centrally managed policies. Sandboxing. Analytics on AI-assisted code quality. Cross-team standardization of instruction file practices. Measured impact.

Getting from Level 0 to Level 1 takes an afternoon. From Level 1 to Level 2 takes a day. From Level 2 to Level 3 takes a week. Each step delivers visible improvement. Don’t let the gap between where you are and Level 4 stop you from taking the first step.

Series wrap-up

Six posts. One idea in each.

1. Instruction files are engineering artifacts. Treat them like production config. Keep them under 200 lines.
2. Scope aggressively. The AI should only see instructions relevant to the current task.
3. Document what AI can’t infer. Module boundaries, dependency directions, design decisions.
4. Hallucination is predictable. Version pins, DO NOT lists, and reference files prevent most of it.
5. Configure for the codebase you have. Migration markers, do-not-touch zones, incremental specification.
6. Instructions suggest. Hooks enforce. Close the 70% compliance gap with deterministic checks.

The common thread: less is more. Fewer instructions, better scoped, deterministically enforced. That’s the whole framework.

Start with a clean 50-line root instruction file for your project. Add one scoped file for the module you work on most. Set up one post-edit hook that runs compilation. You’ll see the difference in your first AI-assisted coding session.