Autonomous AI coding: what 10 weeks in production showed

Over the last 4 weeks, an autonomous pipeline closed 133 issues in my repository. 88% on the first try, median time from start to merge — 17 minutes. I didn’t write that code, and I didn’t review it.

This is not a demo or marketing. It’s a production pipeline running 24/7, merging code every day into a branch I also work on directly. This article is about the architectural principles that make this reliable — and the metrics that prove it. I’ve removed specific product names; the principles are universal.

Why autonomous coding usually breaks

The idea is old: an AI agent picks a task from a tracker, writes code, runs tests, pushes a branch, opens a pull request. Human out of the loop. There are dozens of implementations, but almost all stumble on one of three things:

1. The agent produces broken code. A model confidently writes something that looks right but doesn’t compile, fails tests, or breaks neighboring modules. Without a check, garbage flows into main.
2. The pipeline is unstable. Parallel workers conflict, sessions crash, git worktrees break the build. One stable run out of ten attempts.
3. The human is still in the loop. The “autonomous” pipeline creates a PR, but a human must review it — because there’s no trust in quality. That’s not autonomy, that’s autocomplete with extra steps.

I hit all three. After several iterations, the system reached metrics I used to think were unreachable for AI generation. Below — what actually fixed each one.

Principle 1: Architecture beats prompts

The main shift: 90% of reliability comes from the pipeline around the agent, not the prompt itself.

Everyone is looking for “the prompt” — the right metaphor, the right few-shot example, the right tone of voice. That gives you 5-10%. The rest is structure: who does what, where it’s checked, what happens on error.

My pipeline looks like this:

idea → AI analyst: interview → specification → task decomposition
                                                          ↓
issue in tracker (labeled "ready")  ← I label after review
        ↓
orchestrator: claim issue → create branch → start AI agent session
        ↓
agent: writes code → commits
        ↓
deterministic gate: linter + tests + build
        ↓ green                       ↓ red
PR created → CI green         errors → back to the same session
→ auto-merge (no human)              ↓
                               retry (max 5) → pause

Important nuance: the tasks in the tracker are also written by AI, not by me manually. A specialized AI analyst conducts an interview, produces a specification, and decomposes it into tasks with acceptance criteria, file maps, edge cases, and dependencies (the SDD methodology). My job at this stage is to provide direction, answer questions, and apply the ready label. That’s hours, not weeks.

Every block here is a point where things can break — and every one is covered:

• Agent didn’t commit → gate catches “no changes” → rework.
• Tests failed → output goes back to the same session → agent sees the context and fixes it.
• Session died (API timeout, OOM) → issue returns to the queue → new session.
• Agent got stuck (same errors repeatedly) → after two identical fingerprints → pause.

The clearest proof is what I call the “Do Nothing Paradox.” Over 10 weeks of operation I collected 7 recommendations for improving the agent’s prompt: “read the file before editing,” “check file existence before writing,” “wrap long-running bash in timeout.” Reasonable, obvious, safe fixes. None were implemented. Yet metrics over the same period improved from 29% to 75% CI success rate.

The takeaway: the architecture — a linear pipeline + deterministic gate + retry loop — is what fixed the system, not prompt patches. When the foundation works, incremental prompt improvements are long-tail optimization. The risk of introducing regression into a working system outweighs the potential gain. Burden of proof is on the change, not on the status quo.

Principle 2: Don’t trust AI to review AI

A deterministic gate is more reliable than AI review. You can’t ask a model to evaluate another model’s code — that’s a closed loop.

In an early version of the system, I had AI review: a separate agent read the PR and issued a verdict — “accept / request changes.” Sounds logical — AI wrote it, AI checks it. In practice, this produced two effects:

• False positives: the reviewer approved code that didn’t compile.
• False negatives: the reviewer rejected working code over stylistic preferences.

Both errors stem from the same root: a model cannot reliably assess code correctness from text alone. It sees “plausible” code and can’t tell it apart from “correct” code. Tests and the compiler always can.

So I replaced AI review with a deterministic gate:

Gate runs sequentially — all steps blocking:
1. Linter (static analysis + style)
2. Type checking and static analysis (go vet / tsc)
3. Build all affected packages
4. Unit tests with coverage check
5. Special checks (DB schemas, shell scripts, proto files)

Each step either passes or fails — no interpretation. The gate runs on the HEAD commit (not the working directory), so it checks exactly what will go into the PR. If something is red, the error with full output goes back to the same agent session that just wrote the code. Context is preserved; the agent fixes things with full context, not blindly.

Tests are a mandatory part of the code, not optional. The agent writes tests alongside implementation. The gate checks not only “tests pass” but also “coverage didn’t drop below threshold.” If the agent added 100 lines of code without a single line of tests — the threshold is breached → rework. On a new project, all checks are blocking from day one, including style: it’s cheaper than cleaning up accumulated debt later.

The result: over 4 weeks of stable operation, only 7 out of 133 merges had code-quality failures on CI. That’s a 5% failure rate — the deterministic gate consistently catches what matters.

Principle 3: One process beats parallelism

Single-process, one task at a time turned out to be more reliable and faster in the long run than parallel workers.

This is counterintuitive. Parallelism = more throughput, right? In theory, yes. In practice, parallel AI agents in the same repository create a cascade of problems:

In a previous version I tried parallelism: a worker pool, isolation via git worktrees, several simultaneous tasks. Over a week of use — not a single stable production run. The cancel cascade burned 60% of CI time in one week, quota was exhausted, CI went down completely.

In the final version I chose single-process: one task → one agent → one branch → one session. The queue is processed strictly sequentially. Sounds slow, but:

• Cancel cascade disappeared — no parallel pushes, nothing to cancel. Waste dropped from 60% to ~15%.
• Throughput turned out higher — no overhead for isolation, cleanup, conflict resolution.
• Easier to debug — one log, one session, one root cause when something breaks.

Parallelism makes sense for distributed teams of 10+ developers. For a single repository with a single maintainer, sequential processing wins on every metric, including speed.

What it takes to build such a pipeline

Principles are nice, but what do you actually need? The pipeline consists of five components. The first two are off-the-shelf products you choose. The third and fourth are your code. The fifth is a separate AI agent configuration.

Five components

Contracts between components

Tracker ←──REST──→ Orchestrator ──HTTP──→ AI coding agent
                       ↓ bash                ↓ bash
                      Git ←──────────────────┘
                       ↓
                     Gate (bash: lint + test + build)

• Tracker ↔ Orchestrator: REST API. The orchestrator polls the tracker every 30 seconds, picks up tasks labeled ready, respects blocked_by links between tasks (a task is not claimed until its blockers are resolved — critical for multi-task features), and closes the task after the PR merges.
• Orchestrator ↔ AI agent: the agent’s HTTP API. Create session → send work-on-task <slug> command → poll status every 2 seconds → on gate failure, send the output back to the same session (context preserved).
• Orchestrator ↔ Git: shell commands. Create branch ai/<slug>, checkout, pull —rebase, detect changed files via git diff --name-only main...HEAD.
• AI agent ↔ Git: shell commands. Commit in conventional format (feat(scope): description), push to the task branch, create PR via the tracker’s REST API.

Critical permission. Most AI coding agents deny git push and PR creation by default — “for safety.” You need to explicitly allow push to ai/* branches and deny push to main. Without this, the agent can’t complete the cycle, and the orchestrator has to push for it — extra code and a failure point.

Observability through comments. Every state transition — claiming a task, creating a branch, starting a session, gate result, rework, PR creation — is accompanied by a comment on the issue. The full timeline is visible in the tracker, without access to daemon logs. This is critical for trust: you open the task and see what happened, when, and why. Gate failed — the comment contains the full error output. Session hung — the comment notes the timeout. Rework — attempt number and reason. Without this, the pipeline is a black box you’re afraid to look into.

Auto-merge — what makes the pipeline autonomous. When CI on the PR is green, the orchestrator merges via the tracker’s REST API (squash merge). Between green CI and merge there is no “human pressed a button” step. This is the definition of autonomy: from spec to code in main — without a single manual action. If CI is red — rework. If CI is red 5 times in a row — pause, human investigates.

Session lifecycle and crash recovery. The agent session is polled every 2 seconds. Statuses: running → completed (agent finished, gate runs next) or error/timeout. Timeout is 60 minutes of inactivity: if no new messages arrive within that window, the session is killed, the task returns to the queue. The orchestrator keeps state in memory — on restart (crash, deploy) all in_progress tasks are reset to ready and re-claimed. The branch already exists, the code is in place — work continues without loss.

The gate: the key detail

If you get only one thing right from this article — get the gate right. It’s the one component you can’t replace with an off-the-shelf solution.

1. Check: is everything committed (git status --porcelain)
   No → rework: "uncommitted changes detected"
2. git diff --name-only main...HEAD  →  which files changed
3. Dispatch checks by extension:
   .go  → golangci-lint, go vet, go build, go test -cover
   .ts  → eslint, tsc --noEmit, vitest run --coverage
   .sql → DB schema checks
4. All steps blocking: fail on any → rework
5. Coverage must not drop:
   new code without tests → threshold breached → rework
6. Flaky rerun: up to 3 attempts, any green = step passed

The gate runs on the HEAD commit, not the working directory. First confirm everything is committed, then verify that what’s committed works. This guarantees: what the gate checks = what goes into the PR. No surprises on CI.

Coverage must grow, not drop. The coverage threshold is not “minimum 50%” — it’s a regression guard. If coverage was 70% before the agent’s commit and 68% after, the gate should catch it. New code without tests = red gate = rework. This forces the agent to write testable code immediately, not defer “I’ll add tests later.” For control: compare coverage before and after (on main vs on HEAD), not just the absolute value.

Gate vs CI — what’s the difference. The gate runs locally on HEAD before the PR is created. CI runs in a clean environment after the PR is created. Both run lint + tests + build, but CI catches what the local gate can’t: missing dependencies, environment-specific failures, integration with services not running locally. The residual 5% CI failures are this gap.

Custom linters: standard tools won’t enforce your architecture

Standard linters catch syntax and style. Architecture — they don’t. golangci-lint will find an unused variable, but won’t notice that a service imports another service’s library bypassing restrictions, that an interface is named with an I prefix, or that a repository writes raw SQL instead of calling a stored procedure. To a human, these violations are obvious. To an AI agent — not: it sees “working code” and doesn’t understand it broke an architectural principle.

I have 14 custom rules for Go and 5 for SQL. Key ones from the Go set:

The SQL checker adds: ban on dynamic SQL, function naming validation, requirement for get_v0 + get_batch_v0 per table.

These linters are part of the gate. Architecture violation = red gate = rework. Without them, the agent will create code that “works” but is unmaintainable within a week — typical spaghetti, just faster.

Skills + linters: two-level control

Linters are the detective layer: they catch violations after the fact. But it’s better to prevent them beforehand. For this, the AI agent loads skill files (SKILL.md) describing conventions — naming, package structure, design patterns — before it starts writing code.

• Skills explain “how it should be done” (preventive): “interfaces are defined in the consumer package, not the implementation,” “errors are wrapped via fmt.Errorf("pkg: op: %w", err),” “config via functional options.”
• Linters check “how it was done” (detective): if a rule is broken — the gate is red.

Skills reduce violations at the source — the agent makes fewer mistakes because it knows the rules. Linters catch the remainder. One without the other works worse: linters alone — the agent hits the same rules in a rework loop; skills alone — sometimes ignored, and the violation ships to production.

Metrics: 10 weeks in production

Abstract principles are nice, but numbers are honest. Below are real metrics from 10 weeks (April–June 2026), collected via the task tracker API and CI system. The period covers two architectures: the early (unstable) one and the final (single-process + deterministic gate).

Headline metrics

The gap between median (17 minutes) and P90 (~10 hours) — 35×. This is a bimodal distribution: most tasks fly through in minutes, but the long tail (complex codegen, integration tests) drags P90 out to hours. Not a bug — a reflection of task complexity.

For context — manual PRs over the same period: 206 total, merge rate 50%. But this is apples-to-oranges: manual PRs include experiments, WIP, and exploratory branches the pipeline would never attempt. The point is not “AI beats humans” — it’s that for well-specified routine tasks, the pipeline is reliable.

Weekly dynamics

The architectural shift shows up on CI success rate as a break: from 29% to 75% in a single week. This isn’t gradual optimization — it’s a foundation swap. All 7 incremental prompt improvements proposed over this period were not implemented. Metrics grew on architecture alone.

Where AI still fails

Not everything is perfect. Out of 147 unique issues over 4 weeks:

• 130 (88%) — solved with 1 PR, merged on the first try.
• 3 (2%) — required rework (2-8 attempts), but eventually merged.
• 9 (6%) — 1 PR, but discarded (agent failed, PR rejected).
• 5 (3%) — rework exhausted (5 attempts), task paused.

Totals: 133 merged (90%), 14 discarded (10%). Full reconciliation: 147 attempted → 133 closed + 14 discarded.

Rework concentrates in two categories: complex code generation (pointer generation, linters) and cross-service integration tests. This is the “long tail” of complexity — not a systemic problem, but a natural limitation of current models on tasks with high contextual load.

The maximum — one issue required 8 attempts over 29 hours (annotations for pointer codegen). That’s a boundary case, not the norm. The other rework issues fit in 2-3 attempts.

Where humans are still needed

Autonomy is not omnipotence. Humans are needed in three places, and these can’t be replaced yet.

1. Direction and validation. I have an idea — “need feature X” or “fix problem Y.” Then an AI system analyst kicks in: it conducts an interview, writes a specification, and decomposes it into tasks with acceptance criteria, file maps, edge cases, and formal dependencies (blocked_by). I don’t write specs by hand — my job at this stage is to answer the analyst’s questions and apply the ready label after review. Without this step, the system has nothing to do. But it’s hours of work, not weeks. More on how the AI analyst is built — in the article on spec-driven design.

2. The long tail. 6% of issues that require rework often end in a pause after 5 attempts. That’s a signal: either the task is poorly described, or it’s too complex for the current model. I read the gate output (which tests failed, which step is red), refine the spec — usually adding specifics to edge cases or the file map — and restart. Ten minutes of my time, and the task goes back to work.

3. Direction control. The pipeline does what it’s given. It doesn’t decide whether this feature is needed, what the architectural approach should be, whether to rewrite a module. That’s human work — strategy, priorities, system design.

What humans don’t need to do anymore is write routine code, run tests, fix minor bugs, create PRs, merge reverts. Those 80% of a developer’s time are now handled by the pipeline while I focus on what machines can’t do.

Conclusion

Autonomous AI coding is not science fiction and not “in 5 years.” It’s production in 2026. Not for everyone, not for every task — but for well-specified routine development, it works now.

Three principles made it work:

1. Architecture beats prompts — pipeline, retry, deterministic gate deliver 90% of reliability. The prompt is the last 10%.
2. Don’t trust AI to review AI — deterministic tests and builds are more reliable than AI review. The compiler has no opinions.
3. One process beats parallelism — simplicity and stability matter more than the illusion of throughput. One agent, one branch, one task.

The metrics speak for themselves: 90% merge rate, 17-minute median, 133 issues in a month — without a single line of manual code review on my part.

If you’re building or evaluating autonomous AI coding right now — check your system against these three principles. Most often the bottleneck isn’t the model or the prompt. It’s the pipeline around them.

FAQ

How much does it cost per month to run such a pipeline?

The main cost is LLM tokens. Thanks to context caching (cache hit rate of 97%), my cost turned out to be close to zero on the provider’s free tier. Without caching, calculate by the formula: average context per task × number of tasks × price per token. For my volume (200K input tokens per session, 4 tasks/day) that’s a few dollars a day on a paid tier.

What tasks are suited for autonomous coding, and which aren’t?

Good fit: isolated features with clear acceptance criteria (CRUD, endpoints, tests, module refactoring, bugfixes with repro). Bad fit: tasks requiring architectural decisions (designing a new system), cross-team changes, tasks with vague requirements (“make it good”). Simple rule: if you can write a spec from which another developer would do the task without questions — the agent will do it too.

Do I need dedicated hardware for the orchestrator?

No. My orchestrator is a single Go binary running on the same VM where the AI agent lives. Resource consumption is minimal: the main load is HTTP polling of the tracker and running git commands. All the heavy work (code generation) is on the LLM provider’s side.

What to do if the agent gets stuck on a task?

The pipeline automatically pauses after 5 failed attempts and writes a comment with the reason. I read the gate output (which tests failed, which step is red), refine the spec — usually adding specifics to edge cases or the file map — and restart. Usually, the second iteration passes on the first try. If not — the task is too complex for the current approach, and it needs to be decomposed.

Can this be replicated without a team of engineers?

Yes, I did it solo. The section “What it takes to build such a pipeline” lists the five components and the contracts between them. Two of them are off-the-shelf products (tracker and AI agent) that you pick based on your preferences. The orchestrator and gate are your code (~3000-5000 lines), but it’s a state machine with predictable behavior, not a research project. The AI analyst is a separate prompt. The main difficulty isn’t the code — it’s discipline: getting the deterministic gate right (advisory vs blocking is critical), resisting the urge to “help” the agent manually during the process, and writing quality specifications as input.