AI infrastructure case study

AUTONOMOUS
ENGINEERING

Problem

Engineering time is finite, and most of it competes with a day job. The question behind this project was narrow and practical: could a personal engineering backlog keep moving between 1 a.m. and 7 a.m., without a human present, without paying frontier-model API rates for every task, and without silently losing work if something broke overnight?

The constraints made this harder than "run an agent in a loop." A local model running on a personal machine is free but limited in context and reliability. Cloud coding agents are capable but cost money per task and are easy to over-provision if used for everything. And any system that runs unattended for six hours needs a way to know, the next morning, whether it actually did what it reported doing.

Approach

The system is built around three roles rather than one model:

  1. Task source. A brainstorming pass (using a larger hosted model) turns rough ideas into concrete tasks, which are created and tracked as issues in Linear. Linear is the single source of truth for what is queued, in progress, or done.
  2. First-pass local delegation. A cron-driven startup script brings up LM Studio and loads Qwen3.6-35B-A3B locally on a Mac with 48GB of RAM. Simple, narrowly-scoped tasks are dispatched to this local model first, since it costs nothing to run.
  3. Overflow to paid agents. Tasks that exceed the local model's context budget or complexity are routed to Codex CLI, Claude Code, or a larger hosted model (GLM-5.2 via OpenRouter), which run externally and are not subject to the local model's context limits.

Architecture

Linear (task source)
      |
      v
1am-7am cron dispatcher
      |
      v
 Try: local Qwen3.6-35B-A3B (LM Studio, 32K context)
      |
      +--> succeeds (task is small/simple) --> write output --> verify artifact
      |
      +--> fails / too complex
              |
              v
      Overflow: Codex CLI / Claude Code / GLM-5.2 (OpenRouter)
              |
              v
      write output --> verify artifact
              |
              v
   Verification gate (confirm output files and
   loaded model context actually exist, not just
   a reported status)
              |
              v
   Linear reconciliation (Done only if verified, else Backlog)

Outcome and Lessons

The most important result of this project was not a feature. It was catching a failure mode where the system looked healthy while doing nothing.

For ten nights (July 7-17), every task dispatched to the local Qwen3.6 model failed immediately: each call hit a context-length-exceeded error after a single API call, completed in two to three seconds, and wrote no files. The delegation framework's own status field showed "completed" for these runs, which was an artifact of the process exiting, not a genuine report that any work had been done. The cause was a mismatch between LM Studio's default loaded context window (8,192 tokens) and the actual context injected by the delegation framework for each subagent (system prompt, tool schemas, and task prompt, typically 10-20K tokens). The model was technically loaded and would answer simple prompts directly, so a basic health check ("is the model running?") passed every night. But the moment a real delegated task arrived with its full context payload, the request exceeded the loaded window and failed after that single API call. The system's fallback logic routed the actual work to paid APIs, so tasks still completed, but the local model contributed nothing for over a week while consuming 20GB of RAM.

The fix was straightforward once diagnosed: reload the model explicitly with --context-length 32768 rather than relying on the API request to override context length (it cannot), and verify the loaded context via the model listing endpoint before declaring startup successful. The startup script now checks loaded_context_length directly against the running model and refuses to proceed if it falls short, instead of trusting that a loaded model is a working model.

The broader lesson was procedural, not just technical: a subagent's own "completed" status is not evidence that anything happened. The local model still has a real role in this system, but only for short, narrowly-scoped tasks that fit comfortably inside its verified context budget. Everything else goes to Codex CLI or Claude Code, and a task is only marked Done in Linear after its actual output, not its reported status, has been checked.

All case studiesNext: Discord CLI