A few years ago, the idea of running a large language model on a custom FPGA would have sounded like a research project. Now, with the help of coding agents, you can get from zero to a working, fast inference chip in days—even if you have never touched Verilog.
That's the story here: the author, who had never programmed an FPGA before, used AI coding agents (mostly Codex and Claude, with both Astra and Opus models) to build a working LLM inference pipeline on an AWS F2 instance. The agent wrote almost all of the C++ and Verilog, ran experiments, and improved throughput from a baseline of 6 tokens per second to 213 tokens per second—a 33× speedup. The total experiment cost was about $10,000 in API tokens. Most prompts were simple encouragement or context, not detailed technical instructions.
The agentic workflow: from zero to running model
The project started with a basic goal: see if a coding agent could get a model running on an FPGA, a domain the author had never worked in. Using AWS F2 (16 GB HBM), the agent handled most of the C++ and Verilog needed to port the DeepGrove ternary weight architecture. The author mostly checked in periodically, offering occasional ML advice (like suggesting speculative decoding or moving a slow operation off the CPU), but most of the heavy lifting was done by the agent.
The process involved a cloud agent running for several days straight, iterating through hundreds of experiments, and ultimately generating about 3 million lines of code and parsing 30 million lines of output. The author reports that both Claude Opus and Astra models were up to the task.
Optimisation sequence: 6 to 213 tokens/sec
The agent didn’t just get a model running—it made it fast. Key optimisations included:
- A head loader that reads both memory banks at once, giving a 17.3% boost (measured on/off)
- A runtime expert-prefetch policy
- A router-job timing fix
- Dropping the clock speed slightly after a soak test found random faults at higher frequencies (down to 225/425 MHz from 231.25)




