All posts
2 min readby Romiel Inolino

AWS Emulator, built entirely by agents, shows why agents need a test setup: one stalled for four nights, then finished in two hours with a spec

AWSAI coding agentstestingengineering

The best line in AWS's emulator launch is a confession: an agent was supervised for four nights before anyone saw it had no way to check its own results.

What happened

AWS released AWS Emulator, a local copy of AWS that coding agents can test against, free and offline on your own machine. In a guest post on Werner Vogels' blog, Swami Sivasubramanian writes that all of its code was written and tested by agents. It is built from more than 400 API models covering more than 200 AWS services.

The agents work from fidelity tests, end-to-end scenarios checked against real AWS, then build the emulator until it passes the same tests. Every change goes through a merge gate with automated checks, including several security agents doing adversarial reviews. The first prototype, an SQS emulator, came back from one prompt to Kiro in about 15 minutes. The code is public domain under CC0, and a folder in the repository lists behaviors that do not yet match real AWS.

Swami also shares a personal test. He used Kiro to rebuild a replication engine that originally took about four months to build by hand. He supervised the agent for four nights before realizing it had no way to check its results. Once it had a spec and a test environment, the rebuild took about two hours.

Simon Willison adds that it is written in Rust, logged 29,684 commits in its first three weeks, and currently needs Docker on macOS and Windows.

My take

Most of my work is not on AWS, but the lesson transfers straight to CRM and automation builds. When I let an AI coding agent help with a client workflow, the slowest runs are always the ones where it cannot see whether its change worked.

What I set up before an agent touches a client build:

  • A sandbox CRM account or a staging copy, never the live pipeline.
  • A short written spec: the inputs, the expected records and the fields that must never change.
  • A handful of test records with known correct outcomes, so the agent can run, compare and fix on its own.

Four nights versus two hours is the gap between a vague request and a checkable one.

More posts