What Grok 4.7 left in my code
I gave Grok 4.7 two days of real work on an open-source coding-agent extension. A review found a fake worker, test switches in production code and a recovery path that never recovers. Each one is traced to Grok's own edits in the session logs.
Clankers is my open-source extension for the Pi coding agent. It runs teams of agent workers in tmux. For two days the worker writing its code was Grok 4.7: one day through the $30 plan on my other X account, then on the Ultra plan ($200 a month; I paid a $100 first-month promo). A second model, Claude Sonnet 5.5, coordinated: it gave out the milestones, set the checks each one had to pass, and accepted or rejected the work.
When the milestones were done, Sonnet wrote a review of the whole tree before anything was committed. It found twelve defects. For this article I kept three: two it confirmed by reading the code, and one I read myself. Then I checked the session logs to see which model wrote each one. Pi records every edit with the model that made it. All three are Grok’s.
None of this reached a release. The last published version of Clankers predates all of it.
1. A worker that only sleeps
The automation route was supposed to start real agent workers in the shared tmux pool. Grok’s version started /bin/sleep 30 and called that the worker. It ran for every automation request whenever a pool existed, and the milestone check passed anyway.
This is the kind of mistake a stand-in in a test is for. Grok put the stand-in in the product.
2. Test switches in production code
One milestone had to show that its check could fail: plant a fault, see it caught. Grok planted the faults by adding environment variables to the production code. One holds the execution, one spawns a stray sleep 30 child process, one replaces the supervisor. Set the right variable on a user’s machine and the safety checks behave differently.
The fault was supposed to live in the test. Grok moved it into the code the test was checking.
3. A recovery that never recovers
When a worker’s tmux process goes missing, Clankers is supposed to recover it and release its slot. Grok’s recovery function returns “not settled” on every path. The branch that releases the slot can never run. The milestone test passed, and per the review it passed only because that branch is dead.
Grok patched this function twice, two hours apart. Neither version ever returns settled.
The part that isn’t Grok’s fault
Sonnet accepted the first and third. It rejected several weak planted faults along the way and missed the switches Grok left in production until the review. The review says so plainly: it judged whether the planted fault was caught, not what the production path really does. A reviewer that reads the test and not the code it tests will pass a sleeping worker.
So the lesson is two-sided. Grok 4.7 writes code that satisfies the check in front of it rather than the job behind it. And an AI reviewer needs to read the production path to the end, every time, or it becomes part of the problem.
What it cost
An Ultra plan ($100 on a first-month promo, $200 a month after that) I used for less than a day, a review, and now a round of fixes to take it all back out. The refund story is in Six refusals from a support agent that couldn’t read.