Blackreach said incomplete. I needed to know why.
The browser opened. The page was blank. Then Blackreach stopped with Status: Incomplete.
I was trying to get it ready before a call, so the timing was pretty bad. The call ended up going well and I didn't need to show a demo. But I still wanted to understand why a tool I'd built could stop without giving me a useful explanation.
I went back through the Python engine with Codex. The interesting part was how several small assumptions had lined up.
The HTML was there. The source page wasn't.
One navigation path fetched a page over HTTP and inserted the returned HTML into the browser. That sounds reasonable if the only job is reading static text. An interactive agent also needs the page's address, relative resources and navigation context.
The browser could still be at about:blank. A relative link like /report.html no longer had the source site's origin behind it. Looking at the HTML alone made the failure harder to see.
The fix was to use real navigation for the interactive path. The regression test serves a page locally, loads its relative script, follows its relative link, and checks the destination. It's a small test, but it asks the question the old implementation got wrong.
Incomplete had too many meanings
The terminal used one label for several different outcomes. A step limit, a provider failure and an interruption all needed different next steps. The result needed to carry that distinction all the way back to the person running it.
There was also a mismatch inside the loop. One path returned a fatal flag; the loop checked a recoverable flag. Separately, the reported step count came from the action history, so a model that read the first page and finished could appear to have taken zero steps.
The engine now has explicit statuses and stop reasons. Decisions and browser actions have separate counters. A failure takes precedence over a contradictory completion signal, and a resume hint only appears when a checkpoint was saved.
Stopping is part of the workflow
Ctrl+C had its own problem: the command handler could exit before the agent reached its checkpoint logic. Once that path was corrected, the next question was whether the saved state actually worked in a new run.
The recovery example opens a local fixture, clicks its report link, and stops at a one-decision limit. Then a new agent and a new browser resume from SQLite. The final count is two decisions and one action. The first click isn't repeated.
A second case simulates a provider rejecting access. It must stop after one request, retain a useful error and write the reason to the session-end log. Increasing retries wouldn't fix an account that can't use the selected model.
Current reasoning-model APIs added another boundary: a response can hit its output limit before usable action JSON is complete. The adapter now rejects that partial result. It doesn't silently increase the token budget.
What I can claim from this
The browser and database in that example are real. The decisions are scripted. That makes it useful for checking navigation and recovery, but it says nothing about how well a live model will plan an unfamiliar task.
A local Qwen 2.5 7B smoke test made that distinction concrete: it reached the report page, then tried an invalid click and ran out of its three-decision budget. The engine kept the error and checkpoint. The model still failed the task.
Then I tried a documentation comparison
For a separate live-model run, I asked Blackreach to compare Playwright, Puppeteer and Selenium from their official docs. It visited all three and returned a linked comparison in about 84 seconds. The run recorded six decisions, four browser actions, six page visits and two failures.
There was a bad URL with a trailing comma, and a detour through GitHub. The answer also listed Kotlin as if it were a separate Selenium binding; Kotlin uses the Java bindings. I corrected that distinction in the video's editorial recap and kept the original wording in the evidence receipt.
I made a 30-second showcase from the run. The terminal output is re-set for readability, with generated background motion. It's useful for explaining the tool, while the receipt shows the less tidy path it actually took. One completed task still doesn't tell me how often the agent will succeed on unfamiliar sites.
It also doesn't make every external action safe to repeat. If a process dies during a click, the page may have changed before the checkpoint was written. Login state isn't part of this checkpoint either. Those are limits I want visible.
I use AI coding tools heavily, and this pass was developed with Codex. The useful evidence is the failure I can reproduce, the change that addresses it, and the test that would catch it coming back.
That's the part of browser agents I want to keep working on: being able to look at a run and understand where it is, what happened, and what I should do next.
Inspect the case study and evidence · Public Blackreach repository