neatlogs
Traces every run.
The planner, executor and verifier show up as spans, so a bad run has a cause you can read.
neatlogs.com
Docs that work (literally).
Paste a docs URL. Your coding agent runs the quickstart in a clean container, finds the step that breaks, fixes it, and proves the fix.
Try
A partial URL works. So does a tool name. Nothing leaves your browser.
Verified
Watch the replayA saved run. Nothing to install.
Run it fresh with your agent
- Your agent installs FirstRun and checks Docker
- Runs your quickstart in a clean container
- Fixes and proves it in a second clean run
- Opens the PR when you say yes
Read
FirstRun reads the page and plans one step per command.
Run
Every step runs in a clean container, like a new developer. The first failure stops the run.
Prove
A fix is not done until a second clean container passes the whole quickstart.
Watch a Real Run.
Every run ends with a share link. Replay one.
Known-broken fixtureA pip pin that cannot install
dbt + DuckDBA real quickstart
DagsterA real quickstart
$ pip install git+https://github.com/HighnessAtharva/firstrun
$ firstrun https://docs.getdbt.com/guides/duckdb
$ firstrun pr <run> # prints the gh commands for the docs PR
Prefer the terminal? You need Docker and a Claude token from claude setup-token.
Three Tools. One Job Each.
Entire
Records every session.
Each commit carries the Claude Code session that wrote it, so any line traces to a prompt.
entire.io
cfo.ai
Turns cost into a plan.
The measured cost per run from neatlogs goes to Ari, the cfo.ai agent, which models the pricing.
cfo.ai
One Engine, Three Answers
A clean container and stored commands answer three questions a docs owner asks.
Does my quickstart work?
Break → Fix → Prove
Runs the page in a fresh container, patches what breaks, and proves the patch.
firstrun <url>
Replay a broken fixture
Which release broke it?
0.2.10.3.10.4.60.4.100.4.14
The dbt guide step jafgen --years 6 broke in jafgen 0.4.6, 11 months before the guide shipped.
firstrun bisect <run> --package jafgen
Replay the dbt bisect
What does it silently assume?
curlca-certificatesgnupgsudoproven 5/5
The AccuKnox install steps need four packages the docs never name. A fresh container proved the list.
firstrun prereqs <run>
Replay the AccuKnox proof
Bisect and Prerequisite Proof replay stored commands, so they cost $0 in model calls.
The Numbers Come From Real Runs.
4upstream docs PRs opened on 3 doc sites. Each fixes a step a new developer hits.dbt #10149 dbt #10150 Entire #2724 AccuKnox #671
5 of 5open-source quickstarts verified in a fresh container
2 → 3of 4 quickstarts finish, v1 to v2
0 → 3failed steps recovered
$0.02model cost per run, measured by neatlogs (list price, not billed)
Every PR, in Public.
Opened upstream on real doc sites. Status and stars read live from GitHub.
dbt-labs/docs.getdbt.comOpen
Fix the jafgen command in the DuckDB quickstart #10149
Broke: jafgen --years 6 fails with No such option: --years.
Fixed: years is now positional, so both mentions change.
- +2 −2 in 1 file
- 218 repo stars
- Opened Oct 10, 2026
dbt-labs/docs.getdbt.comOpen
State the Python version the DuckDB quickstart needs #10150
Broke: the cloned repo pins networkx==3.7, which needs Python 3.12.
Fixed: the guide now names the Python version.
- +1 −0 in 1 file
- 218 repo stars
- Opened Oct 10, 2026
entireio/cliOpen
Say where install.sh puts entire on Linux #2724
Broke: after a good install, entire version fails with command not found.
Fixed: the docs say the binaries land in ~/.local/bin, which a clean shell lacks on PATH.
- +2 −0 in 1 file
- 5,169 repo stars
- Opened Oct 10, 2026
Also run, no PR needed
AccuKnox knoxctl verified 5/5
MkDocs verified 9/10
Dagster verified, no docs PR after a source check
How FirstRun Compares.
| Feature |  FirstRun | Claude Code | GitHub Actions | GitHub Agents | Custom workflow |
|---|---|---|---|---|---|
| Runs the quickstart as a new developer | Yes. Every step runs in a fresh Docker container | Runs on your machine unless you set up a sandbox [FILL: sandbox setup] | Yes. A fresh runner per job, but you script each step | [FILL: execution environment] | Depends on the build [FILL: environment] |
| Plans steps from a docs URL | Yes. The planner writes one step per command | Yes, when you prompt it | No. You write each step | [FILL: input it accepts] | You write the parser |
| Improvises when a step failsStands out | Yes. It classifies the failure, patches, and retries up to 3 times per step | Yes, inside one session | No. The job fails and stops | [FILL: retry behavior] | Only what you code |
| Proves the fix | Yes. A second clean container must pass the whole quickstart | Only if you ask | Only if you script it | [FILL: verification step] | Only if you code it |
| Telemetry per runStands out | Yes. neatlogs spans for planner, executor and verifier. Measured cost of $0.02 per run | [FILL: telemetry setup] | Logs and run history. No agent traces | [FILL: telemetry] | You build it |
| Opens the docs PR | Yes, after you approve once | Yes, when you ask | Only if you script it | [FILL: PR behavior] | Only if you code it |
| Ease of useStands out | Paste a docs URL, then paste the prompt into your coding agent | One prompt, no failure classes | A YAML file per quickstart | [FILL: setup steps] | Highest effort |
| Proof in public repos | 3 upstream PRs, all open (see the record above) | [FILL: examples] | [FILL: examples] | [FILL: examples] | [FILL: examples] |
FirstRun reads a docs quickstart and runs every step in a clean container, the way a new developer would. It stops at the first failure and classifies it. It tries a fix up to 3 times, then proves the fix with a second clean run. It asks once before it opens a PR.
FirstRun opened 4 upstream docs PRs, and all are open today. The dbt DuckDB quickstart had a broken jafgen command (#10149) and no stated Python version (#10150). The Entire CLI installer put entire on a path that a clean shell lacks (entireio/cli#2724). The AccuKnox knoxctl page named 0.9.0 as the latest release while its installer fetched 0.9.58, and its APT steps assume curl, gnupg and sudo (accuknox/help#671). MkDocs and Dagster passed with no docs change, so a run also shows when docs hold up.
Every run is a neatlogs trace, so a bad run has a readable cause. A docs bug surfaces in a run instead of in a reader who leaves without filing an issue. [FILL: merge status after the maintainers respond]
The Build, in Public.
Every step is a post on @cultist_dev.
- Sat Oct 10, 10:45
my neatlogs hackathon project has already caught doc issues for a chroma (24K+ github stars). chromadb's deprecated-config error sends you to its own migration guide. that page returns 404.
Read the post on X - Sat Oct 10, 12:30
#neatHack kickoff, 12:30 IST. FirstRun code before kickoff: 0 lines. neatlogs doctor probe: 7/7 PASS. Solo build. Submission opens Monday, 7 PM IST.
Read the post on X - Sat Oct 10, 13:14
#neatHack, day one. all three required tools are up. Entire: trail #1 created. neatlogs: first agent trace, 3 steps, 12s. cfo.ai: Ari read my public profile.
Read the post on X - Sat Oct 10, 13:26
your quickstart is the only part of your product tested by people who leave instead of filing a bug. so for #neatHack i'm building FirstRun. break. fix. prove.
Watch the video on X - Sat Oct 10, 15:15
a quickstart breaks the way an agent breaks. quietly. so the plan for this weekend: every FirstRun run becomes a trace in @neatlogs.
Read the post on X - Sun Oct 11 to Mon Oct 12More posts land here as the weekend goes.
Atharva Shah
Atharva is a DevRel engineer. He owns developer docs at AccuKnox and works with Altimate AI.
A quickstart that breaks without a sound is a docs bug nobody reports. FirstRun is his answer. He built it alone during the hackathon, in public.