Sprint Notes: What Shipped in the Past Six Weeks

Why we publish sprint notes

Most companies publish blog posts about features after they have been polished, tested, and positioned for marketing value. That is fine, but it creates a gap between when work happens and when it gets documented, and a gap between the actual development experience and the story told afterward.

These notes are different. They are a plain record of what shipped over the past six weeks — what the work actually was, what it took, and what the state of things looks like now. No narrative gloss. Just the log.

QA gate hardening

The QA agent gate that was described in detail in an earlier post shipped in its initial form in week one of this period. Over weeks two through four, we hardened it based on what we learned from running it on real PRs.

The main changes: dependency validation was added as a pre-run step, not an afterthought. The test-scoping logic was improved to handle monorepo structures where a change in one package can affect tests in another. Failure output capture was made more robust — the early version occasionally truncated output for long test runs, which made rejection tasks less useful. The retry logic for transient failures (test environment startup race conditions, external service timeouts) was tuned down to avoid masking real failures as transient ones.

Total PRs reviewed by the QA agent in this period: 47. Gate rejection rate: 34%. Of those rejections, all resulted in a follow-up PR that passed. Zero false positives escalated to human review.

Budget auto-split

The auto-split mechanism for over-budget tasks shipped in week two. The implementation went through two designs.

The first design decomposed tasks after they had already exceeded the budget — the split happened as cleanup. This meant the overrunning agent had already done some unknown amount of work before the split, and the sub-tasks had to account for partial progress in ways that were hard to specify cleanly.

The second design added a budget-check at regular intervals during execution, allowing the split to be triggered proactively when the task was projected to exceed the budget rather than reactively when it had already done so. This is the version that shipped. It requires agents to emit progress signals at defined intervals, which was an API change, but the resulting sub-tasks are cleanly scoped from a known state rather than from a partially-completed unknown state.

Thirty-one tasks auto-split in this period. Average sub-task count per split: 3.4. None required human escalation after splitting.

Onboarding paywall fix

Before this period, the paywall appeared immediately after account creation — before the user had seen any of the product. This was the default configuration from early development that we never changed.

The fix moved the paywall to after the onboarding demo, which runs a live agent task in a sandboxed workspace. The demo takes between eight and twelve minutes for most users. The paywall presentation is triggered by demo completion, not by account age.

The implementation involved: building a demo completion signal into the agent runtime, wiring the completion signal to the paywall trigger, designing the transition screen that connects the demo to the upgrade prompt, and making sure the paywall skips gracefully for users who are already on a paid plan or who have a trial active.

This change took longer to implement than it should have because the paywall logic had several implicit dependencies on account state that were not well documented. We found them by breaking things in staging.

Billing integration

Stripe billing went from partially implemented to fully operational in week three. The outstanding items were: webhook handling for subscription lifecycle events (created, updated, canceled, past-due), the customer portal flow for users managing their own billing, and the admin view for the team to see subscription status without touching Stripe directly.

Webhook handling is the part that is easy to get wrong. We added an idempotency check on all webhook handlers — the same event can be delivered more than once, and the handler has to produce the same outcome on the second delivery as on the first. We also added an explicit dead-letter queue for webhook events that fail processing, so a failed handler does not silently drop an event that should have updated account state.

The billing integration is tested with Stripe's test mode event simulator. Every webhook event type has a corresponding test that verifies the outcome. Switching from test mode to live keys is the only remaining step.

Trial tier

The trial tier shipped in week four. Trial accounts get full access to the platform for fourteen days, capped at a limited number of agent-hours. At the end of the trial, the account is prompted to upgrade. If it does not, access is restricted to read-only.

The fourteen-day window was a decision we made without strong data, based on the observation that users who are going to see the value of the platform will see it within the first week of real use. The second week is for teams that need time to buy — getting budget approval, looping in colleagues, going through a procurement process. Two weeks is long enough for both.

The agent-hour cap exists to prevent trial accounts from running expensive workloads at zero cost. The cap is set conservatively — high enough that a genuine evaluation can complete, low enough that it is not a meaningful subsidy to users who are clearly not converting.

Eleven trial accounts are currently active. None has converted yet. The trial period is still open for most of them.

What is next

The trial conversion data will be the main input to the next sprint. We will have a clearer picture of where users are getting stuck once the first cohort of trials either converts or lapses. The qualitative signal from those outcomes is more valuable than any metric at this stage.

On the engineering side: the agent runtime needs performance work — some workloads that should complete in under a minute are taking three to five. The root cause is not identified yet. That investigation is the first thing on next sprint's backlog.

That is the six weeks. Plain as we can make it.