Letting AI Debug Production

Fri, 04 Sep 2026 00:00:00 +0000

It’s 2 in the morning, and you’re reading an incident channel: checkout is failing, orders are stuck, and support is starting to field angry messages from customers who can’t get past payment. After quite a bit of digging through the logs, you find the cause. It was a rejected payment call buried deep in thousands of entries. And you’re left thinking: this is exactly the kind of grunt work an AI agent should be able to take off your hands.

For AI coding assistants, writing new code is the easy part. The hard part has always been maintenance. But you can’t just set an agent loose in production. It can easily corrupt the database, and all it can do afterwards is apologize and promise not to do it again.

An agent debugging a codebase can read the code and guess, or execute the code and risk side effects. Test suites by themselves are not enough because a production incident is, by definition, a case the tests didn’t cover. What AI, or humans, for that matter, actually need is a way to run the failing scenario with the real data, without touching production.

Deterministic replay solves exactly this problem. If every side effect the business logic performs is recorded as it happens, and the logic can later be rerun against those recordings, a production failure effectively becomes a sandbox.

This is the core idea behind Pure Effect, a lightweight JavaScript/TypeScript library I’ve been developing for almost a year now. Business logic returns plain data describing the I/O it wants. Then an interpreter executes it. Because a flow is data until interpreted, a production run can be recorded into a trace: what every step returned or threw. And here’s the neat trick: that exact trace can be fed right back through the same flow later, completely offline without a database or network connection.

This general approach isn’t new. Record/replay debugging has existed in various forms. What’s different here is that the business logic is expressed as plain data, and the library needs no infrastructure to run.

Think of the trace as a flight recorder, and replay as the simulator loaded with it. During replay, your business logic runs for real, every branch, every decision, but each time it reaches a step that would talk to the outside world, it gets the recorded answer from the incident instead. The function that would call your payment provider never actually runs.

Giving an AI agent the failing trace allows it to rerun the incident as many times as it needs. If a change alters which steps execute, the replay says so. If the fix changes how the flow handles a recorded answer, the same recording now plays through to success. The worst thing an agent can do in this loop is be wrong, and being wrong costs some tokens, not a duplicate charge to a customer.

One nice side effect is that the incident itself becomes a permanent part of the regression test suite, and since there’s no I/O, the test runs in microseconds.

Here’s our deliberately buggy checkout flow. Each function returns either a plain result or a description of one I/O step. Nothing here directly executes any I/O.

import { Success, Command, effectPipe } from 'pure-effect';

const fetchCart = (order) => {
    const cmdFetchCart = () => db.getCart(order.cartId);
    return Command(cmdFetchCart, (cart) => Success({ ...order, cart }));
};

const applyPromo = (order) => {
    const cmdValidatePromo = () => promos.validate(order.promoCode);
    return Command(cmdValidatePromo, (promo) =>
        promo.valid
            ? Success({ ...order, total: order.cart.total * (1 - promo.discount) })
            : Success({ ...order, total: order.cart.total })
    );
};

const chargeCard = (order) => {
    const cmdChargeCard = () => payments.charge(order.customerId, order.total);
    return Command(cmdChargeCard, (receipt) => Success({ ...order, receipt }));
};

const checkoutFlow = (order) => effectPipe(fetchCart, applyPromo, chargeCard)(order);

Every step in effectPipe receives the value the previous step produced, so the pipeline reads top-to-bottom as the flow of data. Each I/O step is wrapped in a Command: a function that makes the I/O call, plus a “next” function that receives the answer and decides what happens next: a Success, a Failure, or another Command.

No one wrote the case where a promo discounts the cart to zero, so when a 100% VIP promo is used, applyPromo computes a total of 0, chargeCard dutifully asks the payment provider to charge nothing, and the provider rightfully refuses.

In production, recording is a hook installed once at startup, so failure traces are captured without touching the existing code. We can record specific flows with recordEffect as well:

const { result, trace } = await recordEffect(checkoutFlow, order);
// result.type === 'Failure', and trace is plain JSON

The recording can be replayed without needing access to the production environment:

await timeTravel(checkoutFlow, trace);

Replaying 'flow' (3 recorded steps)
Initial input: { "cartId": "cart_42", "promoCode": "FREE_YEAR_VIP", ... }
Step 1: cmdFetchCart returned { "total": 120 }
Step 2: cmdValidatePromo returned { "valid": true, "discount": 1 }
Step 3: cmdChargeCard threw { "message": "Amount must be non-zero.", "code": "invalid_amount" }
Replay finished with state: Failure

Three lines make the whole flow visible: valid promo, full discount, zero-amount charge attempt. No log hunting or reproducing the customer’s cart in staging is necessary.

With the trace at hand, an agent can replay the incident, spot the unconditional charge, and produce a fix:

const chargeCard = (order) => {
    if (order.total <= 0) return Success({ ...order, receipt: { amount: 0, waived: true } });
    const cmdChargeCard = () => payments.charge(order.customerId, order.total);
    return Command(cmdChargeCard, (receipt) => Success({ ...order, receipt }));
};

For verification, the agent replays the same incident recording against the fixed flow:

const { result, unreached } = await replayEffect(checkoutFlow(trace.initialInput), trace);
// result.type === 'Success', result.value.receipt.waived === true
// unreached.map((e) => e.command) is ['cmdChargeCard']: the charge was never issued

The recording that used to end in a failure now plays through to success because the flow no longer asks the payment step to run. The final commit includes the incident as a permanent test:

it('incident 42: a 100% promo checks out without a charge', async () => {
    const { result, unreached } = await replayEffect(checkoutFlow(trace.initialInput), trace);
    assert.equal(result.type, 'Success');
    assert.equal(result.value.receipt.waived, true);
    assert.deepEqual(unreached.map((e) => e.command), ['cmdChargeCard']);
});

You’ve probably noticed that replay cannot confirm fixes that insert or reorder steps in the pipeline, so a fix that alters the step sequence causes replay to stop at that exact point. This is by design to ensure that the simulation never violates causality. Asking for a different step than the one recorded at that point halts with a literal TimeParadox error.

In practice, there is a 2-tier solution for this:

Tier 1 (Agent): Tries to fix the bug within the existing effect topology (fixing edge cases, null checks, etc.). Verifies via replay.
Tier 2 (Human): If the fix requires architectural changes, new external I/O, or a change to what a failing step sends, the agent gives up and pages the engineer on call.

Pure Effect doesn’t require a rewrite of everything you own. Structure the logic you actually care about as flows. Install recording once at startup, keep the failure traces, and run redact over anything sensitive before a trace leaves the process. From there, the rules for your agent are simple: keep pipeline steps pure, push all non-determinism into Commands, and don’t alter the execution sequence.

Granted, letting an agent auto-patch a live billing flow might be pushing it. Maybe someday we’ll trust them with that. But even today, an agent can verify a fix offline and open a pull request with an attached regression test before paging you.

GitHub Repository: pure-effect

Ai on Lack of Imagination

Letting AI Debug Production