A retry is a write

You tap Pay, the spinner spins, and the app says "Something went wrong. Please try again." Did they charge you?

You don't know. And here's the uncomfortable part: neither does the app.

The request left your phone. Somewhere, a server may have received it, charged your card, written a row to a database and sent back "OK". Or the request died in a tunnel before it reached anyone. From your side of the wire, those two worlds look exactly the same: a spinner, then nothing.

That is the maybe state. Every system that talks over a network lives in it, and most code is written as if it didn't exist.

A timeout is a fact about you

When your code says a request "timed out", it sounds like news about the server. It isn't. A timeout means you stopped waiting. It tells you how patient you were and nothing about what happened on the other end.

Hanging up does not cancel the pizza.

Brandur Leach, then at Stripe, listed the ways a call can fail back in 2017, and the list is worth memorizing. The connection can fail before it reaches the server. The call can fail midway, "leaving the work in limbo." Or the call can succeed and the connection can break before the server tells the client. Only the first one is safe to retry blindly, and from the client you can't tell which one you got.

A better network won't save you. The textbook version of this is from the 1970s, the Two Generals' Problem, and it was proved unsolvable in 1975. Two generals agree on an attack by messenger, and any messenger can be captured. No number of confirmations, or confirmations of confirmations, makes both of them sure. The last messenger is always a maybe.

So the client gives up waiting and does what every tutorial tells it to do. It tries again.

Some writes repeat. Yours don't.

The HTTP spec draws this line for you. The current standard, RFC 9110, calls a method (the verb on a web request, like GET or POST) idempotent when sending it ten times has the same intended effect as sending it once. GET and the other read-only methods are on the list, and so are PUT and DELETE. Fetch a page ten times and you have the page. Delete a file ten times and it is just as gone. Those repeat safely because they describe an end state, not an action: make this file gone. POST, the method behind almost every "create this" and "charge that", describes an action, and doing an action twice is twice. It is not on the list.

The RFC is blunt about what follows: a client "SHOULD NOT automatically retry a request with a non-idempotent method" unless it has some way to know the request is safe to repeat, or that the first one never landed.

Now go look at the retry helper in your codebase. I'd bet it wraps every call the same way, three attempts and a sleep, as if every call were a read. It works for years, because most failures really are the first kind. Then one afternoon an answer gets lost on the way back, and somebody gets two orders, two welcome emails, two charges.

A retry is a write. Treat it as one until you can prove otherwise.

Make the second write do nothing

The proof most people reach for is an idempotency key, and Stripe made it famous. The client generates a unique ID for the operation (a long random string is fine) and sends it in an Idempotency-Key header. The server writes it down. In Stripe's v1 API, once the work starts, the server saves the status code and body of the first request with a given key, and for at least a day every retry with that key gets the saved response back, even when the saved response was a 500 error. Send the same key with different parameters and Stripe refuses, because that is a bug wearing a retry's clothes.

The trick works because it turns "did this happen?" from a question about the past into a lookup. The client only needs to ask the same question the same way, and the server answers from its notes.

One detail decides whether any of this works. The server has to write down "I'm working on this key" before it does the work, not after. Otherwise two copies of the same request arrive a millisecond apart, both look up the key, both find nothing, and both charge the card. The claim has to be taken once: a database rule that lets only one row carry that key, so when two copies race to insert it, exactly one wins.

The IETF draft that would make the header a standard even has a status code for that moment. A retry that arrives while the original is still being processed should get a 409 Conflict. So even the fix has a maybe state. It just gives it a name.

Retries are selfish

Say every write is idempotent now. You still aren't done, because retries do damage even when they're correct.

Marc Brooker's essay in the Amazon Builders' Library says it plainly: retries are selfish. A client that retries spends more of the server's time to raise its own chance of success. When the server is slow because it's overloaded, every client doing that at once makes it slower. And retries stack. In his example of a call that goes five services deep, where every layer makes three attempts on its own, the database at the bottom sees 243 times the load (three, to the fifth), "making it unlikely to ever recover." Amazon's stated practice for cheap calls is to retry at a single point in the stack, not at every layer.

The other half of the fix is to stop clients from retrying in lockstep. Exponential backoff doubles the wait after each failure. Jitter adds randomness, so a thousand clients that failed at the same moment don't all come back at the same moment. Brooker compared the variants in 2015, and the best jittered versions showed "a substantial decrease in client work and server load."

One subtlety bites people. A Retry-After header is the server telling you how long to wait before you come back. "Full jitter" picks a random wait anywhere from zero up to the current backoff. Apply that to the server's number and you will sometimes come back after zero seconds, the same instant as everybody else who got that answer. Add the jitter on top of Retry-After, never under it.

When you can't ask

Everything so far assumes the other side cooperates. It accepts a key, it keeps notes, it tells the truth on a retry. A lot of software does none of that.

I build browser agents: programs that drive web apps the way a person would, because the app has no API, no door a program can use. There is no idempotency key on a Save button. The agent fills in a form, clicks Save, and waits for the page to show the new record's number.

The dangerous moment is the gap between the click and reading that number. If the run dies before the click, nothing happened, and running it again is safe. If it dies after the number is on screen, you know the record exists and you know which one it is. If it dies in between, you're back on the phone with the pizza place, asking whether the order went through, and nobody picks up.

A retry there is a guess that writes to somebody else's system.

So I don't retry it. The job parks in a state that means "maybe created", with a note saying exactly where it died, and a person checks before anything runs again. That feels slow until you compare it with a duplicate record in a system you don't own, found and cleaned up by hand after somebody already acted on it. Guessing quietly to keep the run green is how silent fallbacks are born.

The rule I use is short. If the outcome is unknown and the write can't be undone, the next step belongs to a human.

I opened by telling you a timeout says nothing about the other side. You still need one.

Closing the browser is a write too. It frees a session that somebody pays for and a slot that the next job is waiting on. And if await browser.close() simply never returns, nothing after it runs: not the cleanup, not the log line, not the next job in the queue. A try/except won't help, because there's no exception to catch. Nothing failed. It's still waiting.

A try block can catch a failure. It cannot catch forever. Only a clock gets you out, so every step that touches the outside world gets a deadline, including the one that says goodbye. And when the deadline fires, you're back in maybe: the session might be closed, or still running on someone's bill. Record it that way.

---

Most retry bugs come from code that can only say two things, "done" and "failed", in a world that keeps answering a third. Give maybe a name, a row in a table and an owner, and it stops being an incident and becomes a queue.

The honest answer to "did it go through?" is sometimes "I don't know yet." Build systems that are allowed to say it.