8 min read · 16 December 2025

Your fallback is lying to you

A fallback that nobody hears about is worse than a crash. The crash, at least, is honest.

Here is a function I've met, in one shape or another, in nearly every codebase that talks to a language model. You don't need to read Python to follow it. Count the times it says return None, which is Python for "nothing".

def ask_model(prompt):
    if not API_KEY:
        return None
    try:
        resp = http.post(URL, json={"prompt": prompt}, timeout=30)
    except TransportError:
        return None
    if resp.status_code != 200:
        return None
    try:
        return resp.json()["choices"][0]["message"]["content"]
    except (KeyError, IndexError, TypeError, ValueError):
        return None

Now read it the way the caller does. None means one of four things. Nobody configured a key. The provider said no. The network ate the request. The answer came back in a shape nobody expected.

Those are four different problems with four different owners. One is a deploy mistake. One might be your bill. One is weather. One is a contract that changed under your feet.

The caller can't tell them apart, so it does the only sensible thing a caller can do with a None. It takes the backup path, a pattern match, a cached answer or a default, and carries on.

Four failures go in and one value comes out. The line to the caller carries no reason, so whatever went wrong, the caller hears the same thing: carry on.

I call this polite failure.

Each layer catches the error and hands something harmless to the layer above. That layer does the same. By the time the result reaches a human, it's a green tick on a dashboard. Nobody lied on purpose. Every layer was being defensive, and defensive programming is something we praise in code review.

But look at the price. return None costs one line. The truth costs a type. We keep picking the line.

I'm not against fallbacks. A pattern match that keeps the work moving while the model is down is a fine idea, and I keep mine. The trouble is that a fallback out of its depth still returns something, and something looks a lot like an answer.

A silent fallback is the restaurant that runs out of paneer and brings you tofu without a word. You eat it. You might even like it. You just never ordered it, and nobody in the kitchen thinks they did anything wrong.

The most expensive cousin of this I know of wasn't a fallback at all. It happened inside an order router.

On August 1, 2012, Knight Capital opened the day with old code still live on one of its eight order-routing servers. A technician hadn't copied the new release to the eighth machine, and nobody checked. When orders arrived carrying a flag that had been reused, that server woke up a routine Knight had stopped using in 2003 and started firing orders in a loop. According to the SEC's order, Knight racked up more than 4 million executions in 154 stocks in about 45 minutes and lost over $460 million.

Here's the part that matters for us. From about 8:01 that morning, an internal system sent automated emails that named the affected system and an error called "Power Peg disabled." It sent 97 of them before the 9:30 open. In the SEC's words, Knight "did not design these types of messages to be system alerts," and its staff "generally did not review them."

The system told them. It told them politely, in a channel nobody treated as a warning. The SEC went further: a procedure that fed those messages into Knight's monitoring "could have prevented the events of August 1."

A warning that nobody is expected to read isn't a warning. It's a diary.

Now the other edge, because I can hear someone typing "so just crash."

No. On November 18, 2025, Cloudflare's bot-scoring code got a config file twice its usual size, did the honest thing and crashed, and Cloudflare's network spent about three hours failing to deliver core traffic.

Loud versus quiet is the wrong question. What you want is a degraded mode with a name. Serve the tofu if you must, but say it's tofu, and tell someone who can go buy paneer.

So every fallback gets a name, a type and an alarm.

Start with the name. The function stops returning None and returns a result that carries a reason:

@dataclass
class ModelResult:
    text: str | None
    degraded: str | None  # "no_key", "network", "http_403", "bad_body", ...

Now "no key" and "bad body" are different values. Your logs can count them separately, and a person reading them knows whose problem it is.

Then make the fallback something the caller has to ask for. Declare it as def ask_model(prompt, *, fallback):, with no default, so every call has to read ask_model(prompt, fallback=keyword_match). No caller reaches the backup path without saying so. The decision moves out of the bottom of a library and onto the line a reviewer actually reads. Some languages let you go further. In Rust, a match that forgets the degraded case doesn't compile.

Then stop trusting the status code. A 200 tells you the request arrived, not that you got an answer. OpenAI's chat API, for one, returns a perfectly successful response with finish_reason set to "length" when the answer ran into its token limit, the cap on how much text it may write. HTTP says success. The text stops mid-sentence. A body with an error field, an empty list of choices, an answer cut short: all degraded, and the result should say so.

Then the alarm, and this is where most teams overcorrect. Alert once per reason per hour, not once per call. An alert on every call is as useful as an alert on none, because by the three hundredth one somebody mutes the channel. Knight's 97 emails had a different problem: nobody had made them an alert. Page on every call and you get both problems: first a channel nobody owns, then a channel everybody mutes.

Not every reason should wake someone up. A 400 usually means your own code sent garbage. That's a bug for the logs and the backlog, not a 3 a.m. phone call. But "no API key" in production pages someone, every time.

The quietest failure of all is money.

A lot of AI APIs run on prepaid credit, and a balance makes no noise on its way to zero. When it gets there, every call fails. On OpenAI's API, an empty balance comes back as a 429, the same status code as "slow down, you're sending too fast."

Now go look at your retry wrapper. What does it do with a 429? It backs off and tries again. And again. No amount of backoff buys credit, and OpenAI's own docs say so: retrying billing errors "won't restore API access." The error body carries a code that tells the two 429s apart. Your wrapper probably never reads it. Retries tell their own lies; I wrote those up in A retry is a write.

Treat the balance as a production dependency, like the database. Read it on a schedule and warn well before zero. A page that arrives after the money is gone is late.

Polite failure doesn't even need a model. Logs have their own version, and it's called truncation.

Say your system uses UUIDv7, the newer kind of ID that sorts by time. RFC 9562 puts a 48-bit millisecond timestamp at the front. Now say someone keeps the log lines tidy by printing only the first eight characters. Eight hex characters are 32 bits, which is just the top of that timestamp. Every ID minted in the same 65-second window gets the same eight characters:

019b2670-1bc0-75ce-b6b7-cd26f50f5808  ->  019b2670
019b2670-2c32-786e-9c2b-3360402e94c6  ->  019b2670
019b2670-61a5-7ef4-8c77-fa4277514adc  ->  019b2670

Nothing throws. Eight characters look a lot like an ID, the way tofu looks a lot like paneer, and the logs tell you, calmly, that three different orders are one order. You lose an afternoon chasing a customer who apparently did three things at once.

Git got this right years ago. Its short hashes are a prefix that names the object uniquely, and it uses a longer prefix when a short one would collide. Shortening is fine. Shortening that pretends to be a name is the bug.

Truncation is not identity.

The last rule is the one I'd paint on the wall above every job scheduler.

A run that does no work must not report success.

Picture a nightly job that copies records from one system to another. One night the source hands back an empty list because a token expired. The job loops over zero records, writes zero rows and exits cleanly. The scheduler shows green. It will show green tomorrow, and the day after, until somebody downstream asks why the numbers stopped moving.

Zero is a result, and sometimes it's the right one. But "there was nothing to do" and "I couldn't see anything to do" are different sentences, and your job should know which one it's saying.

None of this is expensive. The expensive part is admitting that the green tick was never evidence of anything.

Open your model client and find the return None. Count the things it can mean.