Designing an idempotent payout runner

A payout run that can't be safely re-run is not a payout run. It's a one-shot script you'll be afraid of every Friday.

The first thing engineers ask when they integrate mass payouts is "where's the batch endpoint?" There isn't one. There is no POST /payouts/batches. A mass payout is a fan-out: your code loops a recipient list and calls POST /api/v1/payouts once per line. The platform doesn't hold a batch object you can query later — the batch lives in your runner and your ledger.

That sounds like we offloaded work onto you. We did, and on purpose. A monolithic batch call has exactly one interesting failure mode and it's the worst one: the run dies at recipient 1,840 of 4,000 and you have no idea whether the server half-applied it. A fan-out of independent, idempotent calls makes that question disappear. You re-run the whole file and the platform deduplicates. This post is about the runner that earns that property.

The unit of safety is the key, not the call

Every payout body is four fields: currency (asset plus network, e.g. USDT_TRC20), amount (a decimal string), destination (the recipient address), and idempotency_key. That last one is a snake_case JSON field in the request body — not an Idempotency-Key header. The only headers you send are X-API-Key and Content-Type. People miss this constantly, put the key in a header, get no deduplication, and pay everyone twice on the retry. The full payout request schema lives at docs.thehalfin.com; read the idempotency key definition once and put it in the body where it belongs.

{
  "currency": "USDT_TRC20",
  "amount": "125.50",
  "destination": "T...recipient...",
  "idempotency_key": "payout:run_2026-05-18:affiliate_8842"
}

Submit that same key twice and the second call returns the existing payout instead of creating a new one. That's the entire safety model. Everything else is plumbing around it.

Derive the key from your ledger, never from the loop

The most common mistake is generating the key at send time — uuid() per iteration, or worse, the array index. Both break the moment you re-run the file: a fresh UUID is a new payout, and the array index shifts if a single row is added or removed upstream. The whole point evaporates.

The key has to be a deterministic function of what is being paid and which run this is, both read from data you already own:

idempotency_key = `payout:${runId}:${ledgerEntryId}`

• runId is the identity of this disbursement — a settlement period, a withdrawal-batch row in your DB, a payroll cycle. Not Date.now(). Something you can look up and re-derive tomorrow.
• ledgerEntryId is the row in your own commissions / withdrawals / payroll table that authorizes this specific payment.

Now re-running the file is a no-op for every line that already succeeded, because each line re-derives the exact same key. The runner never has to remember what it did — the key is the memory, and your ledger is where it lives. If you find yourself writing a "which rows did I already send" sidecar file, stop: that's the bug the key was supposed to delete.

If two engineers re-derive the same key from the same ledger row, they've issued the same payout — once. That's the property you're buying.

Partial failure is the normal case, not the exception

A run of 4,000 lines will not fail as a unit. It fails per line: one address fails network validation, one amount is malformed, one line's currency names a network that address can't receive on. The good lines still go through. Your runner has to treat each POST as an independent outcome and keep going.

The shape we ship internally:

1. Accepted. The payout was created (or returned from a prior identical key). Record the payout id against the ledger entry. Move on.
2. Rejected at validation (4xx — bad address, bad amount, unknown currency). This line will never succeed as written. Quarantine it, flag the ledger row for a human, do not retry in a loop. Retrying a malformed line just burns requests.
3. Transient (timeout, 5xx, connection reset). You genuinely don't know if it landed. Retry with the same key. This is exactly the situation idempotency exists for — the retry either creates the payout or returns the one the first call already made. Back off with jitter so a partner hiccup doesn't turn into a self-inflicted stampede; the same discipline we wrote about in webhooks that survive everything applies to the send side.

The distinction between (2) and (3) is the whole game. Retrying a permanent rejection is wasted work; not retrying a transient failure is a missing payment. When in doubt, the key makes the retry free of consequence — so a transient-looking failure is always safe to re-send.

for each line in run:
  key = `payout:${runId}:${line.ledgerEntryId}`
  body = { currency, amount, destination, idempotency_key: key }
  result = postWithRetry(POST /api/v1/payouts, body)   // same key on every retry
  switch result:
    accepted -> record payout id on ledger entry
    rejected -> quarantine line, flag for human
    exhausted -> leave ledger entry 'submitting', let the next run re-derive the key

A line you couldn't resolve this run is not lost. The next run re-derives the same key and tries again, converging on exactly one payout per recipient. That convergence is the reason you can re-run the entire file with zero anxiety.

Submission is not settlement — approval sits in between

A payout that returns accepted has not moved funds. New payouts enter a pending-approval state and are released from the dashboard before anything leaves the balance. That's a deliberate control: your runner can fan out 4,000 lines unattended at 02:00, and a human still gates the actual movement in the morning. Don't design the runner as if accepted means done — it means queued for a person to release. The full operator walk-through lives in the how-to-send-mass-payouts guide.

This also means your runner should be comfortable being run twice on purpose: once to fan out and stage everything, and again after a failure to fill the gaps — both before anyone clicks release. Idempotency is what makes that double-run boring instead of dangerous.

Reconcile from webhooks, not from your loop's return value

Here is the discipline that separates a real integration from a demo: the POST response is a receipt, not the truth. The truth arrives later, asynchronously, as a signed event. Two events close the loop:

• payout.completed — funds moved on-chain. Mark the ledger entry paid. This is the only signal that means paid, and it can arrive minutes after submission, after approval and after on-chain confirmation.
• payout.failed — the release didn't go through (e.g. it couldn't be broadcast). The ledger entry goes back to a payable state so the next run re-derives the key and re-submits it.

Verify the HMAC over the raw request bytes before you trust either one — parse-then-verify is how forged events get processed. We covered exactly that failure in verify the webhook signature before acting. And process by the event's own id, not by payout_id, so a duplicate delivery is a no-op on your side too.

Note what is not on this list: invoice.activated and a hypothetical payout.submitted. Neither exists. Don't write a state machine that waits for an event we never send — drive submission state from your own POST results and drive settlement state from payout.completed / payout.failed. The mass-payout sibling post, kill the Friday CSV, walks the same loop from the operator's side if you want the non-engineering framing.

The runner, in one breath

A correct payout runner has five moving parts and no more:

1. A recipient set built from your ledger, one line each, four fields per line.
2. A key derived deterministically from runId + ledger entry — re-derivable, never random, never the index.
3. A per-line submit loop that treats every POST as independent and retries transient failures with the same key.
4. A quarantine lane for permanent rejections that a human resolves.
5. A reconciler that flips ledger state on signed payout.completed / payout.failed, verified on raw bytes.

Build those and the scariest sentence in payments — "I don't know if the run finished" — stops being a sentence you can say. You re-run the file. It pays each recipient once. You go home.

R. Adeyemi, halfin payments engineering