routstrd/FINDINGS-SHARE.md

routstrd 0.4.12 field test: findings

Tested on a real wallet against live nodes: 35 paid requests, about 140 sats, 3 nodes paid. Each finding has: what it means in plain words, whether a normal user can hit it (practical) or only in theory (theoretical), the exact command, what I got, and what it costs to repeat.

What works: Anthropic cache write and read are billed exactly at each node's listed rates (otrta and cypherpunk, /v1/messages, streaming, chat-completions format, Claude Code). Codex models on nostrmodels.top work for chat, streaming, the Responses API and tool calls, with billing exact to the msat. The upgrade kept the balance, nostrmodels turned on by itself, logs now redact keys, and refunds and parallel requests are fine.

Quick verdict table

#	Finding	Verdict	Whose bug	Cost to repeat
1	routstrd update times out on slow internet	Practical	routstrd	0
2	A bad request (400) refunds keys, hops 3 nodes, returns wrong error	Intended behaviour, not a bug	—	~3 sats
3	Provider pin is not strict, prompt goes to other nodes on error	Practical (maybe by design)	SDK	~10 sats
4	Sats on a mint the cheapest node rejects: 402, no failover	Practical	SDK	0
5	Responses API deposits are far too big	Practical (money parked, not lost)	SDK	~0.1 sat + refund rounding
6	nostrmodels ignores max_tokens	Practical	node / upstream	~0.2 sat
7	Default max_tokens 64000 blocks Claude for wallets under ~1.2k sats	Practical	routstrd default	0
8	Old installs stay on the fee mint after upgrade	Practical	routstrd	0
9	otrta Claude cache misses about half the time	Practical	upstream (ppq.ai)	~21 sats per miss
10a	"Model not offered" and "provider disabled" return 500	Practical, cosmetic	routstrd	0
10b	history shows a rolled-back receive as a normal receive	Practical, misleading	routstrd	0
10c	402 error JSON contradicts its own text	Practical, cosmetic	SDK	0
10d	models -m shows context 4,096 from a disabled node	Practical, cosmetic	routstrd	0
10e	Streaming Anthropic usage saved with 0 prompt tokens	Practical, cosmetic	upstream stream	~2 sats
10f	Responses stream final chunk has no event: line	Theoretical (no client broke)	node	~1 sat
10g	history -i --json prints full token strings	Low (tokens shown were spent)	routstrd	0
10h	otrta's claude-sonnet-5-5 costs 35% more than claude-sonnet-5.5	Practical, pricing	node	0

How to reproduce

Everything below runs against a local routstrd 0.4.12 on 127.0.0.1:8008, in apikeys mode, with a wallet holding sats on minibits and cubabitcoin.

• To pin a node, add -H "x-routstr-provider: <node>".
• The daemon does not check the Authorization header, so the curl commands leave it out.
• Check money before and after with routstrd balance. Pull parked sats back with routstrd refund -y.

---

1. routstrd update times out on slow internet

Plain words: the updater gives the whole 38 MB download only 30 seconds. On anything slower than about 1.3 MB/s it fails.

Practical? Yes. Any user on a phone hotspot or a weak connection. It failed for us.

What I ran (on 0.4.11):

routstrd update

What I got:

Updating routstrd from v0.4.11 to v0.4.12...
routstrd command failed: Could not update routstrd in place. The operation timed out.

Then I measured the same file with curl: it took 70 s at 540 KB/s. Checksum OK, manual install worked.

Code: src/utils/standalone-update.ts at tag v0.4.12. Line 16 FETCH_TIMEOUT_MS = 30_000. Line 66 puts AbortSignal.timeout(FETCH_TIMEOUT_MS) on the fetch. Line 168 reads the whole archive with arrayBuffer() under that same signal, so the 30 s covers the full download, not just the connection. "The operation timed out." is the abort message from that signal. The other 30 s timer (PROCESS_TIMEOUT_MS) would say "... timed out." with a label, so it was not that one.

How you can check (free, no install): time the download.

curl -sL -o /dev/null -w 'took %{time_total}s at %{speed_download} bytes/s\n' https://github.com/Routstr/routstrd/releases/download/v0.4.12/routstrd-v0.4.12-linux-x64.tar.gz

If it says more than 30 s, routstrd update would have failed on that connection.

---

2. A bad request (400) refunds keys, hops 3 nodes, returns the wrong error

STATUS: intended behaviour — do not fix. Failover on an upstream error is deliberate: the same request may well succeed on another node (different upstream, different validation, different model build). The refund-then-hop is the cost of that reliability. The two sub-issues that are still real and worth a separate ticket: (a) all_providers_failed should carry the first provider's status/body rather than whichever node happened to error last, so clients see a 400 instead of a fallback 500, and (b) the hop count is unbounded by the pin/no-pin distinction (see #3).

Plain words: you send something the API rejects, like temperature: "hot". That should just come back as a 400 and cost nothing. Instead the SDK pulls all your sats back from the node, tries 2 more nodes with fresh deposits (same 400 each time), pays mint fees on every hop, and in my run the final answer was an unrelated 500.

Practical? Yes. Agents and tools often send a parameter one provider does not like. Each time this costs fees and about 20 s.

What I ran:

curl -s -w '\nHTTP %{http_code}\n' -H 'Content-Type: application/json' -H 'x-routstr-provider: https://api.nostrmodels.top/' -X POST http://127.0.0.1:8008/v1/chat/completions -d '{"model":"gpt-6-luna","max_tokens":10,"temperature":"hot","messages":[{"role":"user","content":"hi"}]}'

What I got:

{"error":"The socket connection was closed unexpectedly. For more information, pass `verbose: true` in the second argument to fetch()"}
HTTP 500

What happened inside:

1. nostrmodels: 400 "expected number" → "Status 400 (auth/server error), attempting refund" → refunded the whole key (28,437 msat).
2. "Failing over to next provider: https://routstr.otrta.me/" → 400 again → refunded that whole key too (9,092 msat).
3. "Failing over to next provider: https://llm.satsandsports.cash/" → the minibits mint dropped the socket while making the deposit → that error became the final answer.

Total balance went from 525.194 to 522.665 sats (about 2.5 sats gone) for a request that should cost 0. The failed minibits send was rolled back safely, so no proofs were stuck.

A second test with "messages":"not-a-list" did the same 3-node walk, but that time the final answer was the correct 400 (all_providers_failed). So the wrong-error part depends on luck. The refund-and-hop part happens every time.

Cost to repeat: about 2 to 3 sats in fees. You may get the correct 400 instead of the 500 if no mint hiccups.

Verdict: not a defect. Marked as intended on review. Errored-request failover is a product decision, and 2-3 sats is an acceptable price for it. Only the error reporting half is a bug — track it as "propagate upstream status, not the last node's failure" rather than as an SDK failover change.

---

3. Provider pin is not strict

Plain words: if you pin a node with x-routstr-provider (or the config provider), you would expect your prompt to go only there. On an upstream error the SDK still fails over to other nodes and sends them the same prompt.

Practical? Yes, I saw it with real Claude Code. It may be intended for reliability, but anyone pinning a node for privacy or trust would not expect it. This is a question for the team.

What I ran: Claude Code through routstrd with ANTHROPIC_CUSTOM_HEADERS="x-routstr-provider: https://routstr.cypherpunk.today/" and model claude-sonnet-5-5.

What happened:

• cypherpunk (Venice) answered 400: system: text content blocks must contain non-whitespace text.
• "Failing over to next provider: https://routstr.otrta.me/". Claude Code got its answer from otrta.
• routstrd usage shows that request under https://routstr.otrta.me/.

Test #2 shows the same thing with a plain curl and a pin to nostrmodels.

Simple way to check: repeat test #2 and look at the log for "Failing over to next provider" even though you pinned nostrmodels.

---

4. Sats on a mint the cheapest node rejects: 402, no failover

Plain words: the wallet has sats on two mints: most on minibits, a little on cubabitcoin. The cheapest node for gpt-6-astra (cypherpunk) only takes cubabitcoin. When the request needs more than your cubabitcoin balance, you get "insufficient balance" even though you have plenty in minibits, and nostrmodels (which takes minibits and sells the same model) is never tried.

Practical? Yes. Anyone whose sats are mostly on one mint will hit it on any model where the cheapest node does not take that mint.

Needs an unpinned daemon (no provider in config, no header). Then:

curl -s -w '\nHTTP %{http_code}\n' -H 'Content-Type: application/json' -X POST http://127.0.0.1:8008/v1/chat/completions -d '{"model":"gpt-6-astra","max_tokens":3000,"reasoning_effort":"low","messages":[{"role":"user","content":"Reply with just the word: ok"}]}'

What I got:

{"error":"Insufficient balance: need 100 sats, have 497 sats available. Largest mint balance: 434 sats from https://mint.minibits.cash/Bitcoin", ...}
HTTP 402

Inside: picks cypherpunk → sends a 105-sat minibits token → cypherpunk rejects the mint → token is reclaimed (safe) → cubabitcoin is short → 402. No "Failing over" line.

Proof the other node works: same request pinned to nostrmodels with max_tokens: 500 returned "ok" for 0.28 sats.

Cost to repeat: 0. The minibits token goes out and comes back, and minibits has no fee.

Smaller part of the same issue: every top-up to cypherpunk tries minibits first and bounces before using cubabitcoin. Look for topUp: mint https://mint.minibits.cash/Bitcoin failed ... retrying with another supported mint in the log. It adds 2 to 4 s each time, no money lost.

---

5. Responses API deposits are far too big

Plain words: before a request, the SDK makes sure the node holds enough of your sats. For /v1/responses (what Codex uses) it ignores max_output_tokens and assumes the longest possible answer. So it moves far more sats to the node than the request needs. They are not lost (refund brings them back), but they sit on a third-party node.

Practical? Yes. Codex CLI uses Responses. With a bigger wallet, each new node will hold around 1,000 sats.

What I ran:

curl -s -H 'Content-Type: application/json' -H 'x-routstr-provider: https://api.nostrmodels.top/' -X POST http://127.0.0.1:8008/v1/responses -d '{"model":"gpt-6-luna","max_output_tokens":30,"input":"Write a 200-word paragraph about mints."}' | jq -c '.usage.cost.total_msats'

routstrd balance

What I got: the request cost 82 msat (0.08 sat). The nostrmodels key went from 6.358 to 55.276 sats, so 49 sats were moved for it.

Log: required=38.9949 x 1.4 → createProviderToken ... adjustedAmount=49. The same request as chat completions estimated 0.014 sats.

Bigger example: gpt-6.1-sol on Responses, max_output_tokens: 200: amount=778.04895 and a top-up of 1084 sats. The wallet was too small so the top-up failed, and then the node served the request from a 5.3-sat key for 0.62 sats. So the node never needed that money.

Why: in the SDK, nodeGateSats (client/ProviderManager.ts) says "The node ignores Responses max_output_tokens" and reserves the full completion budget. The sol request above shows the node actually served it from 5.3 sats, so that assumption looks wrong for these nodes.

Cost to repeat: about 0.1 sat, plus ~1 sat of rounding when you refund.

---

6. nostrmodels ignores max_tokens

Plain words: you ask for at most 30 tokens and get 277. Billing is for what you got, so the limit does not limit cost.

Practical? Yes, on nostrmodels. Cheap on luna, but on gpt-6-astra (0.029 sats per output token) a cap of 100 tokens could bill thousands. This is the node or its Codex upstream, not routstrd.

What I ran:

curl -s -H 'Content-Type: application/json' -H 'x-routstr-provider: https://api.nostrmodels.top/' -X POST http://127.0.0.1:8008/v1/chat/completions -d '{"model":"gpt-6-luna","max_tokens":30,"messages":[{"role":"user","content":"Write a 200-word paragraph about mints."}]}' | jq -c '[.choices[0].finish_reason, .usage.completion_tokens]'

What I got: ["stop",277]. Responses with max_output_tokens: 30 gave 274. A stream with max_tokens: 1500 billed 1,787 tokens.

Cost to repeat: about 0.1 sat each.

---

7. Default max_tokens 64000 blocks Claude for small wallets

Plain words: if a client does not send max_tokens, routstrd fills in 64,000. The deposit for Claude is then sized for 64,000 output tokens, which needs about 1,200 sats in your wallet even for a "say hi" request.

Practical? Yes, for chat-completions clients that leave out max_tokens and wallets under about 1.2k sats. /v1/messages clients always send max_tokens, so they are not affected.

What I ran:

curl -s -w '\nHTTP %{http_code}\n' -H 'Content-Type: application/json' -H 'x-routstr-provider: https://routstr.otrta.me/' -X POST http://127.0.0.1:8008/v1/chat/completions -d '{"model":"claude-sonnet-5.5","messages":[{"role":"user","content":"Say hi in 3 words."}]}'

What I got:

{"error":"Insufficient balance: need 1207 sats, have 485.092 sats available. Largest mint balance: 476 sats from https://mint.minibits.cash/Bitcoin","error_type":"insufficient_balance","required":1207,"available":485,"maxMintBalance":0,"maxMintUrl":""}
HTTP 402

The same request with max_tokens: 20 worked and cost 0.2 sats.

Cost to repeat: 0.

---

8. Old installs stay on the fee mint after upgrade

Plain words: 0.4.12 makes Minibits (no fee) the default mint, but only for new installs. An upgraded wallet still sends from cubabitcoin, which charges 100 ppk (0.1 sat per proof, rounded up). Small deposits and refunds lose 1 to 2 sats each.

Practical? Yes. About 22 of my 141 sats went to fees, rounding and the #2 loop.

Check (free):

routstrd wallet status | jq '{activeMint, defaultMint}'

curl -s https://mint.cubabitcoin.org/v1/keysets | jq -c '[.keysets[]|select(.active)|{unit,input_fee_ppk}]'

curl -s https://mint.minibits.cash/Bitcoin/v1/keysets | jq -c '[.keysets[]|select(.active)|{unit,input_fee_ppk}]'

I got activeMint and defaultMint = cubabitcoin, cubabitcoin fee 100, minibits fee 0.

Example from history: a 7-sat send left the wallet 8 sats lighter, and the node credited 6 sats.

---

9. otrta Claude cache misses about half the time

Plain words: otrta sends Claude through ppq.ai. Same prompt, same cache marker, and about half the time it writes the cache again (~21 sats) instead of reading it (~2 sats). cypherpunk hit every time.

Practical? Yes, and it is where most of my sats went. But it is upstream, not routstrd. Billing was correct for what upstream reported.

My results:

Request	Result	Charged
cache-write (otrta)	write 6,740	20.99 sats
cache-read, 20 s later (otrta)	write again	21.00 sats
otrta-cache-repeat, same body	read	1.94 sats
otrta-cache-q2, other question	read	1.94 sats
otrta-cache-stream, stream: true	write again	21.03 sats
cp-cache1 / cp-cache2 (cypherpunk)	write, then read	13.5 then 3.4 sats

Cost to repeat: 2 to 21 sats per request. I would skip it unless you want to see it yourself.

---

10. Smaller things

10a. Wrong status codes (cosmetic, free):

curl -s -w ' HTTP %{http_code}\n' -H 'Content-Type: application/json' -H 'x-routstr-provider: https://api.nostrmodels.top/' -X POST http://127.0.0.1:8008/v1/chat/completions -d '{"model":"no-such-model-xyz","max_tokens":5,"messages":[{"role":"user","content":"hi"}]}'

Got {"error":"Provider https://api.nostrmodels.top/ does not offer model: no-such-model-xyz"} HTTP 500. Pinning a disabled node (for example https://api.routstr.com/) also gives 500. These should be 4xx.

10b. History shows a receive that never happened (free):

routstrd history -n 20

Row 91 shows RECEIVE | cubabitcoin | 14 sat. But:

routstrd history -i 91 --json | jq '{type, amount, state}'

shows "state": "rolledBack". The table view does not show state, so it looks like money arrived.

10c. 402 JSON contradicts its text: see #7. The text says "476 sats from minibits", but minibits had 448 (476 was both mints together), and the JSON fields say maxMintBalance: 0, maxMintUrl: "".

10d. Wrong context size in models -m (free):

routstrd models -m gpt-6.1-sol | head -4

Shows Context: 4,096 tokens. /v1/models says 922,000. The header seems to come from the first listed node (blazelight, disabled).

10e. Streaming Claude usage saved with 0 prompt tokens: after otrta-cache-stream, routstrd usage showed promptTokens: 0 (cache tokens and cost were right). The upstream stream sends input_tokens: 0. Cosmetic.

10f. Responses stream missing event: line (theoretical): in a streamed gpt-6.1-sol Responses call, every chunk has an event: line except the final response.completed one (the node rewrites it to add cost). Clients that read the JSON type field are fine. Only a strict client keyed on event: would break. I did not see one break.

10g. history -i <id> --json prints full token strings. The ones I looked at were already spent. For a still-pending send it would show a live token, which is the point of that command. Low risk.

10h. otrta pricing (node side, free):

curl -s https://routstr.otrta.me/v1/models | jq -r '.data[]|select(.id|test("sonnet-5"))|"\(.id) in=\(.pricing.prompt) cache_write=\(.pricing.input_cache_write)"'

claude-sonnet-5-5 is $3/M input, claude-sonnet-5.5 is $2.2155/M. Claude Code used the dash id, so it paid about 35% more on otrta. otrta also adds its markup to input but not to cache rates (cache read is exactly 0.1x of the base price before markup).

---

Things I did not test

• Failover from a truly dead node. I could not trigger it without sending real tokens to a fake node.
• xcashu mode, NWC funding, Lightning send or receive, the monitor TUI, real pi or opencode runs.