Compare commits

...

93 Commits

Author SHA1 Message Date
9qeklajc
407f4745a0 update lock file 2026-07-07 16:29:00 +02:00
9qeklajc
806771df13 Merge pull request #591 from Routstr/fix/refund-race-condition
fix(wallet): return 425 when refund is pending, not 404
2026-07-07 16:27:00 +02:00
9qeklajc
3c1b6d5f17 Merge pull request #578 from Routstr/token-error-code
fix: specific, sanitized errors for failed Cashu token redemption
2026-07-07 15:59:49 +02:00
redshift
a9d5a52b32 fix(wallet): return 425 when refund is pending, not 404
The X-Cashu refund endpoint raised 404 "Refund not found" when the
"in" transaction existed with a request_id but the "out" (refund)
transaction had not been written yet. This is a timing race: the
endpoint is polled while the upstream request is still in flight,
before send_refund() has minted and stored the refund token.

The 404 was indistinguishable from a genuinely-missing refund, so
clients had no signal that retrying would succeed — leading to
stranded refunds when clients gave up.

Replace the third 404 branch with 425 Too Early + Retry-After: 2.
The two earlier 404 branches (no "in" row, no request_id) remain 404
since those genuinely mean no refund will ever exist.

Also adds debug logging on all three not-found/pending branches so
the race is no longer invisible to operators (middleware does not log
request headers).

Adds unit tests for the new 425 pending path and the no-request_id
404 path.

Refs: refund-race-condition
2026-07-07 21:49:32 +08:00
9qeklajc
b36bb56a30 Merge pull request #571 from Routstr/fix/security-pip-deps
fix(deps): patch reachable pip security advisories (aiohttp, pyjwt, python-multipart, pydantic-settings)
2026-07-07 14:40:09 +02:00
9qeklajc
97172641f8 Merge pull request #581 from jeroenubbink/fix/generic-provider-pricing
Resolve generic provider pricing instead of fabricating it
2026-07-07 14:38:32 +02:00
9qeklajc
46bbba7027 resolve reivews 2026-07-07 14:29:39 +02:00
9qeklajc
65ccf83dbd update lock file 2026-07-07 12:30:02 +02:00
Jeroen Ubbink
2baea4149e fix(upstream): break OpenRouter bare-tail ties on combined token cost
The bare-tail tie-break ranked candidates by prompt price alone, so two
entries sharing a tail where one is cheaper on prompt but far dearer on
completion could resolve to the entry that undercharges output-heavy
traffic — contradicting the "highest-priced wins for money safety" promise.

Rank by the combined prompt + completion per-token cost instead, so the
choice stays deterministic and money-safe whichever way traffic leans. As
before there are zero bare-tail collisions in the live feed, so this changes
no resolved price today; it only governs the latent case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:25:09 +02:00
Jeroen Ubbink
64d9460711 fix(upstream): validate native model_spec prices before trusting them
The generic provider treated any Venice-style model_spec.pricing as
authoritative after only a None check, so a native both-zero price served
the model free, a negative one credited the caller on every request, and a
non-numeric string threw while parsing — the outer catch then dropped the
provider's entire catalog.

Coerce both native prices through the resolver's _as_float and reject
absent / non-numeric / negative / both-zero values, falling through to the
shared litellm→OpenRouter→fail-closed chain instead. This extends the same
money-safety guard the litellm and OpenRouter rungs already apply to the
native source, and keeps one malformed entry from emptying the catalog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-07 12:24:01 +02:00
9qeklajc
f55c3e9c8c Merge pull request #587 from Routstr/bump-version
bump version
2026-07-06 23:02:54 +02:00
9qeklajc
e3ac06342f bump version 2026-07-06 23:00:41 +02:00
9qeklajc
e2aa02307b explicit code for error report & update doc 2026-07-05 23:05:53 +02:00
9qeklajc
66bc1260a4 refactor: align bearer redemption errors with shared X-Cashu taxonomy
Route the Authorization: Bearer cashu… redemption failures through the
same failure taxonomy the X-Cashu path already uses (token_already_spent /
invalid_token / mint_error / cashu_error, carried in the error "type"),
with matching statuses, so both redemption paths agree on the
classification for a given failure class instead of introducing a third
parallel code.

- "already spent" -> token_already_spent (400)
- fee/melt failures -> mint_error (422)
- invalid/undecodable token -> invalid_token (400, was 401)
- other expected wallet errors -> cashu_error (400)
- unexpected faults still -> internal_error (500)

Also align the msats<=0 defense-in-depth envelope and note it is now
practically unreachable (credit_balance rejects non-positive amounts).
2026-07-05 17:14:30 +02:00
9qeklajc
9dc4c979b8 fix: tighten redemption error mapping and stop X-Cashu raw-text leak
- Anchor the broad invalid/decode buckets to co-occur with "token" so
  internal faults (e.g. "Invalid isoformat string") fall through to 500
  instead of masquerading as a 401 token error.
- Map the "estimate fees"/"exceed token amount" wallet errors to the
  specific "too small to cover swap fees" message instead of the generic
  bucket.
- Stop the X-Cashu redemption path from interpolating raw mint/exception
  text into mint_error/cashu_error responses (both duplicated blocks).
- Add tests for the fee-gap mapping and the anchored gate.
2026-07-05 16:53:54 +02:00
Jeroen Ubbink
ec0fd1143b fix(upstream): preserve source modality instead of flattening it
The generic provider computed modality as "text->text" for any vision model
and "text" otherwise, discarding the source's own modality and mislabelling
image-capable models. Carry OpenRouter's modality string through verbatim, and
when a source doesn't supply one, derive it from the captured input/output
modalities in the same "in->out" shape (e.g. "text+image->text").

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 15:40:57 +02:00
Jeroen Ubbink
ae9748db02 fix(upstream): break OpenRouter bare-tail ties by highest price
When a bare model id matched several OpenRouter entries by tail, the first feed
entry won, making the resolved price depend on feed order and risking an
undercharge if a cheaper reseller happened to sort first. Pick the
highest-priced candidate instead: deterministic and money-safe, since
undercharging is the hazard. An exact id match still wins ahead of any tail
match. Measured against the live feed there are zero bare-tail collisions
today, so this only governs the latent case.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 15:40:43 +02:00
Jeroen Ubbink
ccee76b31e fix(upstream): source litellm context from max_input_tokens only
litellm's max_tokens is the completion cap (it equals max_output_tokens for
~94% of models), not the context window, so falling back to it overstated the
context as the output limit. Take context from max_input_tokens alone; a model
that reports none falls through to the id-based estimate downstream, which is
honest about being a guess rather than mislabelling the output cap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 15:39:41 +02:00
Jeroen Ubbink
9f04525c82 fix(upstream): reject unusable litellm prices instead of resolving them
A litellm cost entry that lists a model but prices both tokens at 0 (free
moderation/rerank tiers) was resolved as a real price, importing the model
enabled and served for free. Reject a both-zero (or negative) litellm hit so
the caller falls through to the next source or fails closed, mirroring the
_has_valid_pricing check the OpenRouter feed already applies.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 15:39:03 +02:00
9qeklajc
c81de0de0a Merge pull request #583 from Routstr/fix/cache-pricing-ui
make cache prices configurable
2026-07-04 21:54:30 +02:00
9qeklajc
12c4c1f030 make cache prices configurable 2026-07-04 21:49:05 +02:00
9qeklajc
81658d0ba6 Merge pull request #561 from Routstr/fix/refund-truly-empty-responses
fix(payment): refund truly-empty upstream responses billed a non-zero USD cost
2026-07-04 21:42:46 +02:00
9qeklajc
ee5ced10c7 refactor(payment): extract shared _empty_cost helper; drop .gitignore change
Address PR #561 review:
- Extract the all-zero refund cost object into _empty_cost(), reused by
  both the no-usage-data path and the truly-empty USD-cost path.
- Revert the .gitignore additions; those belong in a separate PR.
2026-07-04 20:13:01 +02:00
Jeroen Ubbink
2a27fb239e fix(upstream): resolve generic provider pricing instead of fabricating it
A generic (OpenAI-compatible) upstream whose /models response omits pricing
was silently defaulting to $0.001/M tokens and a 4096 context window. For a
provider like DeepSeek that reports no price, this undercharged real usage by
~280x — a direct money leak — while presenting a plausible-looking price.

Resolve each model through trust-ordered sources instead: the provider's
native schema (Venice's model_spec) first, then litellm's bundled cost map
(curated list prices), then the OpenRouter feed. Capture the richer metadata
those sources carry (cache rates, modalities, max output tokens, context)
rather than only price and context. When no source knows the model, import it
disabled with a warning rather than invent a number, so an operator can price
it before it serves traffic.

Context has no trustworthy source of last resort, but it is not a billing
input, so a model priced without a reported context window falls back to an
id-based estimate. The whole source-incomplete fallback (price chain +
context estimate) lives in one pricing_resolver module so it can later be
hoisted into the base provider unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 14:42:33 +02:00
Jeroen Ubbink
ffde661d93 refactor(payment): extract litellm_cost_entry lookup helper
Pull the litellm model_cost lookup (both id spellings + case-insensitive
fallback) out of backfill_cache_pricing into a reusable litellm_cost_entry
helper. No behaviour change; the upstream price resolver reuses the same
lookup semantics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-03 14:42:17 +02:00
9qeklajc
7f49ba1771 fix: cast HTTPException detail in tests for mypy 2026-07-02 00:57:31 +02:00
9qeklajc
949dc433f1 fix: return sanitized, specific errors when redeeming an incoming Cashu token fails 2026-07-02 00:47:56 +02:00
9qeklajc
b7fcf000af Merge pull request #577 from jeroenubbink/refactor/provider-identity
refactor(upstream): resolve a provider's own row by primary key
2026-07-01 22:11:29 +02:00
9qeklajc
a1af1383e1 Merge pull request #514 from Routstr/add-provider-indentifier
add provider slug
2026-07-01 17:46:13 +02:00
9qeklajc
5bedbd129f use file path 2026-07-01 17:08:28 +02:00
9qeklajc
f588147b41 use slug base 2026-07-01 17:01:32 +02:00
9qeklajc
17bc949597 add test 2026-07-01 16:53:52 +02:00
9qeklajc
7705656016 resolve review comment 2026-07-01 16:40:31 +02:00
Jeroen Ubbink
434283c58d refactor(upstream): resolve a provider's own row by primary key
A live upstream provider re-finds its own database row in two places —
PPQ.AI's insufficient-balance self-disable and the base
refresh_models_cache — with WHERE base_url == self.base_url AND
api_key == self.api_key. That uses a rotatable secret as a self-handle:
if the row's key rotates under a live object, it can no longer find
itself.

Carry the row's primary key on the instance as db_id, stamped centrally
by from_db_row via a _build_from_row construction hook that subclasses
override, and look the row up with session.get(UpstreamProviderRow,
db_id). This also closes a latent gap where providers built outside the
init path (auto-topup) never received db_id.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-01 12:53:31 +02:00
9qeklajc
9fcf870e3f add provider slug 2026-06-29 23:30:09 +02:00
9qeklajc
782b233d40 Merge pull request #573 from Routstr/fix-deepseek-calculation
update-pricing
2026-06-29 22:29:42 +02:00
9qeklajc
0c7373675f solidify logic 2026-06-27 22:53:35 +02:00
9qeklajc
a33ea5da07 update-pricing 2026-06-27 16:01:45 +02:00
9qeklajc
b509581950 fix(deps): bump aiohttp, pyjwt, python-multipart, pydantic-settings for security advisories
Resolves Dependabot alerts:
- aiohttp 3.12.15 -> 3.14.1 (#192-200: websocket/parser/SSRF/CRLF fixes)
- pyjwt 2.10.1 -> 2.13.0 (#183-187: HMAC/JWK forgery, SSRF, DoS)
- python-multipart 0.0.20 -> 0.0.32 (#188-191: quadratic-time DoS, smuggling)
- pydantic-settings 2.14.0 -> 2.14.2 (#210: symlink secrets_dir escape)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 14:49:47 +02:00
9qeklajc
38d7075652 Merge pull request #567 from Routstr/feat-prune-dead-api-keys
feat: prune dead zero-balance API keys with background janitor
2026-06-25 14:21:20 +02:00
9qeklajc
ed27ad0dea Merge pull request #569 from jeroenubbink/docs/admin-ui-serving
docs(ui): document dev/build/serve workflow; make missing-bundle warning actionable
2026-06-25 14:20:34 +02:00
Jeroen Ubbink
6b1b4285fa docs(ui): document dev/build/serve workflow; make missing-bundle warning actionable
The ui/README.md was still stock create-next-app boilerplate. Replace it with the
real story: the UI is a Next.js static export served by FastAPI from ui_out/, the
two-process dev loop (backend :8000 + `make ui-dev` :3000 with hot reload, auto-
targeting :8000), the build/integration commands, and the cross-origin CORS note.

Also make the "ui_out not found" startup warning actionable — it now points to
`make ui-build` / `make ui-dev` instead of silently saying it skipped serving.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-25 10:54:26 +02:00
9qeklajc
7b56dcb902 resolve review comments 2026-06-24 22:54:07 +02:00
9qeklajc
2131cfc7f9 Merge pull request #566 from Routstr/fix-reservation-overrun-charge
fix: always charge balance on reservation overrun even after stale sweep
2026-06-24 22:38:28 +02:00
9qeklajc
d6d9af91e4 refactor: drop dead rowcount conditional in overrun finalize
WHERE guard now matches on hashed_key, so the UPDATE always affects one
row and result.rowcount is always truthy. Remove the conditional and its
unreachable else branch.
2026-06-24 22:21:50 +02:00
9qeklajc
cbd97e8e3a Merge pull request #549 from jeroenubbink/fix/swap-fee-retry
fix: retry foreign mint swaps with observed fees instead of trusting fee_reserve
2026-06-23 22:42:26 +02:00
Jeroen Ubbink
e4b26dd27b fix: require shortfall text for generic 11000 melt errors
11000 is nutshell's generic, unregistered TransactionError covering many
unrelated failures, so trusting the code alone made any 11000 retryable.
Gate the generic code on the 'not enough inputs' detail text; keep trusting
the registered 11005 (TransactionUnbalanced) on the code alone. Behavior is
unchanged for every existing case; a bare non-shortfall 11000 now surfaces
immediately instead of burning the retry budget.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 09:38:04 +02:00
9qeklajc
1e50afbd82 feat: prune dead zero-balance API keys with background janitor 2026-06-21 15:04:16 +02:00
9qeklajc
4d6e2cdf59 fix: always charge balance on reservation overrun even after stale sweep 2026-06-21 15:03:50 +02:00
9qeklajc
d079f45ba3 Merge pull request #564 from Routstr/fix-ratelimit-upstream-error-handling
better rate limit forwarding
2026-06-21 14:13:57 +02:00
9qeklajc
d08f07d6fb Merge pull request #565 from Routstr/tool-support
make openrouter default to provider supporting tool use
2026-06-21 14:06:20 +02:00
9qeklajc
af0796689f make openrouter default to provider supporting tool use 2026-06-20 23:55:03 +02:00
9qeklajc
f408785409 better rate limit forwarding 2026-06-20 20:03:40 +02:00
9qeklajc
90f1ba89e5 Merge pull request #563 from Routstr/fix-streaming-issue
consolidate streaming
2026-06-20 19:46:22 +02:00
9qeklajc
e29c9241ce Merge pull request #560 from Routstr/fix-deepseek-cache-pricing
Fix deepseek cache pricing
2026-06-20 19:46:07 +02:00
9qeklajc
5a3774a414 consolidate streaming 2026-06-20 11:33:52 +02:00
Shroominic
e972b62758 fix(payment): refund truly-empty responses with a non-zero USD cost
When an upstream reports a non-zero USD cost but the response carries no
tokens at all (input, output, cache-read and cache-creation all zero), the
USD path billed the full USD-derived amount for an empty response. Return an
all-zero cost (full refund) for that case only.

The gate is tightened relative to the superseded PR #489: a cache-read- or
cache-creation-only turn legitimately reports zero prompt/completion tokens
with a real cost and must still be billed, so the refund only fires when every
token bucket is zero.

Adds unit tests covering both the refund and the still-billed cache-only path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-20 13:10:24 +08:00
9qeklajc
b57f2d408e display cr as i 2026-06-20 00:57:11 +02:00
9qeklajc
70d9e1a1ce do correct deepseek cache price calc. 2026-06-20 00:26:34 +02:00
9qeklajc
c0a34a391f Merge pull request #558 from Routstr/display-balance-correctly
make sure balance is set correctly
2026-06-19 23:41:22 +02:00
9qeklajc
a97ea2995a Merge pull request #550 from jeroenubbink/fix/cached-token-overcharge
fix: bill cached input tokens at their real rates across vendor dialects
2026-06-19 23:41:04 +02:00
9qeklajc
5b50a78d95 Merge pull request #559 from Routstr/fix-provider-models-view
collapse when focus change
2026-06-19 14:33:58 +02:00
9qeklajc
ccab5e4216 collapse when focus change 2026-06-19 11:53:30 +02:00
9qeklajc
ff9c645bb1 make sure balance is set correctly 2026-06-19 11:20:56 +02:00
9qeklajc
d4318dcc07 Merge pull request #556 from jeroenubbink/fix/trusted-mint-input-fee
fix: deduct mint NUT-02 input fee when crediting trusted-mint topups
2026-06-18 21:48:24 +02:00
Jeroen Ubbink
75ed865a5f fix: deduct input fee in swap_to_primary_mint same-mint shortcut
The same-mint shortcut in swap_to_primary_mint did a same-mint
split(include_fees=True) — which burns the mint's NUT-02 per-proof input
fee — but returned the full token amount, over-crediting the user (the
same bug already fixed for the trusted-mint receive path). It also
skipped DLEQ verification that the trusted path performs.

Extract the shared same-mint redeem into _redeem_same_mint (load mint,
verify DLEQ, split, credit amount - input_fees) and delegate from both
recieve_token and the shortcut, so the two paths can't drift again. This
shortcut is reachable when PRIMARY_MINT_URL is set outside CASHU_MINTS.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 15:48:49 +02:00
Jeroen Ubbink
7969a8da55 test: clarify mocked input-fee comment in trusted-mint test
The comment described "21 proofs @ 100 ppk" arithmetic, but the mocked
token has one proof and get_fees_for_proofs is hard-mocked to 3, so the
math wasn't exercised. Describe what the mock actually does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-18 12:26:44 +02:00
9qeklajc
830c299130 Merge pull request #557 from Routstr/rollback-token-generation-when-request-failed
rollback when token generation failed
2026-06-17 22:17:55 +02:00
9qeklajc
e9dced8148 rollback when token generation failed 2026-06-17 17:27:08 +02:00
Jeroen Ubbink
ecd46975b4 fix: deduct mint NUT-02 input fee when crediting trusted-mint topups
recieve_token swaps the incoming proofs at the same mint with
include_fees=True (paying the mint's NUT-02 per-proof input fee) but credited
the full face value. On every topup from a fee-charging trusted mint, routstr
over-credited the user by the fee and its own wallet drifted toward insolvency.

Subtract get_fees_for_proofs(proofs) from the credited amount, mirroring the
foreign-mint swap path which already accounts for it. Adds a fee-charging
trusted-mint unit test (credited == face - input_fee).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-17 17:10:37 +02:00
9qeklajc
8f5f3d9738 resolve review comments 2026-06-13 23:40:39 +02:00
9qeklajc
355e3f19ef explicit cache 2026-06-13 23:40:39 +02:00
9qeklajc
439ac48216 update to all providers 2026-06-13 23:40:39 +02:00
9qeklajc
bf05c96dfc Merge pull request #552 from Routstr/fix-reset-reserve-balance
Fix reset reserve balance
2026-06-12 21:52:39 +02:00
9qeklajc
ed6e0c0189 Merge main into fix-reset-reserve-balance
Resolve pay_for_request conflict: keep main's atomic child-key
balance_limit guard and post-rowcount-check ordering, and stamp
reserved_at on both billing and child reservations.
2026-06-12 21:08:53 +02:00
9qeklajc
412bfe479c Merge pull request #547 from jeroenubbink/fix/child-key-balance-limit-atomic
fix: enforce balance_limit atomically on child key requests
2026-06-12 20:23:25 +02:00
9qeklajc
861fda21b7 resolve review comments 2026-06-12 20:20:38 +02:00
9qeklajc
919dcf5535 make key reservation reset predictable 2026-06-12 19:45:39 +02:00
Jeroen Ubbink
cbc424e8e7 build: type-check the entire repo in make targets, matching CI
CI runs 'uv run mypy .' while the Makefile only checked routstr/, so test
files could pass locally and fail the pipeline. lint, type-check and
ci-lint now check everything; --ignore-missing-imports is dropped since
the CI invocation passes without it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 21:22:28 +02:00
Jeroen Ubbink
068fb3572f refactor: drop unused session parameter from calculate_cost
The session was needed when model pricing lived in the DB (73d3613) and has
been dead since pricing moved to the in-memory model map (0da08fb), yet every
caller was still obliged to supply one. get_x_cashu_cost even opened a DB
session per x-cashu request solely to feed it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 21:17:11 +02:00
Jeroen Ubbink
eaf74edbba fix: bill cached input tokens at their real rates across vendor dialects
Cached prompt tokens were billed at the full input rate whenever a vendor's
usage dialect or cache pricing was unknown, overcharging DeepSeek topups
~5-10x on agentic workloads (hits are 10x cheaper upstream) and silently
mispricing OpenAI cached reads and Anthropic cache writes the same way.

Two root causes, two fixes:

- Usage dialects: DeepSeek reports prompt_cache_hit_tokens /
  prompt_cache_miss_tokens, which billing never parsed. Usage normalization
  now lives in payment/usage.py as a union parser over the known,
  non-colliding dialects (OpenAI prompt_tokens_details, Anthropic additive
  cache fields, DeepSeek hit/miss), producing one canonical NormalizedUsage.
  Providers expose it as an overridable BaseUpstreamProvider.normalize_usage
  hook — the escape hatch for future vendors whose fields genuinely
  conflict — and every settlement call site passes the provider's result
  through, so calculate_cost holds no vendor knowledge of its own.

- Cache rates: the OpenRouter model feed omits input_cache_read/-write for
  most DeepSeek models (and e.g. openai/gpt-4o), so billing fell back to the
  full input rate. Missing rates are now backfilled from litellm's bundled
  cost map before the provider fee is applied; the input-rate fallback
  remains only as the documented last resort.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 21:17:11 +02:00
Jeroen Ubbink
eb6dc5189c test: pin the verbatim production error from issue #468 as retryable
The reported error carries cashu-py's "could not pay invoice" wrapper
around the mint detail; the classifier must still find the code and the
shortfall inside it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
Jeroen Ubbink
cbdbe4b899 test: cover all branches of the foreign-mint swap path
Every statement and branch of the topup path (recieve_token,
swap_to_primary_mint and its helpers, credit_balance) is now covered,
including the previously untested mint-failure/recovery block: recovery
must never credit proofs the wallet does not hold, and post-melt failures
must propagate unwrapped since the foreign proofs are already spent.

On testing internals: orchestration and side effects are tested only
through the public functions, with mocks at the external mint boundary
(the cashu Wallet object) — no internal code is ever mocked. The two pure
leaf helpers (_melt_insufficient_shortfall, _net_minted_amount) are tested
directly instead: their input spaces (seven error formats across mint
implementations, sat/msat conversions both ways) would cost ~30 lines of
quote/melt mock choreography per case through the swap loop, for the same
assertion a parametrize row makes. The same-mint branch of
_calculate_swap_amount is unreachable from public callers today; its test
documents the contract for the planned pre-flight fee estimation, which
will call it directly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
Jeroen Ubbink
7870db9af1 fix: retry foreign mint swaps with observed fees instead of trusting fee_reserve
A foreign mint's melt fee_reserve is a non-binding estimate (NUT-05): the
mint may demand more on the real quote or reject the melt outright (e.g.
mint.cubabitcoin.org charging input fees beyond its quote). Instead of
padding the estimate with a safety buffer that strands funds at the
foreign mint on every swap, retry the mint-quote/melt-quote/melt cycle
(max 3 attempts) with the amount recomputed from the fees the mint
actually demands.

Melt failures are classified by the NUT-00 error code (11005 registered
TransactionUnbalanced as sent by cdk, 11000 nutshell's generic
TransactionError): fee-related rejections retry, others (e.g. 20004
Lightning payment failed) surface immediately. Nutshell's Provided/needed
amounts refine the retry step when present; otherwise shrink by 1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 15:08:38 +02:00
Jeroen Ubbink
28c4008892 fix: enforce balance_limit atomically on child key requests
The Python pre-check on balance_limit provided no concurrency guarantee —
two concurrent requests could both pass the check on stale in-memory state
and both proceed to reserve, exceeding the limit.

Add the balance_limit guard to the child key UPDATE's WHERE clause so the
enforcement is atomic. The HTTP 402 is raised without an explicit rollback:
since session.commit() was never called, the uncommitted billing key update
is discarded when the session context manager closes on exception exit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-09 11:37:39 +02:00
9qeklajc
a16bc1220c Merge pull request #546 from Routstr/fix-parent-key-leak-to-child-key
do not leak parent key to child key
2026-06-07 15:27:32 +02:00
9qeklajc
5d3e687876 Merge pull request #545 from Routstr/secure-test-endpoint
secure endpoint
2026-06-07 15:25:53 +02:00
9qeklajc
68a7cd1dfb do not leak parent key to child key 2026-06-07 15:25:13 +02:00
9qeklajc
d96dc20e86 secure endpoint 2026-06-07 15:18:49 +02:00
9qeklajc
934afaa091 Merge pull request #544 from Routstr/pr-reactive-correction
feat: reactive request-correction retry for recoverable upstream 400s
2026-06-07 15:09:18 +02:00
9qeklajc
9ef01acac9 Merge branch 'main' into pr-reactive-correction 2026-06-07 13:10:29 +02:00
9qeklajc
cbd38c15fe Merge pull request #543 from Routstr/pr-sse-parser
fix: buffer-based SSE parser for all supported providers
2026-06-07 13:06:16 +02:00
9qeklajc
06c8a071a5 feat: reactive request-correction retry for recoverable upstream 400s 2026-06-07 13:00:19 +02:00
77 changed files with 9632 additions and 1050 deletions

View File

@@ -98,7 +98,7 @@ docker-down:
lint:
@echo "🔍 Running linting checks..."
$(RUFF) check .
$(MYPY) routstr/ --ignore-missing-imports
$(MYPY) .
format:
@echo "✨ Formatting code..."
@@ -107,7 +107,7 @@ format:
type-check:
@echo "🔎 Running type checks..."
$(MYPY) routstr/ --ignore-missing-imports
$(MYPY) .
# Development setup
dev-setup:
@@ -234,7 +234,7 @@ ci-test:
ci-lint:
@echo "🤖 Running CI linting..."
$(RUFF) check . --exit-non-zero-on-fix
$(MYPY) routstr/ --ignore-missing-imports --no-error-summary
$(MYPY) . --no-error-summary
# Debug helpers
test-debug:

View File

@@ -113,42 +113,109 @@ All errors follow a consistent JSON structure:
**Status:** 402
**Resolution:** Top up API key balance
#### Invalid Token
### Cashu Token Redemption Errors
These errors are returned when a Cashu token you pay with cannot be redeemed.
They apply to every endpoint that accepts a token:
- **Per-request payment** via the `X-Cashu` header (chat completions + Responses API).
- **API key top-up** via `POST /v1/wallet/topup`.
- **Minting an API key** from a token sent in `Authorization: Bearer <cashu-token>`.
All three share one classifier, so the same failure yields the same HTTP status
and sanitized message everywhere. Structured error envelopes (`X-Cashu` and
`Authorization: Bearer <cashu-token>`) also expose the same `type` and `code`
branch on `type` (or `code` for finer granularity). `POST /v1/wallet/topup`
keeps its existing plain-string `detail` envelope, so branch on status there.
| `type` | Status | `code` | Retryable | Meaning |
|--------|--------|--------|-----------|---------|
| `token_already_spent` | 400 | `cashu_token_already_spent` | No | The token was already redeemed. |
| `invalid_token` | 400 | `invalid_cashu_token` | No | The token is malformed or cannot be decoded. |
| `mint_error` | 422 | `cashu_token_swap_fees_exceed_amount` | No | Token value is too small to cover the mint's swap/melt fees. |
| `mint_error` | 422 | `cashu_foreign_mint_swap_failed` | No | Swapping the token from a foreign mint to the primary mint failed. |
| `mint_unreachable` | 503 | `cashu_mint_unreachable` | **Yes** | The mint could not be reached (DNS failure, refused/reset connection, timeout). The token is fine — retry once the mint recovers. |
| `cashu_error` | 400 | `cashu_token_redemption_failed` | No | The token could not be redeemed for another expected reason. |
| `cashu_error` | 400 | `cashu_token_zero_value` | No | The token redeemed to zero (empty/dust token, or value fully consumed by fees). |
| `token_consumed` | 500 | `cashu_token_consumed` | No | The token was **spent** (melted/redeemed) but crediting it then failed. Do not retry — the token is gone; contact support to reconcile. |
| `api_error` | 500 | `internal_error` | Maybe | Unexpected server-side fault during redemption. |
!!! important "Retry only `mint_unreachable`"
Only `mint_unreachable` (503) means the same token will work again later —
everything else is a permanent property of the token and must not be
blindly retried. Use exponential backoff for the 503. In particular, a
`token_consumed` 500 means the mint already spent the token, so a retry
would fail as `token_already_spent`.
#### Mint Unreachable (retryable)
```json
{
"error": {
"type": "payment_error",
"message": "Invalid Cashu token",
"code": "invalid_token",
"details": {
"reason": "Token already spent"
}
"type": "mint_unreachable",
"message": "Cashu mint is unreachable",
"code": "cashu_mint_unreachable"
}
}
```
**Status:** 400
**Resolution:** Use a valid, unspent token
**Status:** 503
#### Mint Unavailable
**Resolution:** The token is valid — the mint is temporarily down. Retry with
backoff, or pay with a token from a different mint.
#### Token Already Spent
```json
{
"error": {
"type": "payment_error",
"message": "Cannot connect to Cashu mint",
"code": "mint_unavailable",
"details": {
"mint_url": "https://mint.example.com",
"retry_after": 60
}
"type": "token_already_spent",
"message": "Cashu token already spent",
"code": "cashu_token_already_spent"
}
}
```
**Status:** 503
**Resolution:** Try again later or use different mint
**Status:** 400
**Resolution:** Use a fresh, unspent token. Do not retry with the same token.
#### Response envelope differs by endpoint
The `error` object above is identical everywhere, but the surrounding envelope
depends on how you paid:
- **`X-Cashu` header payments** (chat + Responses API) return the object at the
top level, alongside a `request_id`:
```json
{
"error": { "type": "mint_unreachable", "message": "Cashu mint is unreachable", "code": "cashu_mint_unreachable" },
"request_id": "req-abc123"
}
```
The original token is echoed back in the `X-Cashu` **response header only when
it is still spendable** (e.g. `mint_unreachable`, `invalid_cashu_token`, fee
errors) so you can recover/retry it. It is **not** echoed for spent/consumed
tokens (`cashu_token_already_spent`, `cashu_token_consumed`,
`cashu_token_zero_value`, `internal_error`) — retrying those can never succeed.
- **`Authorization: Bearer <cashu-token>`** (API key minting) wraps it in
FastAPI's `detail` field:
```json
{ "detail": { "error": { "type": "mint_unreachable", "message": "Cashu mint is unreachable", "code": "cashu_mint_unreachable" } } }
```
- **`POST /v1/wallet/topup`** returns a plain string message under `detail` —
it carries the shared HTTP **status** and **message** (e.g. `503` for an
unreachable mint) but not the structured `type`/`code`, so branch on the
status code here:
```json
{ "detail": "Cashu mint is unreachable" }
```
### Validation Errors
@@ -347,7 +414,7 @@ class ErrorHandler:
'rate_limit',
'upstream_timeout',
'model_overloaded',
'mint_unavailable'
'cashu_mint_unreachable'
}
# Errors requiring user action

View File

@@ -0,0 +1,25 @@
"""add reserved_at to api_keys
Revision ID: b5e7c9d1f3a2
Revises: a2b3c4d5e6f7
Create Date: 2026-06-12 00:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "b5e7c9d1f3a2"
down_revision = "a2b3c4d5e6f7"
branch_labels = None
depends_on = None
def upgrade() -> None:
# existing keys keep NULL
# New reservations populate it via pay_for_request.
op.add_column("api_keys", sa.Column("reserved_at", sa.Integer(), nullable=True))
def downgrade() -> None:
op.drop_column("api_keys", "reserved_at")

View File

@@ -0,0 +1,88 @@
"""add slug to upstream_providers
Revision ID: c6d7e8f9a0b1
Revises: b5e7c9d1f3a2
Create Date: 2026-06-29 00:00:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
from routstr.core.provider_slugs import provider_slug_base, provider_slug_candidate
revision = "c6d7e8f9a0b1"
down_revision = "b5e7c9d1f3a2"
branch_labels = None
depends_on = None
def _allocate_backfill_slug(provider_type: str, reserved_slugs: set[str]) -> str:
base = provider_slug_base(provider_type)
suffix_number = 1
while True:
candidate = provider_slug_candidate(base, suffix_number)
if candidate not in reserved_slugs:
reserved_slugs.add(candidate)
return candidate
suffix_number += 1
def _backfill_provider_slugs(conn: sa.Connection) -> None:
existing_rows = conn.execute(
sa.text(
"SELECT slug FROM upstream_providers "
"WHERE slug IS NOT NULL AND slug != ''"
)
)
reserved_slugs = {str(row.slug).lower() for row in existing_rows}
rows_to_backfill = conn.execute(
sa.text(
"SELECT id, provider_type FROM upstream_providers "
"WHERE slug IS NULL OR slug = '' "
"ORDER BY id"
)
)
for row in rows_to_backfill:
slug = _allocate_backfill_slug(str(row.provider_type), reserved_slugs)
conn.execute(
sa.text("UPDATE upstream_providers SET slug = :slug WHERE id = :id"),
{"slug": slug, "id": row.id},
)
def upgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
columns = {c["name"] for c in inspector.get_columns("upstream_providers")}
if "slug" not in columns:
op.add_column(
"upstream_providers",
sa.Column("slug", sa.String(), nullable=True),
)
_backfill_provider_slugs(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("upstream_providers")}
if "ix_upstream_providers_slug" not in existing_indexes:
op.create_index(
"ix_upstream_providers_slug",
"upstream_providers",
["slug"],
unique=True,
)
def downgrade() -> None:
conn = op.get_bind()
inspector = sa.inspect(conn)
existing_indexes = {idx["name"] for idx in inspector.get_indexes("upstream_providers")}
if "ix_upstream_providers_slug" in existing_indexes:
op.drop_index("ix_upstream_providers_slug", table_name="upstream_providers")
columns = {c["name"] for c in inspector.get_columns("upstream_providers")}
if "slug" in columns:
op.drop_column("upstream_providers", "slug")

View File

@@ -1,6 +1,6 @@
[project]
name = "routstr"
version = "0.4.3"
version = "0.4.4"
description = "Payment proxy for your LLM endpoint using cashu and nostr."
readme = "README.md"
requires-python = ">=3.11"

View File

@@ -20,7 +20,11 @@ from .payment.cost_calculation import (
MaxCostData,
calculate_cost,
)
from .wallet import credit_balance, deserialize_token_from_string
from .wallet import (
classify_redemption_error,
credit_balance,
deserialize_token_from_string,
)
logger = get_logger(__name__)
payments_logger = get_logger("routstr.payments")
@@ -78,6 +82,37 @@ async def check_and_reset_limit(key: ApiKey, session: AsyncSession) -> bool:
return False
def redemption_error_to_http_exception(error: Exception) -> HTTPException:
"""Map a Cashu token redemption failure to a sanitized client-facing error.
Thin wrapper over the shared :func:`classify_redemption_error` so the bearer
path stays identical to the X-Cashu and top-up paths.
"""
classified = classify_redemption_error(error)
if classified is None:
return HTTPException(
status_code=500,
detail={
"error": {
"message": "Internal error during token redemption",
"type": "api_error",
"code": "internal_error",
}
},
)
error_type, status_code, message, error_code = classified
return HTTPException(
status_code=status_code,
detail={
"error": {
"message": message,
"type": error_type,
"code": error_code,
}
},
)
async def validate_bearer_key(
bearer_key: str,
session: AsyncSession,
@@ -216,7 +251,17 @@ async def validate_bearer_key(
try:
hashed_key = hashlib.sha256(bearer_key.encode()).hexdigest()
token_obj = deserialize_token_from_string(bearer_key)
try:
token_obj = deserialize_token_from_string(bearer_key)
except Exception as decode_error:
# A malformed token is a bad token (400 invalid_cashu_token via
# the shared taxonomy), not an auth failure (401) — otherwise it
# would fall through to the generic "Invalid API key" handler.
raise redemption_error_to_http_exception(
ValueError(
f"Invalid Cashu token: could not decode token ({decode_error})"
)
) from decode_error
logger.debug(
"Generated token hash", extra={"hash_preview": hashed_key[:16] + "..."}
)
@@ -334,19 +379,32 @@ async def validate_bearer_key(
"error_type": type(credit_error).__name__,
},
)
raise credit_error
await session.rollback()
raise redemption_error_to_http_exception(credit_error) from credit_error
if msats <= 0:
logger.error(
"Token redemption returned zero or negative amount",
extra={"msats": msats, "key_hash": hashed_key[:8] + "..."},
)
# Defense-in-depth: credit_balance now refuses to commit on a
# zero/negative redemption, but if a row was nonetheless
# persisted, drop it so we never leave an orphan zero-balance key.
# Defense-in-depth: credit_balance already raises
# ValueError("Redeemed token amount must be positive…") before
# returning (wallet.py), so this branch is only reachable if a
# zero/negative row was somehow persisted; drop it so we never
# leave an orphan zero-balance key. Reuse the shared taxonomy
# (cashu_error) so the envelope matches the mapper above.
await session.delete(new_key)
await session.commit()
raise Exception("Token redemption failed")
raise HTTPException(
status_code=400,
detail={
"error": {
"message": "Failed to redeem Cashu token: token yielded no value",
"type": "cashu_error",
"code": "cashu_token_zero_value",
}
},
)
await session.refresh(new_key)
await session.commit()
@@ -364,6 +422,7 @@ async def validate_bearer_key(
except HTTPException:
raise
except Exception as e:
await session.rollback()
logger.error(
"Cashu token redemption failed",
extra={
@@ -378,7 +437,7 @@ async def validate_bearer_key(
status_code=401,
detail={
"error": {
"message": f"Invalid or expired Cashu key: {str(e)}",
"message": "Invalid or expired Cashu key",
"type": "invalid_request_error",
"code": "invalid_api_key",
}
@@ -534,31 +593,19 @@ async def pay_for_request(
)
# Charge the base cost for the request atomically to avoid race conditions
reserved_at_now = int(time.time())
stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == billing_key.hashed_key)
.where(col(ApiKey.balance) - col(ApiKey.reserved_balance) >= cost_per_request)
.values(
reserved_balance=col(ApiKey.reserved_balance) + cost_per_request,
reserved_at=reserved_at_now,
total_requests=col(ApiKey.total_requests) + 1,
)
)
result = await session.exec(stmt) # type: ignore[call-overload]
# Also increment total_requests and reserved_balance on the child key if it's different
if billing_key.hashed_key != key.hashed_key:
child_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.values(
total_requests=col(ApiKey.total_requests) + 1,
reserved_balance=col(ApiKey.reserved_balance) + cost_per_request,
)
)
await session.exec(child_stmt) # type: ignore[call-overload]
await session.commit()
if result.rowcount == 0:
logger.error(
"Concurrent request depleted balance",
@@ -570,7 +617,6 @@ async def pay_for_request(
},
)
# Another concurrent request spent the balance first
raise HTTPException(
status_code=402,
detail={
@@ -582,6 +628,44 @@ async def pay_for_request(
},
)
# Also increment total_requests and reserved_balance on the child key if it's different.
# The balance_limit guard is enforced atomically here — the Python pre-check above
# is a fast-path rejection only and provides no concurrency guarantee.
if billing_key.hashed_key != key.hashed_key:
child_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.where(
(col(ApiKey.balance_limit).is_(None))
| (
col(ApiKey.total_spent)
+ col(ApiKey.reserved_balance)
+ cost_per_request
<= col(ApiKey.balance_limit)
)
)
.values(
total_requests=col(ApiKey.total_requests) + 1,
reserved_balance=col(ApiKey.reserved_balance) + cost_per_request,
reserved_at=reserved_at_now,
)
)
child_result = await session.exec(child_stmt) # type: ignore[call-overload]
if child_result.rowcount == 0:
raise HTTPException(
status_code=402,
detail={
"error": {
"message": f"Balance limit exceeded: {key.balance_limit} mSats limit. {key.total_spent} already spent ({key.reserved_balance} reserved), {cost_per_request} required for this request.",
"type": "insufficient_quota",
"code": "balance_limit_exceeded",
}
},
)
await session.commit()
await session.refresh(billing_key)
if billing_key.hashed_key != key.hashed_key:
await session.refresh(key)
@@ -620,12 +704,19 @@ async def revert_pay_for_request(
False if the reservation was already released (prevents negative reserved_balance)."""
billing_key = await get_billing_key(key, session)
# Keep reserved_at while other reservations remain
cleared_reserved_at = case(
(col(ApiKey.reserved_balance) - cost_per_request > 0, col(ApiKey.reserved_at)),
else_=None,
)
stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == billing_key.hashed_key)
.where(col(ApiKey.reserved_balance) >= cost_per_request)
.values(
reserved_balance=col(ApiKey.reserved_balance) - cost_per_request,
reserved_at=cleared_reserved_at,
total_requests=col(ApiKey.total_requests) - 1,
)
)
@@ -641,6 +732,7 @@ async def revert_pay_for_request(
.values(
total_requests=col(ApiKey.total_requests) - 1,
reserved_balance=col(ApiKey.reserved_balance) - cost_per_request,
reserved_at=cleared_reserved_at,
)
)
await session.exec(child_stmt) # type: ignore[call-overload]
@@ -675,12 +767,18 @@ async def revert_pay_for_request(
async def adjust_payment_for_tokens(
key: ApiKey, response_data: dict, session: AsyncSession, deducted_max_cost: int
key: ApiKey,
response_data: dict,
session: AsyncSession,
deducted_max_cost: int,
) -> dict:
"""
Adjusts the payment based on token usage in the response.
This is called after the initial payment and the upstream request is complete.
Returns cost data to be included in the response.
The response's usage object is normalized with the default union parser in
``calculate_cost``.
"""
billing_key = await get_billing_key(key, session)
model = response_data.get("model", "unknown")
@@ -763,7 +861,7 @@ async def adjust_payment_for_tokens(
extra={"error": str(e), "fee_msats": fee_msats},
)
match await calculate_cost(response_data, deducted_max_cost, session):
match await calculate_cost(response_data, deducted_max_cost):
case MaxCostData() as cost:
logger.debug(
"Using max cost data (no token adjustment)",
@@ -981,32 +1079,37 @@ async def adjust_payment_for_tokens(
# actual cost exceeded discounted reservation (due to tolerance_percentage)
if cost_difference > 0:
# Always release the reservation and charge min(actual_cost, balance).
# Using a CASE expression makes this a single atomic UPDATE — no
# multi-level fallback needed and balance can never go negative.
# CASE expressions keep this atomic and safe even when the
# stale-reservation sweeper has already released the reservation.
chargeable = case(
(col(ApiKey.balance) >= total_cost_msats, total_cost_msats),
else_=col(ApiKey.balance),
)
overrun_safe_reserved = case(
(
col(ApiKey.reserved_balance) >= deducted_max_cost,
col(ApiKey.reserved_balance) - deducted_max_cost,
),
else_=0,
)
finalize_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == billing_key.hashed_key)
.where(col(ApiKey.reserved_balance) >= deducted_max_cost)
.values(
reserved_balance=col(ApiKey.reserved_balance) - deducted_max_cost,
reserved_balance=overrun_safe_reserved,
balance=col(ApiKey.balance) - chargeable,
total_spent=col(ApiKey.total_spent) + chargeable,
)
)
result = await session.exec(finalize_stmt) # type: ignore[call-overload]
await session.exec(finalize_stmt) # type: ignore[call-overload]
if billing_key.hashed_key != key.hashed_key:
child_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.where(col(ApiKey.reserved_balance) >= deducted_max_cost)
.values(
reserved_balance=col(ApiKey.reserved_balance) - deducted_max_cost,
reserved_balance=overrun_safe_reserved,
total_spent=col(ApiKey.total_spent) + min(billing_key.balance, total_cost_msats),
)
)
@@ -1014,51 +1117,38 @@ async def adjust_payment_for_tokens(
await session.commit()
if result.rowcount:
await session.refresh(billing_key)
if billing_key.hashed_key != key.hashed_key:
await session.refresh(key)
cost.total_msats = total_cost_msats
logger.info(
"Finalized payment with additional charge",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"charged_amount": total_cost_msats,
"new_balance": billing_key.balance,
"model": model,
},
)
await _accumulate_fee(total_cost_msats)
payments_logger.info(
"FINALIZE",
extra={
"event": "finalize",
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"model": model,
"cost_reserved": deducted_max_cost,
"cost_charged": total_cost_msats,
"input_tokens": cost.input_tokens,
"output_tokens": cost.output_tokens,
"balance": billing_key.balance,
"reserved_balance": billing_key.reserved_balance,
"total_spent": billing_key.total_spent,
"finalize_type": "overrun",
},
)
else:
# Guard fired: reservation was already released by a concurrent
# finalization for this key. Nothing left to do.
logger.warning(
"Finalization skipped - reservation already released",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"attempted_charge": total_cost_msats,
"model": model,
},
)
await session.refresh(billing_key)
if billing_key.hashed_key != key.hashed_key:
await session.refresh(key)
cost.total_msats = total_cost_msats
logger.info(
"Finalized payment with additional charge",
extra={
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"charged_amount": total_cost_msats,
"new_balance": billing_key.balance,
"model": model,
},
)
await _accumulate_fee(total_cost_msats)
payments_logger.info(
"FINALIZE",
extra={
"event": "finalize",
"key_hash": key.hashed_key[:8] + "...",
"billing_key_hash": billing_key.hashed_key[:8] + "...",
"model": model,
"cost_reserved": deducted_max_cost,
"cost_charged": total_cost_msats,
"input_tokens": cost.input_tokens,
"output_tokens": cost.output_tokens,
"balance": billing_key.balance,
"reserved_balance": billing_key.reserved_balance,
"total_spent": billing_key.total_spent,
"finalize_type": "overrun",
},
)
else:
# Refund some of the base cost
refund = abs(cost_difference)
@@ -1263,3 +1353,53 @@ async def periodic_key_reset() -> None:
break
except Exception as e:
logger.error(f"Error in periodic_key_reset: {e}")
async def periodic_dead_key_prune() -> None:
"""Periodically prune dead API keys. Interval <= 0 disables it.
See ``prune_dead_api_keys`` for eligibility.
"""
from .core.db import create_session, prune_dead_api_keys
interval = settings.dead_key_prune_interval_seconds
if interval <= 0:
logger.info("Dead-key pruning disabled (interval <= 0)")
return
while True:
try:
await asyncio.sleep(interval)
except asyncio.CancelledError:
break
try:
async with create_session() as session:
await prune_dead_api_keys(
session, settings.dead_key_min_age_seconds
)
except asyncio.CancelledError:
break
except Exception as e:
logger.error(f"Error in periodic_dead_key_prune: {e}")
STALE_RESERVATION_SWEEP_INTERVAL_SECONDS: int = 60
async def periodic_stale_reservation_sweep() -> None:
"""Background task that releases reservations leaked by client disconnects,
crashes or abandoned streams.
"""
from .core.db import create_session, release_stale_reservations
while True:
try:
async with create_session() as session:
await release_stale_reservations(
session, settings.stale_reservation_timeout_seconds
)
except Exception:
logger.exception("Error in periodic_stale_reservation_sweep")
await asyncio.sleep(STALE_RESERVATION_SWEEP_INTERVAL_SECONDS)

View File

@@ -7,7 +7,7 @@ from typing import Annotated, NoReturn
from fastapi import APIRouter, Depends, Header, HTTPException
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from sqlmodel import col, select, update
from sqlmodel import col, or_, select, update
from .auth import get_billing_key, validate_bearer_key
from .core.db import (
@@ -20,7 +20,14 @@ from .core.db import (
from .core.logging import get_logger
from .core.settings import settings
from .lightning import lightning_router
from .wallet import credit_balance, recieve_token, send_to_lnurl, send_token
from .wallet import (
classify_redemption_error,
credit_balance,
is_mint_connection_error,
recieve_token,
send_to_lnurl,
send_token,
)
router = APIRouter()
balance_router = APIRouter(prefix="/v1/balance")
@@ -48,7 +55,6 @@ async def get_balance_info(key: ApiKey, session: AsyncSession) -> dict:
"balance": billing_key.total_balance,
"reserved": billing_key.reserved_balance,
"is_child": key.parent_key_hash is not None,
"parent_key": "sk-" + key.parent_key_hash if key.parent_key_hash else None,
"total_requests": key.total_requests,
"total_spent": key.total_spent,
"balance_limit": key.balance_limit,
@@ -56,7 +62,9 @@ async def get_balance_info(key: ApiKey, session: AsyncSession) -> dict:
"validity_date": key.validity_date,
}
if not key.parent_key_hash:
if key.parent_key_hash:
info["parent_key_preview"] = key.parent_key_hash[:8] + "..."
else:
# Fetch child keys if this is a parent key
statement = select(ApiKey).where(ApiKey.parent_key_hash == key.hashed_key)
results = await session.exec(statement)
@@ -155,30 +163,18 @@ async def topup_wallet_endpoint(
raise HTTPException(status_code=400, detail="Invalid token format")
try:
amount_msats = await credit_balance(cashu_token, billing_key, session)
except ValueError as e:
error_msg = str(e)
if "already spent" in error_msg.lower():
raise HTTPException(status_code=400, detail="Token already spent")
elif "invalid" in error_msg.lower() or "decode" in error_msg.lower():
raise HTTPException(status_code=400, detail="Invalid token format")
elif "insufficient" in error_msg.lower() or "melt fee" in error_msg.lower():
raise HTTPException(
status_code=400,
detail=f"Token value is too small to cover swap fees. {error_msg}",
)
elif "failed to melt" in error_msg.lower():
raise HTTPException(
status_code=400,
detail=f"Failed to swap foreign mint token. {error_msg}",
)
else:
raise HTTPException(status_code=400, detail=f"Failed to redeem token: {error_msg}")
except Exception as e:
logger.error(
"topup_wallet_endpoint: unhandled error",
extra={"error": str(e), "error_type": type(e).__name__},
)
raise HTTPException(status_code=500, detail="Internal server error")
# Shared taxonomy so top-up matches the bearer/X-Cashu paths (503 for an
# unreachable mint, 422 for fee/swap failures, 400 for token faults).
classified = classify_redemption_error(e)
if classified is None:
logger.error(
"topup_wallet_endpoint: unhandled error",
extra={"error": str(e), "error_type": type(e).__name__},
)
raise HTTPException(status_code=500, detail="Internal server error")
_type, status_code, message, _code = classified
raise HTTPException(status_code=status_code, detail=message)
return {"msats": amount_msats}
@@ -273,7 +269,20 @@ async def refund_wallet_endpoint(
)
out_tx = out_tx_result.first()
if out_tx is None:
raise HTTPException(status_code=404, detail="Refund not found")
# The "in" row exists with a request_id, but the "out" (refund)
# row hasn't been written yet — the upstream request is still in
# flight and the refund will be minted once it completes. Tell the
# client to retry instead of 404ing permanently (race condition
# where /v1/wallet/refund is polled before the refund exists).
logger.debug(
"refund_wallet_endpoint: refund pending (in row exists, out row not yet created)",
extra={"request_id": in_tx.request_id},
)
raise HTTPException(
status_code=425,
detail="Refund is pending; retry shortly.",
headers={"Retry-After": "2"},
)
if out_tx.swept:
raise HTTPException(status_code=410, detail="Refund has been swept")
@@ -312,9 +321,36 @@ async def refund_wallet_endpoint(
)
if key.reserved_balance > 0:
raise HTTPException(
status_code=400,
detail="Cannot refund key. There are ongoing requests for this api key.",
# Release the reservation if it is stale
cutoff = int(time.time()) - settings.stale_reservation_timeout_seconds
stale_release_stmt = (
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.where(col(ApiKey.reserved_balance) > 0)
.where(
or_(
col(ApiKey.reserved_at).is_(None),
col(ApiKey.reserved_at) < cutoff,
)
)
.values(reserved_balance=0, reserved_at=None)
)
stale_result = await session.exec(stale_release_stmt) # type: ignore[call-overload]
await session.commit()
if stale_result.rowcount == 0:
raise HTTPException(
status_code=400,
detail="Cannot refund key. There are ongoing requests for this api key.",
)
await session.refresh(key)
logger.warning(
"refund_wallet_endpoint: released stale reservation before refund",
extra={
"hashed_key": key.hashed_key,
"stale_timeout_seconds": settings.stale_reservation_timeout_seconds,
},
)
remaining_balance_msats: int = key.total_balance
@@ -341,7 +377,7 @@ async def refund_wallet_endpoint(
.where(col(ApiKey.hashed_key) == key.hashed_key)
.where(col(ApiKey.balance) == pre_debit_balance)
.where(col(ApiKey.reserved_balance) == pre_debit_reserved)
.values(balance=0, reserved_balance=0)
.values(balance=0, reserved_balance=0, reserved_at=None)
)
debit_result = await session.exec(debit_stmt) # type: ignore[call-overload]
await session.commit()
@@ -413,14 +449,10 @@ async def refund_wallet_endpoint(
"has_refund_address": bool(key.refund_address),
},
)
if (
"mint" in error_msg.lower()
or "connection" in error_msg.lower()
or "ConnectError" in str(type(e))
):
raise HTTPException(status_code=503, detail=f"Mint service unavailable: {error_msg}")
if is_mint_connection_error(e):
raise HTTPException(status_code=503, detail="Mint service unavailable")
else:
raise HTTPException(status_code=500, detail=f"Refund failed: {error_msg}")
raise HTTPException(status_code=500, detail="Refund failed")
await _refund_cache_set(bearer_value, result)

View File

@@ -1,5 +1,6 @@
import asyncio
import json
import re
import secrets
from datetime import datetime, timezone
from pathlib import Path
@@ -8,6 +9,7 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Request
from pydantic import BaseModel, RootModel
from pydantic.v1 import ValidationError as PydanticValidationError
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from ..payment.models import _row_to_model, list_models
from ..proxy import refresh_model_maps, reinitialize_upstreams
@@ -29,6 +31,7 @@ from .db import (
)
from .log_manager import log_manager
from .logging import get_logger
from .provider_slugs import allocate_unique_provider_slug
from .settings import SettingsService, settings
logger = get_logger(__name__)
@@ -456,19 +459,18 @@ class ModelCreate(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def upsert_provider_model(
provider_id: int, payload: ModelCreate
provider_id: str, payload: ModelCreate
) -> dict[str, object]:
print(payload)
logger.info(
f"UPSERT_PROVIDER_MODEL called: provider_id={provider_id}, model_id={payload.id}"
)
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
# Try to get existing model
existing_row = await session.get(ModelRow, (payload.id, provider_id))
existing_row = await session.get(ModelRow, (payload.id, provider_pk))
if existing_row:
# Update existing model
@@ -524,7 +526,7 @@ async def upsert_provider_model(
alias_ids=(
json.dumps(payload.alias_ids) if payload.alias_ids else None
),
upstream_provider_id=provider_id,
upstream_provider_id=provider_pk,
enabled=payload.enabled,
forwarded_model_id=payload.forwarded_model_id or payload.id,
)
@@ -543,7 +545,7 @@ async def upsert_provider_model(
dependencies=[Depends(require_admin_api)],
)
async def update_provider_model_legacy(
provider_id: int, model_id: str, payload: ModelCreate
provider_id: str, model_id: str, payload: ModelCreate
) -> dict[str, object]:
"""Legacy PATCH endpoint - redirects to upsert POST endpoint for backward compatibility."""
logger.info(
@@ -556,13 +558,12 @@ async def update_provider_model_legacy(
"/api/upstream-providers/{provider_id}/models/{model_id:path}",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_model(provider_id: int, model_id: str) -> dict[str, object]:
async def get_provider_model(provider_id: str, model_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
row = await session.get(ModelRow, (model_id, provider_id))
row = await session.get(ModelRow, (model_id, provider_pk))
if not row:
raise HTTPException(
status_code=404, detail="Model not found for this provider"
@@ -576,9 +577,11 @@ async def get_provider_model(provider_id: int, model_id: str) -> dict[str, objec
"/api/upstream-providers/{provider_id}/models/{model_id:path}",
dependencies=[Depends(require_admin_api)],
)
async def delete_provider_model(provider_id: int, model_id: str) -> dict[str, object]:
async def delete_provider_model(provider_id: str, model_id: str) -> dict[str, object]:
async with create_session() as session:
row = await session.get(ModelRow, (model_id, provider_id))
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
row = await session.get(ModelRow, (model_id, provider_pk))
if not row:
raise HTTPException(
status_code=404, detail="Model not found for this provider"
@@ -593,10 +596,12 @@ async def delete_provider_model(provider_id: int, model_id: str) -> dict[str, ob
"/api/upstream-providers/{provider_id}/models",
dependencies=[Depends(require_admin_api)],
)
async def delete_all_provider_models(provider_id: int) -> dict[str, object]:
async def delete_all_provider_models(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
result = await session.exec(
select(ModelRow).where(ModelRow.upstream_provider_id == provider_id)
select(ModelRow).where(ModelRow.upstream_provider_id == provider_pk)
) # type: ignore
rows = result.all()
for row in rows:
@@ -615,7 +620,7 @@ class BatchOverrideRequest(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def batch_override_provider_models(
provider_id: int, payload: BatchOverrideRequest
provider_id: str, payload: BatchOverrideRequest
) -> dict[str, object]:
"""Batch override models for a specific provider."""
logger.info(
@@ -623,15 +628,14 @@ async def batch_override_provider_models(
)
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
overridden_count = 0
for model_data in payload.models:
# Try to get existing model regardless of whether it's enabled or not
existing_row = await session.get(ModelRow, (model_data.id, provider_id))
existing_row = await session.get(ModelRow, (model_data.id, provider_pk))
if existing_row:
# Update existing
@@ -685,7 +689,7 @@ async def batch_override_provider_models(
if model_data.alias_ids
else None
),
upstream_provider_id=provider_id,
upstream_provider_id=provider_pk,
enabled=model_data.enabled,
)
session.add(row)
@@ -702,6 +706,85 @@ async def batch_override_provider_models(
}
_SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9-]{1,62}[a-z0-9]$")
def _validate_slug(value: str) -> str:
candidate = value.strip().lower()
if not _SLUG_PATTERN.fullmatch(candidate):
raise HTTPException(
status_code=400,
detail=(
"slug must be 3-64 chars, lowercase letters/digits/hyphens, "
"and may not start or end with a hyphen"
),
)
if candidate.isdigit():
raise HTTPException(
status_code=400,
detail="slug must not be all digits",
)
return candidate
async def _ensure_unique_slug(
session: AsyncSession, slug: str, exclude_id: int | None = None
) -> None:
stmt = select(UpstreamProviderRow).where(UpstreamProviderRow.slug == slug)
result = await session.exec(stmt)
existing = result.first()
if existing and existing.id != exclude_id:
raise HTTPException(
status_code=409,
detail="Provider with this slug already exists",
)
async def _get_upstream_provider_by_ref(
session: AsyncSession, provider_ref: str
) -> UpstreamProviderRow:
if provider_ref.isdigit():
provider = await session.get(UpstreamProviderRow, int(provider_ref))
else:
slug = _validate_slug(provider_ref)
result = await session.exec(
select(UpstreamProviderRow).where(UpstreamProviderRow.slug == slug)
)
provider = result.first()
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
return provider
def _provider_pk(provider: UpstreamProviderRow) -> int:
if provider.id is None:
raise HTTPException(status_code=500, detail="Provider has no database id")
return provider.id
def _serialize_provider(
provider: UpstreamProviderRow, redact_api_key: bool = True
) -> dict[str, object]:
return {
"id": provider.id,
"slug": provider.slug,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]"
if (redact_api_key and provider.api_key)
else provider.api_key
if not redact_api_key
else "",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
class UpstreamProviderCreate(BaseModel):
provider_type: str
base_url: str
@@ -710,6 +793,7 @@ class UpstreamProviderCreate(BaseModel):
enabled: bool = True
provider_fee: float = 1.01
provider_settings: dict | None = None
slug: str | None = None
class UpstreamProviderUpdate(BaseModel):
@@ -720,6 +804,50 @@ class UpstreamProviderUpdate(BaseModel):
enabled: bool | None = None
provider_fee: float | None = None
provider_settings: dict | None = None
slug: str | None = None
class UpstreamProviderUpdateBySlug(BaseModel):
slug: str
new_slug: str | None = None
provider_type: str | None = None
base_url: str | None = None
api_key: str | None = None
api_version: str | None = None
enabled: bool | None = None
provider_fee: float | None = None
provider_settings: dict | None = None
async def _apply_provider_update(
session: AsyncSession,
provider: UpstreamProviderRow,
payload: UpstreamProviderUpdate,
new_slug: str | None = None,
) -> None:
if new_slug is not None:
validated = _validate_slug(new_slug)
await _ensure_unique_slug(session, validated, exclude_id=provider.id)
provider.slug = validated
if payload.provider_type is not None:
provider.provider_type = payload.provider_type
if payload.base_url is not None:
provider.base_url = payload.base_url
if payload.api_key is not None:
provider.api_key = payload.api_key
if payload.api_version is not None:
provider.api_version = payload.api_version
if payload.enabled is not None:
provider.enabled = payload.enabled
if payload.provider_fee is not None:
provider.provider_fee = payload.provider_fee
if payload.provider_settings is not None:
provider.provider_settings = json.dumps(payload.provider_settings)
session.add(provider)
await session.commit()
await session.refresh(provider)
@admin_router.get("/api/upstream-providers", dependencies=[Depends(require_admin_api)])
@@ -727,21 +855,7 @@ async def get_upstream_providers() -> list[dict[str, object]]:
async with create_session() as session:
result = await session.exec(select(UpstreamProviderRow))
providers = result.all()
return [
{
"id": p.id,
"provider_type": p.provider_type,
"base_url": p.base_url,
"api_key": "[REDACTED]" if p.api_key else "",
"api_version": p.api_version,
"enabled": p.enabled,
"provider_fee": p.provider_fee,
"provider_settings": json.loads(p.provider_settings)
if p.provider_settings
else None,
}
for p in providers
]
return [_serialize_provider(p) for p in providers]
@admin_router.post("/api/upstream-providers", dependencies=[Depends(require_admin_api)])
@@ -761,7 +875,14 @@ async def create_upstream_provider(
detail="Provider with this base URL and API key already exists",
)
if payload.slug:
slug = _validate_slug(payload.slug)
await _ensure_unique_slug(session, slug)
else:
slug = await allocate_unique_provider_slug(session, payload.provider_type)
provider = UpstreamProviderRow(
slug=slug,
provider_type=payload.provider_type,
base_url=payload.base_url,
api_key=payload.api_key,
@@ -778,99 +899,81 @@ async def create_upstream_provider(
await reinitialize_upstreams()
await refresh_model_maps()
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": payload.provider_settings,
}
return _serialize_provider(provider)
@admin_router.get(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def get_upstream_provider(provider_id: int) -> dict[str, object]:
async def get_upstream_provider(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]" if provider.api_key else "",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
provider = await _get_upstream_provider_by_ref(session, provider_id)
return _serialize_provider(provider)
@admin_router.patch(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def update_upstream_provider(
provider_id: int, payload: UpstreamProviderUpdate
provider_id: str, payload: UpstreamProviderUpdate
) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
if payload.provider_type is not None:
provider.provider_type = payload.provider_type
if payload.base_url is not None:
provider.base_url = payload.base_url
if payload.api_key is not None:
provider.api_key = payload.api_key
if payload.api_version is not None:
provider.api_version = payload.api_version
if payload.enabled is not None:
provider.enabled = payload.enabled
if payload.provider_fee is not None:
provider.provider_fee = payload.provider_fee
if payload.provider_settings is not None:
provider.provider_settings = json.dumps(payload.provider_settings)
session.add(provider)
await session.commit()
await session.refresh(provider)
await _apply_provider_update(session, provider, payload, new_slug=payload.slug)
await reinitialize_upstreams()
await refresh_model_maps()
return {
"id": provider.id,
"provider_type": provider.provider_type,
"base_url": provider.base_url,
"api_key": "[REDACTED]",
"api_version": provider.api_version,
"enabled": provider.enabled,
"provider_fee": provider.provider_fee,
"provider_settings": json.loads(provider.provider_settings)
if provider.provider_settings
else None,
}
return _serialize_provider(provider)
@admin_router.patch(
"/api/upstream-providers", dependencies=[Depends(require_admin_api)]
)
async def update_upstream_provider_by_slug(
payload: UpstreamProviderUpdateBySlug,
) -> dict[str, object]:
lookup = _validate_slug(payload.slug)
async with create_session() as session:
result = await session.exec(
select(UpstreamProviderRow).where(
UpstreamProviderRow.slug == lookup
)
)
provider = result.first()
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
update_payload = UpstreamProviderUpdate(
provider_type=payload.provider_type,
base_url=payload.base_url,
api_key=payload.api_key,
api_version=payload.api_version,
enabled=payload.enabled,
provider_fee=payload.provider_fee,
provider_settings=payload.provider_settings,
)
await _apply_provider_update(
session, provider, update_payload, new_slug=payload.new_slug
)
await reinitialize_upstreams()
await refresh_model_maps()
return _serialize_provider(provider)
@admin_router.delete(
"/api/upstream-providers/{provider_id}", dependencies=[Depends(require_admin_api)]
)
async def delete_upstream_provider(provider_id: int) -> dict[str, object]:
async def delete_upstream_provider(provider_id: str) -> dict[str, object]:
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
deleted_id = _provider_pk(provider)
await session.delete(provider)
await session.commit()
await reinitialize_upstreams()
await refresh_model_maps()
return {"ok": True, "deleted_id": provider_id}
return {"ok": True, "deleted_id": deleted_id}
@admin_router.get("/api/provider-types", dependencies=[Depends(require_admin_api)])
@@ -885,17 +988,16 @@ async def get_provider_types() -> list[dict[str, object]]:
"/api/upstream-providers/{provider_id}/models",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_models(provider_id: int) -> dict[str, object]:
async def get_provider_models(provider_id: str) -> dict[str, object]:
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
provider_pk = _provider_pk(provider)
db_models = await list_models(
session=session,
upstream_id=provider_id,
upstream_id=provider_pk,
include_disabled=True,
apply_fees=False,
)
@@ -985,13 +1087,11 @@ class TopupTokenRequest(BaseModel):
dependencies=[Depends(require_admin_api)],
)
async def topup_provider_with_token(
provider_id: int, payload: TopupTokenRequest
provider_id: str, payload: TopupTokenRequest
) -> dict:
"""Redeem a Cashu token for an upstream provider."""
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
import httpx
@@ -1022,15 +1122,13 @@ async def topup_provider_with_token(
dependencies=[Depends(require_admin_api)],
)
async def initiate_provider_topup(
provider_id: int, payload: TopupRequest
provider_id: str, payload: TopupRequest
) -> dict[str, object]:
"""Initiate a Lightning Network top-up for the upstream provider account."""
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
try:
logger.info(
@@ -1150,15 +1248,13 @@ async def initiate_provider_topup(
"/api/upstream-providers/{provider_id}/topup/{invoice_id}/status",
dependencies=[Depends(require_admin_api)],
)
async def check_topup_status(provider_id: int, invoice_id: str) -> dict[str, object]:
async def check_topup_status(provider_id: str, invoice_id: str) -> dict[str, object]:
"""Check the status of a Lightning Network top-up invoice."""
from ..upstream.helpers import _instantiate_provider
from ..upstream.ppqai import PPQAIUpstreamProvider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
# For Routstr providers, proxy the status check
if provider.provider_type == "routstr":
@@ -1205,14 +1301,12 @@ async def check_topup_status(provider_id: int, invoice_id: str) -> dict[str, obj
"/api/upstream-providers/{provider_id}/balance",
dependencies=[Depends(require_admin_api)],
)
async def get_provider_balance(provider_id: int) -> dict[str, object]:
async def get_provider_balance(provider_id: str) -> dict[str, object]:
"""Get the current balance for an upstream provider account."""
from ..upstream.helpers import _instantiate_provider
async with create_session() as session:
provider = await session.get(UpstreamProviderRow, provider_id)
if not provider:
raise HTTPException(status_code=404, detail="Provider not found")
provider = await _get_upstream_provider_by_ref(session, provider_id)
# For Routstr providers, proxy the balance check
if provider.provider_type == "routstr":
@@ -1591,15 +1685,13 @@ async def get_lightning_invoices_api(
"/api/upstream-providers/{provider_id}/routstr/refund",
dependencies=[Depends(require_admin_api)],
)
async def refund_routstr_provider_balance(provider_id: int) -> dict[str, object]:
async def refund_routstr_provider_balance(provider_id: str) -> dict[str, object]:
"""Refund balance from an upstream Routstr provider back to the local wallet."""
from ..upstream.helpers import _instantiate_provider
from ..upstream.routstr import RoutstrUpstreamProvider
async with create_session() as session:
provider_row = await session.get(UpstreamProviderRow, provider_id)
if not provider_row:
raise HTTPException(status_code=404, detail="Provider not found")
provider_row = await _get_upstream_provider_by_ref(session, provider_id)
if provider_row.provider_type != "routstr":
raise HTTPException(

View File

@@ -9,9 +9,10 @@ from typing import AsyncGenerator
from alembic import command
from alembic.config import Config
from alembic.util.exc import CommandError
from sqlalchemy import UniqueConstraint
from sqlalchemy import UniqueConstraint, delete
from sqlalchemy.exc import OperationalError
from sqlalchemy.ext.asyncio.engine import create_async_engine
from sqlalchemy.orm import aliased
from sqlmodel import Field, Relationship, SQLModel, col, func, select, update
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -33,6 +34,14 @@ class ApiKey(SQLModel, table=True): # type: ignore
reserved_balance: int = Field(
default=0, description="Reserved balance in millisatoshis (msats)"
)
reserved_at: int | None = Field(
default=None,
description=(
"Unix timestamp of the most recent balance reservation. Used to "
"detect and release stale reservations (e.g. after client "
"disconnects). NULL when no reservation has been made yet."
),
)
refund_address: str | None = Field(
default=None,
description="Lightning address to refund remaining balance after key expires",
@@ -87,12 +96,93 @@ class ApiKey(SQLModel, table=True): # type: ignore
async def reset_all_reserved_balances(session: AsyncSession) -> None:
stmt = update(ApiKey).values(reserved_balance=0)
stmt = update(ApiKey).values(reserved_balance=0, reserved_at=None)
await session.exec(stmt) # type: ignore[call-overload]
await session.commit()
logger.info("Reset reserved balances on startup")
async def release_stale_reservations(
session: AsyncSession, max_age_seconds: int
) -> int:
"""Release reservations whose last reserve is older than max_age_seconds.
"""
cutoff = int(time.time()) - max_age_seconds
stmt = (
update(ApiKey)
.where(col(ApiKey.reserved_balance) > 0)
.where(col(ApiKey.reserved_at).is_not(None))
.where(col(ApiKey.reserved_at) < cutoff)
.values(reserved_balance=0, reserved_at=None)
)
result = await session.exec(stmt) # type: ignore[call-overload]
await session.commit()
released = int(result.rowcount or 0)
if released:
logger.warning(
"Released stale balance reservations",
extra={"released_keys": released, "max_age_seconds": max_age_seconds},
)
return released
async def prune_dead_api_keys(session: AsyncSession, min_age_seconds: int) -> int:
"""Delete dead parentless API keys; return the count removed.
Dead = 0 balance/reservation/spend/requests, older than the grace period,
no parent, no children, no pending invoice. Cashu rows are unlinked (not
deleted) first to keep the audit trail.
"""
cutoff = int(time.time()) - min_age_seconds
child = aliased(ApiKey)
has_children = (
select(child.hashed_key).where(
col(child.parent_key_hash) == col(ApiKey.hashed_key)
)
).exists()
pending_invoice = (
select(LightningInvoice.id)
.where(col(LightningInvoice.api_key_hash) == col(ApiKey.hashed_key))
.where(col(LightningInvoice.status) == "pending")
).exists()
eligible_hashes = (
select(ApiKey.hashed_key)
.where(col(ApiKey.balance) == 0)
.where(col(ApiKey.reserved_balance) == 0)
.where(col(ApiKey.total_spent) == 0)
.where(col(ApiKey.total_requests) == 0)
.where(col(ApiKey.parent_key_hash).is_(None))
.where(
(col(ApiKey.created_at).is_(None)) | (col(ApiKey.created_at) < cutoff)
)
.where(~pending_invoice)
.where(~has_children)
)
# Unlink transactions rather than cascade-deleting them, so the financial
# audit trail survives. The eligibility predicate is re-evaluated inside both
# statements so a key that gained balance mid-run is left untouched.
await session.exec( # type: ignore[call-overload]
update(CashuTransaction)
.where(col(CashuTransaction.api_key_hashed_key).in_(eligible_hashes))
.values(api_key_hashed_key=None)
)
result = await session.exec( # type: ignore[call-overload]
delete(ApiKey).where(col(ApiKey.hashed_key).in_(eligible_hashes))
)
await session.commit()
pruned = int(result.rowcount or 0)
logger.info(
"Pruned dead API keys",
extra={"pruned_keys": pruned, "min_age_seconds": min_age_seconds},
)
return pruned
class ModelRow(SQLModel, table=True): # type: ignore
__tablename__ = "models"
id: str = Field(primary_key=True)
@@ -229,6 +319,12 @@ class UpstreamProviderRow(SQLModel, table=True): # type: ignore
),
)
id: int | None = Field(default=None, primary_key=True)
slug: str | None = Field(
default=None,
unique=True,
index=True,
description="Stable external slug used for updates via API key.",
)
provider_type: str = Field(
description="Provider type: custom, openai, anthropic, azure, openrouter, etc."
)

View File

@@ -7,11 +7,26 @@ logger = get_logger(__name__)
class UpstreamError(Exception):
"""Exception raised when an upstream provider fails."""
"""Exception raised when an upstream provider fails.
def __init__(self, message: str, status_code: int = 502):
``code`` carries a stable, machine-readable classification (e.g.
``UPSTREAM_RATE_LIMIT``) so callers can distinguish failure kinds without
string-matching the message. ``details`` holds optional structured,
redaction-safe context. Both default to ``None`` for backwards
compatibility.
"""
def __init__(
self,
message: str,
status_code: int = 502,
code: str | None = None,
details: dict[str, object] | None = None,
):
self.message = message
self.status_code = status_code
self.code = code
self.details = details
super().__init__(message)

View File

@@ -51,6 +51,8 @@ from pythonjsonlogger import jsonlogger
from rich.console import Console
from rich.logging import RichHandler
from .redaction import redact_obj, redact_org_ids
# Only use RichHandler when stdout is a real TTY. In non-TTY contexts
# (docker logs, pipes, CI) Rich pads every line to width and wraps long
# records, producing visually-empty trailing whitespace and split records.
@@ -180,6 +182,37 @@ class RequestIdFilter(logging.Filter):
return True
# Standard ``LogRecord`` attributes that are never user-supplied ``extra``
# fields; skipped when redacting structured extras (``msg``/``message`` are
# handled separately above).
_NON_EXTRA_RECORD_ATTRS = frozenset(
{
"name",
"msg",
"args",
"levelname",
"levelno",
"pathname",
"filename",
"module",
"exc_info",
"exc_text",
"stack_info",
"lineno",
"funcName",
"created",
"msecs",
"relativeCreated",
"thread",
"threadName",
"processName",
"process",
"taskName",
"message",
}
)
class SecurityFilter(logging.Filter):
"""Filter to remove sensitive information from logs."""
@@ -203,6 +236,7 @@ class SecurityFilter(logging.Filter):
"""Filter out sensitive information from log records."""
try:
message = record.getMessage()
message = redact_org_ids(message)
standalone_patterns = [
r"Bearer\s+([a-zA-Z0-9_\-\.]{10,})", # Bearer token (must be 10 characters or more to reduce false-positives)
r"cashu[A-Z]+([a-zA-Z0-9_\-\.=/+]+)", # Cashu tokens
@@ -224,6 +258,16 @@ class SecurityFilter(logging.Filter):
record.msg = message
record.args = ()
# Structured `extra={...}` fields are emitted by the JSON formatter
# straight from the record dict and never pass through the message
# formatting above. Redact organization IDs from any string-valued
# extra so they cannot leak via structured logs.
for attr, value in list(record.__dict__.items()):
if attr in _NON_EXTRA_RECORD_ATTRS:
continue
if isinstance(value, (str, dict, list, tuple)):
record.__dict__[attr] = redact_obj(value)
except Exception:
pass

View File

@@ -11,7 +11,11 @@ from starlette.exceptions import HTTPException
from starlette.responses import Response as StarletteResponse
from starlette.types import Scope
from ..auth import periodic_key_reset
from ..auth import (
periodic_dead_key_prune,
periodic_key_reset,
periodic_stale_reservation_sweep,
)
from ..balance import balance_router, deprecated_wallet_router
from ..lightning import lightning_router, periodic_invoice_watcher
from ..nostr import (
@@ -24,6 +28,7 @@ from ..payment.models import models_router, update_sats_pricing
from ..payment.price import update_prices_periodically
from ..proxy import initialize_upstreams, proxy_router, refresh_model_maps_periodically
from ..upstream.auto_topup import periodic_auto_topup
from ..upstream.deepseek_v4_pricing_shim import register_deepseek_v4_pricing
from ..upstream.litellm_routing import configure_litellm
from ..wallet import periodic_payout, periodic_refund_sweep, periodic_routstr_fee_payout
from .admin import admin_router
@@ -54,6 +59,8 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
models_refresh_task = None
model_maps_refresh_task = None
key_reset_task = None
stale_reservation_task = None
dead_key_prune_task = None
auto_topup_task = None
refund_sweep_task = None
routstr_fee_task = None
@@ -64,6 +71,11 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
# debug logging) before any upstream provider dispatches a request.
configure_litellm()
# TEMPORARY: backfill DeepSeek V4 pricing missing from litellm's cost
# map (BerriAI/litellm#30430). Remove this call and
# deepseek_v4_pricing_shim.py once litellm ships these models.
register_deepseek_v4_pricing()
# Run database migrations on startup
run_migrations()
@@ -122,6 +134,10 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
if global_settings.providers_refresh_interval_seconds > 0:
providers_task = asyncio.create_task(providers_cache_refresher())
key_reset_task = asyncio.create_task(periodic_key_reset())
stale_reservation_task = asyncio.create_task(
periodic_stale_reservation_sweep()
)
dead_key_prune_task = asyncio.create_task(periodic_dead_key_prune())
auto_topup_task = asyncio.create_task(periodic_auto_topup())
refund_sweep_task = asyncio.create_task(periodic_refund_sweep())
routstr_fee_task = asyncio.create_task(periodic_routstr_fee_payout())
@@ -159,6 +175,10 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
model_maps_refresh_task.cancel()
if key_reset_task is not None:
key_reset_task.cancel()
if stale_reservation_task is not None:
stale_reservation_task.cancel()
if dead_key_prune_task is not None:
dead_key_prune_task.cancel()
if auto_topup_task is not None:
auto_topup_task.cancel()
if refund_sweep_task is not None:
@@ -188,6 +208,10 @@ async def lifespan(_: FastAPI) -> AsyncGenerator[None, None]:
tasks_to_wait.append(model_maps_refresh_task)
if key_reset_task is not None:
tasks_to_wait.append(key_reset_task)
if stale_reservation_task is not None:
tasks_to_wait.append(stale_reservation_task)
if dead_key_prune_task is not None:
tasks_to_wait.append(dead_key_prune_task)
if auto_topup_task is not None:
tasks_to_wait.append(auto_topup_task)
if refund_sweep_task is not None:
@@ -355,7 +379,10 @@ if UI_DIST_PATH.exists() and UI_DIST_PATH.is_dir():
else:
logger.warning(
f"UI dist directory not found at {UI_DIST_PATH}, skipping static file serving"
"UI dist directory not found at %s; serving API only. Run `make ui-build` "
"to build the static UI served from here, or `make ui-dev` for the Next.js "
"dev server with hot reload on :3000 (it targets this backend on :8000).",
UI_DIST_PATH,
)
@app.get("/", include_in_schema=False)

View File

@@ -0,0 +1,65 @@
from __future__ import annotations
import re
from itertools import count
from typing import Collection
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from .db import UpstreamProviderRow
_SLUG_BASE_PATTERN = re.compile(r"[^a-z0-9]+")
_MAX_SLUG_LENGTH = 64
def provider_slug_base(provider_type: str) -> str:
"""Return a deterministic slug base for a provider type."""
base = _SLUG_BASE_PATTERN.sub("-", provider_type.lower()).strip("-")
if not base:
base = "provider"
elif base.isdigit():
base = f"provider-{base}"
elif len(base) < 3:
base = f"{base}-provider"
if len(base) > _MAX_SLUG_LENGTH:
base = base[:_MAX_SLUG_LENGTH].rstrip("-") or "provider"
return base
def provider_slug_candidate(base: str, suffix_number: int) -> str:
if suffix_number == 1:
return base
suffix = f"-{suffix_number}"
max_base_length = _MAX_SLUG_LENGTH - len(suffix)
return f"{base[:max_base_length].rstrip('-')}{suffix}"
async def allocate_unique_provider_slug(
session: AsyncSession,
provider_type: str,
reserved_slugs: Collection[str] = (),
) -> str:
"""Allocate a stable, deterministic provider slug.
The first provider of a type gets ``openai``; later collisions get
``openai-2``, ``openai-3``, etc. ``reserved_slugs`` covers rows staged in
memory but not flushed yet, such as settings/env seeding.
"""
base = provider_slug_base(provider_type)
reserved = {slug.lower() for slug in reserved_slugs}
for suffix_number in count(1):
candidate = provider_slug_candidate(base, suffix_number)
if candidate in reserved:
continue
result = await session.exec(
select(UpstreamProviderRow).where(UpstreamProviderRow.slug == candidate)
)
if result.first() is None:
return candidate
raise RuntimeError("unreachable")

51
routstr/core/redaction.py Normal file
View File

@@ -0,0 +1,51 @@
"""Redaction helpers for sensitive provider identifiers.
Single source of truth for stripping account-scoped identifiers (e.g. OpenAI
organization IDs) from any text before it is logged, returned to a caller, or
written to an audit entry.
"""
from __future__ import annotations
import re
from typing import Any
# OpenAI-style organization identifiers look like ``org-<base62>``. Require at
# least 6 trailing chars so the already-redacted literal ``org-[REDACTED]`` is
# never re-matched (``[`` is not in the character class).
_ORG_ID_PATTERN = re.compile(r"\borg-[A-Za-z0-9]{6,}\b")
ORG_ID_PLACEHOLDER = "org-[REDACTED]"
def redact_org_ids(text: str) -> str:
"""Replace OpenAI-style organization IDs with ``org-[REDACTED]``.
Args:
text: Arbitrary text that may embed an ``org-*`` identifier.
Returns:
The text with every organization ID replaced. Non-string input is
returned unchanged after coercion to ``str``.
"""
if not text:
return text
return _ORG_ID_PATTERN.sub(ORG_ID_PLACEHOLDER, text)
def redact_obj(obj: Any) -> Any:
"""Recursively redact organization IDs in arbitrary nested structures.
Strings are redacted in place; dicts and lists/tuples are walked so that
identifiers nested inside structured payloads (e.g. log ``extra`` fields or
error ``details``) are also stripped. Other types are returned unchanged.
"""
if isinstance(obj, str):
return redact_org_ids(obj)
if isinstance(obj, dict):
return {key: redact_obj(value) for key, value in obj.items()}
if isinstance(obj, list):
return [redact_obj(value) for value in obj]
if isinstance(obj, tuple):
return tuple(redact_obj(value) for value in obj)
return obj

View File

@@ -67,6 +67,20 @@ class Settings(BaseSettings):
reset_reserved_balance_on_startup: bool = Field(
default=True, env="RESET_RESERVED_BALANCE_ON_STARTUP"
) # deactivate in horizontal scaling setups
# Reservations older than this are considered leaked (client disconnect,
# crash, abandoned stream) and released by the background sweeper and the
# refund endpoint.
stale_reservation_timeout_seconds: int = Field(
default=300, env="STALE_RESERVATION_TIMEOUT_SECONDS"
)
# Background prune of dead (zero balance, never used) API keys.
# Interval 0 disables it; min-age is a grace period (default 1 week).
dead_key_prune_interval_seconds: int = Field(
default=3600, env="DEAD_KEY_PRUNE_INTERVAL_SECONDS"
)
dead_key_min_age_seconds: int = Field(
default=604_800, env="DEAD_KEY_MIN_AGE_SECONDS"
)
# Network
cors_origins: list[str] = Field(default_factory=lambda: ["*"], env="CORS_ORIGINS")

View File

@@ -20,7 +20,7 @@ import subprocess
from functools import lru_cache
from pathlib import Path
BASE_VERSION = "0.4.3"
BASE_VERSION = "0.4.4"
_REPO_ROOT = Path(__file__).resolve().parents[2]
_GIT_TIMEOUT_SECONDS = 2.0

View File

@@ -3,9 +3,17 @@ import math
from pydantic.v1 import BaseModel
from ..core import get_logger
from ..core.db import AsyncSession
from ..core.settings import settings
from .price import sats_usd_price
from .usage import normalize_usage, parse_token_count
__all__ = [
"CostData",
"CostDataError",
"MaxCostData",
"calculate_cost",
"parse_token_count",
]
logger = get_logger(__name__)
@@ -33,8 +41,31 @@ class CostDataError(BaseModel):
code: str
def _empty_cost(cls: type[CostData] = CostData) -> CostData:
"""Build an all-zero cost object — a full refund for an empty response.
Shared by the two paths that must not bill: an upstream response with no
usage data at all, and one that reports a USD cost but carries zero tokens
in every bucket.
"""
return cls(
base_msats=0,
input_msats=0,
output_msats=0,
total_msats=0,
total_usd=0.0,
input_tokens=0,
output_tokens=0,
cache_read_input_tokens=0,
cache_creation_input_tokens=0,
cache_read_msats=0,
cache_creation_msats=0,
)
async def calculate_cost(
response_data: dict, max_cost: int, session: AsyncSession
response_data: dict,
max_cost: int,
) -> CostData | MaxCostData | CostDataError:
"""Calculate the cost of an API request based on token usage.
@@ -44,6 +75,9 @@ async def calculate_cost(
Returns:
Cost data or error information
The response's usage object is normalized with the default union parser;
this function holds no vendor-dialect knowledge of its own.
"""
logger.debug(
"Starting cost calculation",
@@ -54,8 +88,9 @@ async def calculate_cost(
},
)
# Check for usage data
if "usage" not in response_data or response_data["usage"] is None:
usage = normalize_usage(response_data.get("usage"))
if usage is None:
logger.warning(
"No usage data in response — billing at MaxCostData with zero "
"tokens. Dashboard will show this request as `(0+0)`. Most "
@@ -70,34 +105,41 @@ async def calculate_cost(
else None,
},
)
return MaxCostData(
base_msats=0,
input_msats=0,
output_msats=0,
total_msats=0,
total_usd=0.0,
input_tokens=0,
output_tokens=0,
cache_read_input_tokens=0,
cache_creation_input_tokens=0,
cache_read_msats=0,
cache_creation_msats=0,
)
return _empty_cost(MaxCostData)
usage_data = response_data["usage"]
usage_data = response_data.get("usage") or {}
if not isinstance(usage_data, dict):
usage_data = {}
# Extract token counts
input_tokens = _extract_token_pair(usage_data, "prompt_tokens", "input_tokens")
output_tokens = _extract_token_pair(usage_data, "completion_tokens", "output_tokens")
# Extract cache tokens (handles OpenAI vs Anthropic formats)
cache_read_tokens, cache_creation_tokens, input_tokens = _extract_cache_tokens(
usage_data, input_tokens
)
input_tokens = usage.input_tokens
output_tokens = usage.output_tokens
cache_read_tokens = usage.cache_read_tokens
cache_creation_tokens = usage.cache_write_tokens
# Try USD cost first
usd_cost = _resolve_usd_cost(usage_data, response_data)
if usd_cost > 0:
truly_empty = (
input_tokens == 0
and output_tokens == 0
and cache_read_tokens == 0
and cache_creation_tokens == 0
)
if truly_empty:
logger.warning(
"Upstream reported a USD cost but the response carries no "
"tokens at all (input, output, cache-read and cache-creation "
"are all zero) — refunding in full rather than billing the "
"USD-derived cost for an empty response.",
extra={
"model": response_data.get("model", "unknown"),
"usd_cost": usd_cost,
"usage_keys": sorted(usage_data.keys())
if isinstance(usage_data, dict)
else None,
},
)
return _empty_cost()
if input_tokens == 0 and output_tokens == 0:
logger.warning(
"Upstream reported a USD cost but no token counts — "
@@ -202,22 +244,6 @@ async def calculate_cost(
# ============================================================================
def parse_token_count(value: object) -> int:
"""Parse a token count from various formats (int, float, str, bool)."""
if isinstance(value, bool):
return 0
if isinstance(value, int):
return max(0, value)
if isinstance(value, float):
return max(0, int(value))
if isinstance(value, str):
try:
return max(0, int(float(value)))
except ValueError:
return 0
return 0
def _coerce_usd(value: object) -> float:
"""Coerce a value to USD float, handling various formats safely."""
if value is None or isinstance(value, bool):
@@ -230,37 +256,6 @@ def _coerce_usd(value: object) -> float:
return 0.0
def _extract_token_pair(
usage_data: dict, standard_field: str, alt_field: str
) -> int:
"""Extract token count trying two field names in order."""
value = parse_token_count(usage_data.get(standard_field, 0))
if value > 0:
return value
return parse_token_count(usage_data.get(alt_field, 0))
def _extract_cache_tokens(usage_data: dict, input_tokens: int) -> tuple[int, int, int]:
"""Extract cache tokens, handling OpenAI vs Anthropic formats.
Returns: (cache_read_tokens, cache_creation_tokens, adjusted_input_tokens)
"""
cache_read = parse_token_count(usage_data.get("cache_read_input_tokens", 0))
cache_creation = parse_token_count(
usage_data.get("cache_creation_input_tokens", 0)
)
# OpenAI: cache is included in input_tokens, subtract it
prompt_details = usage_data.get("prompt_tokens_details")
if isinstance(prompt_details, dict) and not cache_read:
openai_cached = parse_token_count(prompt_details.get("cached_tokens", 0))
if openai_cached:
cache_read = openai_cached
input_tokens = max(0, input_tokens - cache_read)
return cache_read, cache_creation, input_tokens
def _resolve_usd_cost(usage_data: dict, response_data: dict) -> float:
"""Resolve USD cost with clear priority order.
@@ -454,10 +449,21 @@ def _calculate_from_tokens(
},
)
# Fold the cache-read/write cost into the visible ``input_msats`` so a
# dashboard that renders I / O / T sees ``input + output == total``
# exactly. This mirrors ``_fold_cache_into_input_tokens`` (which rolls the
# cache token counts into the visible prompt total). The standalone
# ``cache_read_msats`` / ``cache_creation_msats`` fields stay populated for
# clients that want the breakdown; nothing sums the components to derive
# ``total_msats`` (it is computed independently above), so this is
# display-only and does not change what is billed.
visible_output_msats = int(calc_output_msats)
visible_input_msats = token_based_cost - visible_output_msats
return CostData(
base_msats=0,
input_msats=int(calc_input_msats),
output_msats=int(calc_output_msats),
input_msats=visible_input_msats,
output_msats=visible_output_msats,
total_msats=token_based_cost,
total_usd=total_usd,
input_tokens=input_tokens,

View File

@@ -11,6 +11,8 @@ from PIL import Image
from sqlmodel.ext.asyncio.session import AsyncSession
from ..core import get_logger
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..core.settings import settings
from ..wallet import deserialize_token_from_string
@@ -418,16 +420,27 @@ def create_error_response(
status_code: int,
request: Request,
token: str | None = None,
code: str | int | None = None,
details: dict[str, object] | None = None,
) -> Response:
"""Create a standardized error response."""
"""Create a standardized error response.
``code`` is a stable, machine-readable classification (e.g.
``UPSTREAM_RATE_LIMIT``); when omitted it defaults to the HTTP status code
for backwards compatibility. ``details`` carries optional structured,
redaction-safe context.
"""
error_obj: dict[str, object] = {
"message": redact_org_ids(message),
"type": error_type,
"code": code if code is not None else status_code,
}
if details is not None:
error_obj["details"] = details
return Response(
content=json.dumps(
{
"error": {
"message": message,
"type": error_type,
"code": status_code,
},
"error": error_obj,
"request_id": getattr(request.state, "request_id", "unknown"),
}
),
@@ -435,3 +448,20 @@ def create_error_response(
media_type="application/json",
headers={"X-Cashu": token} if token else {},
)
def create_upstream_error_response(
error: UpstreamError,
request: Request,
fallback_status: int = 502,
) -> Response:
"""Build an error response from an :class:`UpstreamError`, preserving its
structured ``code``, ``details``, and original ``status_code``."""
return create_error_response(
"upstream_error",
str(error),
error.status_code or fallback_status,
request=request,
code=getattr(error, "code", None),
details=getattr(error, "details", None),
)

View File

@@ -3,7 +3,7 @@ import json
import random
import httpx
from fastapi import APIRouter, Depends
from fastapi import APIRouter, Depends, HTTPException, Request
from pydantic import BaseModel as V2BaseModel
from pydantic.v1 import BaseModel
from sqlmodel.ext.asyncio.session import AsyncSession
@@ -17,6 +17,24 @@ logger = get_logger(__name__)
models_router = APIRouter()
_MODEL_TEST_ENDPOINT_PATHS = {
"chat-completions": "chat/completions",
"completions": "completions",
"embeddings": "embeddings",
"responses": "responses",
}
# Cap the caller-supplied test payload to avoid forwarding oversized bodies
# upstream on the operator's credentials.
_MODEL_TEST_MAX_REQUEST_BYTES = 64 * 1024
async def _require_admin_api(request: Request) -> None:
"""Require admin auth without creating an import-time cycle with core.admin."""
from ..core.admin import require_admin_api
await require_admin_api(request)
class Architecture(BaseModel):
modality: str
@@ -67,6 +85,64 @@ class Model(BaseModel):
return hash(self.id)
def litellm_cost_entry(model_id: str) -> dict | None:
"""Look up ``model_id`` in litellm's bundled cost map.
litellm ships per-model USD rates keyed by the exact OpenRouter id
(``deepseek/deepseek-chat``) or the bare model name (``gpt-4o``,
``claude-sonnet-4-5``), so both spellings are tried. Keys are lowercase, so
a mixed-case upstream id (``deepseek-ai/DeepSeek-V4-Flash``) is retried via
a case-insensitive scan. Returns the matched cost dict, or ``None``.
"""
import litellm
candidates = (model_id, model_id.split("/", 1)[-1])
for key in candidates:
info = litellm.model_cost.get(key)
if isinstance(info, dict):
return info
lowered = {c.lower() for c in candidates}
for key, info in litellm.model_cost.items():
if isinstance(key, str) and key.lower() in lowered and isinstance(info, dict):
return info
return None
def backfill_cache_pricing(model_id: str, pricing: Pricing) -> Pricing:
"""Fill missing cache rates from litellm's bundled cost map.
The OpenRouter model feed omits ``input_cache_read``/``input_cache_write``
for many models (most DeepSeek entries, openai/gpt-4o, ...). Without a
cache rate, billing falls back to the full input rate, which overcharges
cache reads (DeepSeek hits are 10x cheaper) and undercharges Anthropic
cache writes (1.25x). The lookup (see ``litellm_cost_entry``) tries both
id spellings and a case-insensitive fallback.
Rates already present (e.g. provided by OpenRouter) are authoritative and
never overwritten. Unknown models are returned unchanged.
"""
needs_read = (pricing.input_cache_read or 0.0) <= 0.0
needs_write = (pricing.input_cache_write or 0.0) <= 0.0
if not (needs_read or needs_write):
return pricing
info = litellm_cost_entry(model_id)
if info is None:
return pricing
updated = Pricing.parse_obj(pricing.dict())
if needs_read:
read_rate = info.get("cache_read_input_token_cost")
if isinstance(read_rate, (int, float)) and read_rate > 0:
updated.input_cache_read = float(read_rate)
if needs_write:
write_rate = info.get("cache_creation_input_token_cost")
if isinstance(write_rate, (int, float)) and write_rate > 0:
updated.input_cache_write = float(write_rate)
return updated
def _has_valid_pricing(model: dict) -> bool:
"""Check if model has valid pricing (not free, no negative values)."""
pricing = model.get("pricing", {})
@@ -155,13 +231,29 @@ def _row_to_model(
)
top_provider_dict = json.loads(row.top_provider) if row.top_provider else None
if apply_provider_fee and isinstance(pricing, dict):
pricing = {k: float(v) * provider_fee for k, v in pricing.items()}
if isinstance(pricing, dict) and float(pricing.get("request", 0.0)) <= 0.0:
pricing["request"] = max(pricing.get("request", 0.0), 0.0)
parsed_pricing = Pricing.parse_obj(pricing)
# Fill missing cache-read/write rates from litellm's cost map BEFORE applying
# the provider fee, so they carry the same markup as every other component.
# DB-stored override pricing (e.g. generic providers) omits cache rates;
# without this, ``_row_to_model`` bills cache reads at the full input rate —
# the ``_apply_provider_fee_to_model`` path backfills, but the override path
# used for admin-configured providers did not.
#
# Key on ``forwarded_model_id`` (the actual upstream model name litellm
# prices) when set: an alias row (id="local-alias",
# forwarded_model_id="deepseek-v4-flash") would otherwise look up the alias
# and miss the cache rate.
pricing_model_id = getattr(row, "forwarded_model_id", None) or row.id
parsed_pricing = backfill_cache_pricing(pricing_model_id, parsed_pricing)
if apply_provider_fee:
parsed_pricing = Pricing.parse_obj(
{k: float(v) * provider_fee for k, v in parsed_pricing.dict().items()}
)
model = Model(
id=row.id,
name=row.name,
@@ -418,7 +510,9 @@ class ModelTestRequest(V2BaseModel):
request_data: dict
@models_router.post("/api/models/test")
@models_router.post(
"/api/models/test", dependencies=[Depends(_require_admin_api)]
)
async def test_model(
payload: ModelTestRequest,
session: AsyncSession = Depends(get_session),
@@ -446,16 +540,35 @@ async def test_model(
"status_code": 404,
}
base_url = provider.base_url.rstrip("/")
if payload.endpoint_type == "chat-completions":
url = f"{base_url}/chat/completions"
else:
url = f"{base_url}/{payload.endpoint_type}"
endpoint_path = _MODEL_TEST_ENDPOINT_PATHS.get(payload.endpoint_type)
if endpoint_path is None:
raise HTTPException(status_code=400, detail="Unsupported endpoint_type")
actual_model_id = model_row.forwarded_model_id or model_row.id
request_data = dict(payload.request_data)
request_data["model"] = actual_model_id
try:
request_size = len(json.dumps(request_data).encode("utf-8"))
except (TypeError, ValueError):
raise HTTPException(status_code=400, detail="Invalid request_data")
if request_size > _MODEL_TEST_MAX_REQUEST_BYTES:
raise HTTPException(status_code=413, detail="request_data too large")
base_url = provider.base_url.rstrip("/")
url = f"{base_url}/{endpoint_path}"
logger.info(
"admin model test",
extra={
"model_id": payload.model_id,
"forwarded_model_id": actual_model_id,
"endpoint_type": payload.endpoint_type,
"upstream_provider_id": model_row.upstream_provider_id,
"request_bytes": request_size,
},
)
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {provider.api_key}",

131
routstr/payment/usage.py Normal file
View File

@@ -0,0 +1,131 @@
"""Vendor-agnostic normalization of upstream usage objects.
Upstream providers report token usage in vendor dialects that differ in field
names and in whether cached tokens are included in the input count:
* OpenAI / Azure / xAI / Groq / Moonshot / Qwen / Gemini-compat: cache reads in
``prompt_tokens_details.cached_tokens``, included in ``prompt_tokens``.
* OpenRouter: same as OpenAI plus cache *writes* in
``prompt_tokens_details.cache_write_tokens``, also included in
``prompt_tokens``.
* litellm-normalized: same nesting, but names the write field
``prompt_tokens_details.cache_creation_tokens`` (and additionally mirrors the
Anthropic top-level fields), with ``prompt_tokens`` as the grand total.
* Anthropic native: ``cache_read_input_tokens`` / ``cache_creation_input_tokens``
top-level, additive to (not included in) ``input_tokens``.
* DeepSeek: ``prompt_cache_hit_tokens`` / ``prompt_cache_miss_tokens``, with
``prompt_tokens = hit + miss``.
What decides whether cached tokens must be subtracted out of the input count is
**which prompt field the vendor uses**, not which cache field appears:
* ``prompt_tokens`` present -> cached + cache-write tokens are *included* in it
(OpenAI family, DeepSeek, OpenRouter, litellm); subtract both so
``input_tokens`` holds only the regular-rate portion.
* only ``input_tokens`` (Anthropic native) -> cached tokens are *additive*;
leave ``input_tokens`` untouched.
``normalize_usage`` maps all of them onto one canonical ``NormalizedUsage``
shape so billing code needs no vendor knowledge. The known dialects' field
names do not collide, so a single union parser is safe; a vendor whose fields
would genuinely conflict needs a dedicated branch here.
"""
from pydantic.v1 import BaseModel
class NormalizedUsage(BaseModel):
"""Canonical token usage: input_tokens never includes cached tokens."""
input_tokens: int = 0
output_tokens: int = 0
cache_read_tokens: int = 0
cache_write_tokens: int = 0
def parse_token_count(value: object) -> int:
"""Parse a token count from various formats (int, float, str, bool)."""
if isinstance(value, bool):
return 0
if isinstance(value, int):
return max(0, value)
if isinstance(value, float):
return max(0, int(value))
if isinstance(value, str):
try:
return max(0, int(float(value)))
except ValueError:
return 0
return 0
def _first_token_count(usage_data: dict, *fields: str) -> int:
"""Return the first positive token count among the given fields."""
for field in fields:
value = parse_token_count(usage_data.get(field, 0))
if value > 0:
return value
return 0
def _extract_cache_tokens(usage_data: dict) -> tuple[int, int]:
"""Pull (cache_read, cache_write) across all known dialects.
Precedence (highest first), independent for reads and writes:
* Anthropic top-level: ``cache_read_input_tokens`` /
``cache_creation_input_tokens``.
* Nested ``prompt_tokens_details``: ``cached_tokens`` for reads;
``cache_creation_tokens`` (litellm) or ``cache_write_tokens``
(OpenRouter) for writes.
* DeepSeek: ``prompt_cache_hit_tokens`` for reads (no write concept).
"""
cache_read = parse_token_count(usage_data.get("cache_read_input_tokens", 0))
cache_write = parse_token_count(usage_data.get("cache_creation_input_tokens", 0))
prompt_details = usage_data.get("prompt_tokens_details")
if isinstance(prompt_details, dict):
if not cache_read:
cache_read = parse_token_count(prompt_details.get("cached_tokens", 0))
if not cache_write:
cache_write = _first_token_count(
prompt_details, "cache_creation_tokens", "cache_write_tokens"
)
if not cache_read:
# DeepSeek: prompt_tokens = prompt_cache_hit_tokens + prompt_cache_miss_tokens
cache_read = parse_token_count(usage_data.get("prompt_cache_hit_tokens", 0))
return cache_read, cache_write
def normalize_usage(usage_data: object) -> NormalizedUsage | None:
"""Map a vendor usage dict onto the canonical shape, or None if absent.
Cached reads and writes are subtracted from the input count exactly once,
only for dialects that report a ``prompt_tokens`` grand total that already
includes them (OpenAI family, DeepSeek, OpenRouter, litellm). Anthropic
native reports them additively under ``input_tokens`` and is left untouched.
"""
if not isinstance(usage_data, dict):
return None
output_tokens = _first_token_count(
usage_data, "completion_tokens", "output_tokens"
)
cache_read, cache_write = _extract_cache_tokens(usage_data)
# ``prompt_tokens`` is the inclusive grand total; ``input_tokens`` (Anthropic
# native) excludes cached tokens. The field chosen decides whether to subtract.
if "prompt_tokens" in usage_data:
input_tokens = parse_token_count(usage_data.get("prompt_tokens", 0))
input_tokens = max(0, input_tokens - cache_read - cache_write)
else:
input_tokens = parse_token_count(usage_data.get("input_tokens", 0))
return NormalizedUsage(
input_tokens=input_tokens,
output_tokens=output_tokens,
cache_read_tokens=cache_read,
cache_write_tokens=cache_write,
)

View File

@@ -1,3 +1,4 @@
import asyncio
import json
from typing import Any
@@ -23,11 +24,13 @@ from .payment.helpers import (
calculate_discounted_max_cost,
check_token_balance,
create_error_response,
create_upstream_error_response,
get_max_cost_for_model,
)
from .payment.models import Model
from .upstream import BaseUpstreamProvider
from .upstream.helpers import init_upstreams
from .upstream.request_correction import correct_request, extract_error_message
logger = get_logger(__name__)
proxy_router = APIRouter()
@@ -209,9 +212,7 @@ async def proxy(
e,
)
if i == len(all_upstreams) - 1:
last_error_response = create_error_response(
"upstream_error", str(e), 502, request=request
)
last_error_response = create_upstream_error_response(e, request)
continue
return last_error_response or create_error_response(
"upstream_error", "All upstreams failed", 502, request=request
@@ -281,11 +282,10 @@ async def proxy(
last_error = e
continue
if last_error is not None:
return create_upstream_error_response(last_error, request)
return create_error_response(
"upstream_error",
str(last_error) if last_error else "All upstreams failed",
502,
request=request,
"upstream_error", "All upstreams failed", 502, request=request
)
elif auth := headers.get("authorization", None):
@@ -341,9 +341,7 @@ async def proxy(
except UpstreamError as e:
logger.warning(f"Upstream {upstream.provider_type} failed (GET): {e}")
if i == len(upstreams) - 1:
last_error_response = create_error_response(
"upstream_error", str(e), 502, request=request
)
last_error_response = create_upstream_error_response(e, request)
continue
return last_error_response or create_error_response(
"upstream_error", "All upstreams failed", 502, request=request
@@ -352,50 +350,86 @@ async def proxy(
if request_body_dict:
await pay_for_request(key, max_cost_for_model, session)
# Tracks request params already removed in response to upstream rejections,
# shared across providers so a stripped param stays stripped on failover and
# the reactive retry can never loop unboundedly.
already_stripped: set[str] = set()
for i, upstream in enumerate(upstreams):
headers = upstream.prepare_headers(dict(request.headers))
try:
try:
if is_responses_api:
response = await upstream.forward_responses_request(
request,
path,
headers,
request_body,
key,
max_cost_for_model,
session,
model_obj,
while True:
try:
if is_responses_api:
response = await upstream.forward_responses_request(
request,
path,
headers,
request_body,
key,
max_cost_for_model,
session,
model_obj,
)
else:
response = await upstream.forward_request(
request,
path,
headers,
request_body,
key,
max_cost_for_model,
session,
model_obj,
)
except UpstreamError:
# Let the outer UpstreamError handler manage retry/revert
raise
except Exception as e:
# Unexpected error (not an upstream failure) — revert and propagate
logger.error(
"Unexpected error in upstream request, reverting payment",
extra={
"error": str(e),
"error_type": type(e).__name__,
"path": path,
"key_hash": key.hashed_key[:8] + "...",
"max_cost_for_model": max_cost_for_model,
},
)
else:
response = await upstream.forward_request(
request,
path,
headers,
await revert_pay_for_request(key, session, max_cost_for_model)
raise
# Reactive recovery: some models reject one specific request
# param (e.g. newer Anthropic models deprecating `temperature`).
# When the upstream 400s naming such a param, strip it from the
# body and retry the SAME upstream. ``already_stripped`` bounds
# this to one retry per distinct param so it always terminates.
if response.status_code == 400:
correction = correct_request(
request_body,
key,
max_cost_for_model,
session,
model_obj,
extract_error_message(response),
already_stripped,
)
except UpstreamError:
# Let the outer UpstreamError handler manage retry/revert
raise
except Exception as e:
# Unexpected error (not an upstream failure) — revert and propagate
logger.error(
"Unexpected error in upstream request, reverting payment",
extra={
"error": str(e),
"error_type": type(e).__name__,
"path": path,
"key_hash": key.hashed_key[:8] + "...",
"max_cost_for_model": max_cost_for_model,
},
)
await revert_pay_for_request(key, session, max_cost_for_model)
raise
if correction is not None:
request_body, bad_param = correction.body, correction.label
already_stripped.add(bad_param)
logger.warning(
"Upstream %s rejected param '%s' for model=%s; "
"stripping and retrying same upstream",
upstream.provider_type,
bad_param,
model_id,
extra={
"provider": upstream.provider_type,
"model": model_id,
"stripped_param": bad_param,
"path": path,
},
)
continue
break
if response.status_code != 200:
# Check if we should retry (502 Upstream Error or 429 Rate Limit)
@@ -455,6 +489,21 @@ async def proxy(
return response
except asyncio.CancelledError:
logger.warning(
"Client disconnected mid-request, reverting reservation",
extra={
"path": path,
"model": model_id,
"key_hash": key.hashed_key[:8] + "...",
"max_cost_for_model": max_cost_for_model,
},
)
await asyncio.shield(
revert_pay_for_request(key, session, max_cost_for_model)
)
raise
except UpstreamError as e:
logger.warning(
"Upstream %s failed for model=%s: %s",
@@ -472,9 +521,7 @@ async def proxy(
# If this was the last provider
if i == len(upstreams) - 1:
await revert_pay_for_request(key, session, max_cost_for_model)
return create_error_response(
"upstream_error", str(e), 502, request=request
)
return create_upstream_error_response(e, request)
# Otherwise loop continues to next provider
continue

View File

@@ -24,7 +24,7 @@ class AnthropicUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "AnthropicUpstreamProvider":
return cls(

View File

@@ -97,6 +97,8 @@ async def _check_and_topup(row: UpstreamProviderRow) -> None:
# Instantiate provider and check balance
provider = RoutstrUpstreamProvider.from_db_row(row)
if provider is None:
return
balance = await provider.get_balance()
if balance is None:

View File

@@ -38,7 +38,7 @@ class AzureUpstreamProvider(BaseUpstreamProvider):
self.api_version = api_version
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "AzureUpstreamProvider | None":
if not provider_row.api_version:

View File

@@ -2,16 +2,16 @@ from __future__ import annotations
import asyncio
import json
import math
import traceback
import uuid
from collections.abc import AsyncGenerator, AsyncIterator, Iterator
from typing import Any, Mapping, cast
from typing import Any, Mapping, Self, cast
import httpx
from fastapi import BackgroundTasks, HTTPException, Request
from fastapi.responses import Response, StreamingResponse
from pydantic.v1 import BaseModel
from sqlmodel import select
from ..auth import adjust_payment_for_tokens
from ..core import get_logger
@@ -23,6 +23,7 @@ from ..core.db import (
store_cashu_transaction,
)
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..payment.cost_calculation import (
CostData,
CostDataError,
@@ -35,13 +36,24 @@ from ..payment.models import (
Pricing,
_calculate_usd_max_costs,
_update_model_sats_pricing,
backfill_cache_pricing,
list_models,
)
from ..payment.price import sats_usd_price
from ..wallet import recieve_token, send_token
from ..wallet import (
SPENT_TOKEN_CODES,
classify_redemption_error,
recieve_token,
send_token,
)
from . import messages_dispatch
from .cache_breakpoints import (
inject_anthropic_cache_breakpoints,
is_explicit_cache_model,
)
from .count_tokens import count_tokens_locally
from .litellm_routing import detect_litellm_prefix
from .rate_limit import UPSTREAM_RATE_LIMIT, classify_rate_limit
logger = get_logger(__name__)
@@ -84,6 +96,11 @@ class BaseUpstreamProvider:
base_url: str
api_key: str
provider_fee: float = 1.05
# Primary key of the ``upstream_providers`` row this instance was built
# from. Set by ``from_db_row`` so a live provider can re-find its own row by
# stable identity instead of its rotatable ``api_key``. ``None`` for
# instances not sourced from a row.
db_id: int | None = None
_models_cache: list[Model] = []
_models_by_id: dict[str, Model] = {}
@@ -98,6 +115,7 @@ class BaseUpstreamProvider:
self.base_url = base_url
self.api_key = api_key
self.provider_fee = provider_fee
self.db_id = None
self._models_cache = []
self._models_by_id = {}
@@ -115,10 +133,13 @@ class BaseUpstreamProvider:
return detect_litellm_prefix(self.base_url)
@classmethod
def from_db_row(
cls, provider_row: "UpstreamProviderRow"
) -> "BaseUpstreamProvider | None":
"""Factory method to instantiate provider from database row.
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "Self | None":
"""Instantiate a provider from a database row, carrying its identity.
Construction itself is delegated to the ``_build_from_row`` hook (which
subclasses override to match their constructor); this wrapper stamps the
row's primary key onto the instance as ``db_id`` so the provider can
later re-find its own row by identity rather than by its ``api_key``.
Args:
provider_row: Database row containing provider configuration
@@ -126,6 +147,19 @@ class BaseUpstreamProvider:
Returns:
Instantiated provider or None if instantiation fails
"""
provider = cls._build_from_row(provider_row)
if provider is not None:
provider.db_id = provider_row.id
return provider
@classmethod
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "Self | None":
"""Construct the provider instance from a row (no identity stamping).
Overridden by subclasses whose constructors differ from the base
``(base_url, api_key, provider_fee)`` shape. Callers should use
``from_db_row`` instead, which also attaches ``db_id``.
"""
return cls(
base_url=provider_row.base_url,
api_key=provider_row.api_key,
@@ -432,6 +466,19 @@ class BaseUpstreamProvider:
return body
def _upstream_accepts_cache_control(self) -> bool:
"""True when this upstream accepts explicit ``cache_control`` markers.
Only OpenRouter (documents Anthropic + Alibaba explicit caching) and the
native Anthropic API accept the markers. Stamping them toward an
automatic-cache or non-supporting upstream risks a 400, so injection is
confined to these. Base URL is also checked so an OpenRouter endpoint
configured through the generic provider is still recognised.
"""
if self.provider_type in ("openrouter", "anthropic"):
return True
return "openrouter.ai" in (self.base_url or "")
def prepare_request_body(
self, body: bytes | None, model_obj: Model
) -> bytes | None:
@@ -501,6 +548,27 @@ class BaseUpstreamProvider:
data["stream_options"] = merged
changed = True
# Explicit-cache models (Anthropic Claude, Alibaba Qwen / deepseek-v3.2)
# cache nothing without ``cache_control`` markers in the body. Clients
# that don't recognise a routstr URL as one of these never send them, so
# caching silently never engages over routstr even though it works
# against OpenRouter directly. Stamp the standard breakpoints so caching
# works by default, deferring to any client-set markers. Gated to
# upstreams that accept the markers (OpenRouter / Anthropic) so they
# never leak to an automatic-cache provider that would reject them.
if (
"messages" in data
and isinstance(data.get("messages"), list)
and self._upstream_accepts_cache_control()
and is_explicit_cache_model(
model_obj.id,
model_obj.forwarded_model_id,
model_obj.canonical_slug,
)
):
if inject_anthropic_cache_breakpoints(data):
changed = True
if changed:
return json.dumps(data).encode()
return body
@@ -543,7 +611,7 @@ class BaseUpstreamProvider:
preview = body_bytes.decode("utf-8", errors="ignore").strip()
if preview:
message = preview[:500]
return message, upstream_code
return redact_org_ids(message), upstream_code
async def on_upstream_error_redirect(
self, status_code: int, error_message: str
@@ -586,10 +654,23 @@ class BaseUpstreamProvider:
body_bytes = b""
body_read_error = f"{type(exc).__name__}: {exc}"
# ``message`` is already redacted by ``_extract_upstream_error_message``;
# the raw body preview is redacted here before it reaches logs or the
# forwarded envelope so provider account identifiers never leak.
message, upstream_code = self._extract_upstream_error_message(body_bytes)
body_preview = body_bytes.decode("utf-8", errors="ignore").strip()[:500]
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
is_json_body = _is_json_content_type(content_type)
# Classify upstream rate-limit failures into a stable, structured error.
rate_limit = classify_rate_limit(status_code, message, headers)
error_code: str | int = upstream_code or status_code
error_details: dict[str, object] | None = None
if rate_limit is not None:
error_code = UPSTREAM_RATE_LIMIT
error_details = rate_limit.as_details()
logger.warning(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -603,6 +684,7 @@ class BaseUpstreamProvider:
"model": model_id or "unknown",
"upstream_status": status_code,
"upstream_code": upstream_code,
"error_code": error_code,
"upstream_content_type": content_type,
"upstream_request_id": upstream_request_id,
"message_preview": message[:200],
@@ -637,13 +719,41 @@ class BaseUpstreamProvider:
):
headers.pop(header_name, None)
# Propagate a usable retry hint to the caller when the upstream supplied
# one but did not echo a ``Retry-After`` header. RFC 7231 delta-seconds
# is an integer, so round sub-second hints up to a usable ``1``.
if (
rate_limit is not None
and rate_limit.retry_after_seconds is not None
and "retry-after" not in {k.lower() for k in headers}
):
headers["Retry-After"] = str(max(1, math.ceil(rate_limit.retry_after_seconds)))
if is_json_body:
if not content_type:
headers.pop("content-type", None)
headers.pop("Content-Type", None)
media_type = content_type or None
# Re-serialise the body with organization IDs stripped. The narrow
# ``org-*`` regex preserves the surrounding JSON structure.
redacted_text = redact_org_ids(body_bytes.decode("utf-8", errors="ignore"))
redacted_body = redacted_text.encode()
# Surface the stable rate-limit classification on the forwarded
# body so callers can switch on ``error.code`` without parsing the
# provider-specific message. Fall back to the redacted bytes if the
# body is not a JSON object with an ``error`` mapping.
if rate_limit is not None:
try:
parsed = json.loads(redacted_text)
err = parsed.get("error") if isinstance(parsed, dict) else None
if isinstance(err, dict):
err["code"] = UPSTREAM_RATE_LIMIT
err["details"] = error_details
redacted_body = json.dumps(parsed).encode()
except (ValueError, AttributeError):
pass
return Response(
content=body_bytes,
content=redacted_body,
status_code=status_code,
headers=headers,
media_type=media_type,
@@ -654,15 +764,18 @@ class BaseUpstreamProvider:
for header_name in ("content-type", "Content-Type"):
headers.pop(header_name, None)
error_obj: dict[str, object] = {
"message": message or "Upstream returned a non-JSON error response",
"type": "upstream_error",
"code": error_code,
"upstream_status": status_code,
"upstream_content_type": content_type or None,
"upstream_body_preview": body_preview or None,
}
if error_details is not None:
error_obj["details"] = error_details
envelope = {
"error": {
"message": message or "Upstream returned a non-JSON error response",
"type": "upstream_error",
"code": upstream_code or status_code,
"upstream_status": status_code,
"upstream_content_type": content_type or None,
"upstream_body_preview": body_preview or None,
},
"error": error_obj,
"request_id": getattr(request.state, "request_id", None),
}
@@ -727,7 +840,9 @@ class BaseUpstreamProvider:
except Exception:
pass
def _process_event(raw_event: bytes) -> Iterator[bytes]:
def _process_event(
raw_event: bytes, final: bool = False
) -> Iterator[bytes]:
"""Process one complete SSE event block (lines up to a blank line).
Handles arbitrary upstream framing across every supported
@@ -833,6 +948,12 @@ class BaseUpstreamProvider:
return
yield prefix + b"data: " + json.dumps(obj).encode() + b"\n\n"
else:
if final:
# Final flush of a truncated tail: the upstream closed
# mid-event, so ``data`` is incomplete JSON. Emitting it
# as a ``data:`` frame would hand the client invalid
# JSON (the "unexpected token" parse error). Drop it.
return
# Non-JSON data payload (partial fragment already reassembled
# by buffering, or a provider control string). Re-prefix each
# line so multi-line ``data`` stays valid SSE framing - a bare
@@ -851,7 +972,13 @@ class BaseUpstreamProvider:
# boundary-independent for every provider.
buffer = b""
async for chunk in response.aiter_bytes():
buffer += chunk.replace(b"\r\n", b"\n")
# Normalize the *joined* buffer, not each chunk in
# isolation: a CRLF event delimiter can straddle two
# ``aiter_bytes`` chunks (``...\r`` then ``\n...``). A
# per-chunk replace would leave a stray ``\r`` and the
# ``\n\n`` split would miss the delimiter, merging two
# events into one frame and breaking SSE clients.
buffer = (buffer + chunk).replace(b"\r\n", b"\n")
while b"\n\n" in buffer:
raw_event, buffer = buffer.split(b"\n\n", 1)
for out in _process_event(raw_event):
@@ -859,7 +986,7 @@ class BaseUpstreamProvider:
# Flush any trailing event that lacked a final blank line.
if buffer.strip():
for out in _process_event(buffer):
for out in _process_event(buffer, final=True):
yield out
async with create_session() as session:
@@ -1018,7 +1145,10 @@ class BaseUpstreamProvider:
response_json["id"] = f"chatcmpl-{uuid.uuid4()}"
cost_data = await adjust_payment_for_tokens(
key, response_json, session, deducted_max_cost
key,
response_json,
session,
deducted_max_cost,
)
await session.refresh(key)
@@ -1162,7 +1292,9 @@ class BaseUpstreamProvider:
except Exception:
pass
def _process_event(raw_event: bytes) -> Iterator[bytes]:
def _process_event(
raw_event: bytes, final: bool = False
) -> Iterator[bytes]:
"""Process one complete SSE event block for the Responses API.
Buffers full events (delimited by a blank line) so parsing is
@@ -1232,6 +1364,11 @@ class BaseUpstreamProvider:
yield prefix + b"data: " + json.dumps(obj).encode() + b"\n\n"
else:
if final:
# Final flush of a truncated tail: upstream closed
# mid-event, so ``data`` is incomplete JSON. Dropping it
# avoids handing the client an invalid ``data:`` frame.
return
# Re-prefix each line so multi-line ``data`` stays valid SSE
# framing for the client.
body = b"".join(
@@ -1244,14 +1381,20 @@ class BaseUpstreamProvider:
# delimiter so parsing is independent of byte boundaries.
buffer = b""
async for chunk in response.aiter_bytes():
buffer += chunk.replace(b"\r\n", b"\n")
# Normalize the *joined* buffer, not each chunk in
# isolation: a CRLF event delimiter can straddle two
# ``aiter_bytes`` chunks (``...\r`` then ``\n...``). A
# per-chunk replace would leave a stray ``\r`` and the
# ``\n\n`` split would miss the delimiter, merging two
# events into one frame and breaking SSE clients.
buffer = (buffer + chunk).replace(b"\r\n", b"\n")
while b"\n\n" in buffer:
raw_event, buffer = buffer.split(b"\n\n", 1)
for out in _process_event(raw_event):
yield out
if buffer.strip():
for out in _process_event(buffer):
for out in _process_event(buffer, final=True):
yield out
# Always emit a cost-bearing data chunk
@@ -1436,7 +1579,10 @@ class BaseUpstreamProvider:
response_json["id"] = f"chatcmpl-{uuid.uuid4()}"
cost_data = await adjust_payment_for_tokens(
key, response_json, session, deducted_max_cost
key,
response_json,
session,
deducted_max_cost,
)
await session.refresh(key)
@@ -1631,7 +1777,10 @@ class BaseUpstreamProvider:
"usage": None,
}
cost_data = await adjust_payment_for_tokens(
fresh_key, fallback, new_session, max_cost_for_model
fresh_key,
fallback,
new_session,
max_cost_for_model,
)
usage_finalized = True
return f"event: cost\ndata: {json.dumps({'cost': cost_data})}\n\n".encode()
@@ -1850,7 +1999,10 @@ class BaseUpstreamProvider:
response_json["usage"] = {"input_tokens": input_tokens}
cost_data = await adjust_payment_for_tokens(
key, response_json, session, deducted_max_cost
key,
response_json,
session,
deducted_max_cost,
)
self.inject_cost_metadata(response_json, cost_data, key)
@@ -1952,7 +2104,10 @@ class BaseUpstreamProvider:
response_json["model"] = requested_model
cost_data = await adjust_payment_for_tokens(
key, response_json, session, max_cost_for_model
key,
response_json,
session,
max_cost_for_model,
)
self.inject_cost_metadata(response_json, cost_data, key)
@@ -2440,9 +2595,16 @@ class BaseUpstreamProvider:
body_bytes = await response.aread()
except Exception:
body_bytes = b""
body_preview = body_bytes.decode(
"utf-8", errors="ignore"
).strip()[:500]
# Redact provider account identifiers before the body text
# reaches logs or the raised error.
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
rate_limit = classify_rate_limit(
response.status_code,
body_preview,
dict(response.headers),
)
logger.error(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -2454,6 +2616,7 @@ class BaseUpstreamProvider:
"provider": self.provider_type,
"model": original_model_id or "unknown",
"status_code": response.status_code,
"error_code": rate_limit.code if rate_limit else None,
"reason_phrase": response.reason_phrase,
"path": path,
"body_preview": body_preview,
@@ -2466,6 +2629,8 @@ class BaseUpstreamProvider:
f"for model {original_model_id or 'unknown'}: "
f"{body_preview[:200] or '<empty>'}",
status_code=response.status_code,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
)
try:
@@ -2762,9 +2927,16 @@ class BaseUpstreamProvider:
body_bytes = await response.aread()
except Exception:
body_bytes = b""
body_preview = body_bytes.decode(
"utf-8", errors="ignore"
).strip()[:500]
# Redact provider account identifiers before the body text
# reaches logs or the raised error.
body_preview = redact_org_ids(
body_bytes.decode("utf-8", errors="ignore").strip()[:500]
)
rate_limit = classify_rate_limit(
response.status_code,
body_preview,
dict(response.headers),
)
logger.error(
"Upstream %s returned %s for model=%s path=%s: %s",
self.provider_type,
@@ -2776,6 +2948,7 @@ class BaseUpstreamProvider:
"provider": self.provider_type,
"model": original_model_id or "unknown",
"status_code": response.status_code,
"error_code": rate_limit.code if rate_limit else None,
"path": path,
"body_preview": body_preview,
},
@@ -2787,6 +2960,8 @@ class BaseUpstreamProvider:
f"for model {original_model_id or 'unknown'}: "
f"{body_preview[:200] or '<empty>'}",
status_code=response.status_code,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
)
try:
@@ -3025,44 +3200,46 @@ class BaseUpstreamProvider:
extra={"model": model, "has_usage": "usage" in response_data},
)
async with create_session() as session:
match await calculate_cost(response_data, max_cost_for_model, session):
case MaxCostData() as cost:
logger.debug(
"Using max cost pricing",
extra={"model": model, "max_cost_msats": cost.total_msats},
)
return cost
case CostData() as cost:
logger.debug(
"Using token-based pricing",
extra={
"model": model,
"total_cost_msats": cost.total_msats,
"input_msats": cost.input_msats,
"output_msats": cost.output_msats,
},
)
return cost
case CostDataError() as error:
logger.error(
"Cost calculation error",
extra={
"model": model,
"error_message": error.message,
"error_code": error.code,
},
)
raise HTTPException(
status_code=400,
detail={
"error": {
"message": error.message,
"type": "invalid_request_error",
"code": error.code,
}
},
)
match await calculate_cost(
response_data,
max_cost_for_model,
):
case MaxCostData() as cost:
logger.debug(
"Using max cost pricing",
extra={"model": model, "max_cost_msats": cost.total_msats},
)
return cost
case CostData() as cost:
logger.debug(
"Using token-based pricing",
extra={
"model": model,
"total_cost_msats": cost.total_msats,
"input_msats": cost.input_msats,
"output_msats": cost.output_msats,
},
)
return cost
case CostDataError() as error:
logger.error(
"Cost calculation error",
extra={
"model": model,
"error_message": error.message,
"error_code": error.code,
},
)
raise HTTPException(
status_code=400,
detail={
"error": {
"message": error.message,
"type": "invalid_request_error",
"code": error.code,
}
},
)
return None
async def send_refund(
@@ -3809,9 +3986,19 @@ class BaseUpstreamProvider:
},
)
redeemed = False
try:
headers = dict(request.headers)
amount, unit, mint = await recieve_token(x_cashu_token)
# Reject a zero/negative redemption (empty/dust token, or a value
# fully consumed by fees) before marking the token redeemed, so it
# classifies as cashu_token_zero_value like the bearer/top-up paths
# rather than being forwarded as a free request.
if amount <= 0:
raise ValueError(
f"Redeemed token amount must be positive, got {amount} {unit}"
)
redeemed = True
headers = self.prepare_headers(dict(request.headers))
request_id = getattr(request.state, "request_id", None)
@@ -3855,40 +4042,37 @@ class BaseUpstreamProvider:
},
)
# Use same error handling as regular X-Cashu
if "already spent" in error_message.lower():
# Post-redemption the token is spent; a forwarding failure must not
# be reported as a retryable redemption error (see handle_x_cashu).
if redeemed:
return create_error_response(
"token_already_spent",
"The provided CASHU token has already been spent",
400,
"upstream_error",
"Payment succeeded but the upstream request failed",
502,
request=request,
token=x_cashu_token,
code="upstream_request_failed",
)
if "invalid token" in error_message.lower():
classified = classify_redemption_error(e)
if classified is None:
return create_error_response(
"invalid_token",
"The provided CASHU token is invalid",
400,
"api_error",
"Internal error during token redemption",
500,
request=request,
token=x_cashu_token,
code="internal_error",
)
if "mint error" in error_message.lower():
return create_error_response(
"mint_error",
f"CASHU mint error: {error_message}",
422,
request=request,
token=x_cashu_token,
)
error_type, status_code, message, error_code = classified
# Echo the token back only when it is still spendable, so clients
# can recover it; a spent/consumed token is never re-offered.
echo_token = None if error_code in SPENT_TOKEN_CODES else x_cashu_token
return create_error_response(
"cashu_error",
f"CASHU token processing failed: {error_message}",
400,
error_type,
message,
status_code,
request=request,
token=x_cashu_token,
token=echo_token,
code=error_code,
)
async def forward_x_cashu_responses_request(
@@ -4478,9 +4662,19 @@ class BaseUpstreamProvider:
},
)
redeemed = False
try:
headers = dict(request.headers)
amount, unit, mint = await recieve_token(x_cashu_token)
# Reject a zero/negative redemption (empty/dust token, or a value
# fully consumed by fees) before marking the token redeemed, so it
# classifies as cashu_token_zero_value like the bearer/top-up paths
# rather than being forwarded as a free request.
if amount <= 0:
raise ValueError(
f"Redeemed token amount must be positive, got {amount} {unit}"
)
redeemed = True
headers = self.prepare_headers(dict(request.headers))
request_id = getattr(request.state, "request_id", None)
@@ -4524,52 +4718,56 @@ class BaseUpstreamProvider:
},
)
if "already spent" in error_message.lower():
# Once redeemed the token is spent, so a later forwarding failure
# must not surface as a retryable mint_unreachable (spent-token retry
# bait). Redemption classification only applies while not redeemed.
if redeemed:
return create_error_response(
"token_already_spent",
"The provided CASHU token has already been spent",
400,
"upstream_error",
"Payment succeeded but the upstream request failed",
502,
request=request,
token=x_cashu_token,
code="upstream_request_failed",
)
if "invalid token" in error_message.lower():
classified = classify_redemption_error(e)
if classified is None:
return create_error_response(
"invalid_token",
"The provided CASHU token is invalid",
400,
"api_error",
"Internal error during token redemption",
500,
request=request,
token=x_cashu_token,
code="internal_error",
)
if "mint error" in error_message.lower():
return create_error_response(
"mint_error",
f"CASHU mint error: {error_message}",
422,
request=request,
token=x_cashu_token,
)
error_type, status_code, message, error_code = classified
# Echo the token back only when it is still spendable, so clients
# can recover it; a spent/consumed token is never re-offered.
echo_token = None if error_code in SPENT_TOKEN_CODES else x_cashu_token
return create_error_response(
"cashu_error",
f"CASHU token processing failed: {error_message}",
400,
error_type,
message,
status_code,
request=request,
token=x_cashu_token,
token=echo_token,
code=error_code,
)
def _apply_provider_fee_to_model(self, model: Model) -> Model:
"""Apply provider fee to model's USD pricing and calculate max costs.
Cache rates missing from the upstream pricing feed are backfilled from
litellm's cost map first, so they carry the provider fee like every
other price component.
Args:
model: Model object to update
Returns:
Model with provider fee applied to pricing and max costs calculated
"""
base_pricing = backfill_cache_pricing(model.id, model.pricing)
adjusted_pricing = Pricing.parse_obj(
{k: v * self.provider_fee for k, v in model.pricing.dict().items()}
{k: v * self.provider_fee for k, v in base_pricing.dict().items()}
)
temp_model = Model(
@@ -4722,14 +4920,11 @@ class BaseUpstreamProvider:
"""Refresh the in-memory models cache from upstream API."""
try:
async with create_session() as session:
stmt = select(UpstreamProviderRow).where(
UpstreamProviderRow.base_url == self.base_url,
UpstreamProviderRow.api_key == self.api_key,
provider = (
await session.get(UpstreamProviderRow, self.db_id)
if self.db_id is not None
else None
)
result = await session.exec(stmt)
# .first() returns the object or None if not found
provider = result.first()
if not provider or not provider.id:
raise HTTPException(status_code=404, detail="Provider not found")

View File

@@ -0,0 +1,157 @@
"""Inject explicit prompt-cache breakpoints into OpenAI-shaped requests.
Some upstreams cache *explicitly*: the request must carry
``cache_control: {"type": "ephemeral"}`` markers on the content blocks that
should be cached. Two model families use this identical wire format:
* **Anthropic Claude** — direct or via OpenRouter's ``anthropic/*`` models.
* **Alibaba's explicit-cache models on OpenRouter** — ``qwen/qwen3-max``,
``qwen/qwen-plus``, ``qwen/qwen3.6-plus``, ``qwen/qwen3-coder-plus``,
``qwen/qwen3-coder-flash`` and ``deepseek/deepseek-v3.2`` — which OpenRouter
documents as using "the same syntax as Anthropic explicit caching".
Every other provider routstr proxies (OpenAI, Azure, xAI/Grok, Groq, Moonshot,
default DeepSeek, Gemini implicit, Fireworks) caches *automatically* and needs
no markers — they are left untouched.
A client that doesn't know it is talking to one of these models *through*
routstr (e.g. an OpenAI-compatible coding agent pointed at a routstr URL) never
emits the markers — it only adds them when it recognises the provider as
OpenRouter. So caching silently never engages over routstr even though the same
client caches fine talking to OpenRouter directly.
This module restores caching by stamping the standard breakpoints onto the
forwarded body — the system prompt, the last tool, and the last conversation
message (the format allows up to four; we use three, matching the common
agent convention) — but only when the client supplied none of its own, so
explicit client control always wins. The caller is responsible for only
applying this toward an upstream that accepts the markers (OpenRouter /
Anthropic), so they never leak to a provider that would reject them.
"""
from __future__ import annotations
from typing import Any
# The single ephemeral marker stamped onto each chosen breakpoint. A 5-minute
# TTL (the default for ``ephemeral``) — deliberately not the 1h tier, which
# carries a higher cache-write premium and should stay opt-in.
EPHEMERAL_CACHE_CONTROL: dict[str, str] = {"type": "ephemeral"}
# Alibaba's explicit-cache models on OpenRouter. Matched as substrings of the
# model id (any spelling routstr carries). Snapshot endpoints that OpenRouter
# documents as *not* supporting explicit caching (e.g. ``qwen3.5-plus-02-15``)
# are different families and deliberately absent from this list.
_ALIBABA_EXPLICIT_CACHE_SLUGS: tuple[str, ...] = (
"qwen3-max",
"qwen-plus",
"qwen3.6-plus",
"qwen3-coder-plus",
"qwen3-coder-flash",
"deepseek-v3.2",
)
def is_explicit_cache_model(model_id: str | None, *fallbacks: str | None) -> bool:
"""True when the target model uses the explicit ``cache_control`` dialect.
Covers the Claude family (broadly — every Claude model supports it) and
Alibaba's documented explicit-cache models, across the id spellings routstr
carries: the OpenRouter id (``anthropic/claude-...``, ``qwen/qwen3-max``),
the bare upstream id, and any forwarded/canonical alias.
"""
for candidate in (model_id, *fallbacks):
if not candidate:
continue
lowered = candidate.lower()
if "claude" in lowered or "anthropic/" in lowered:
return True
if any(slug in lowered for slug in _ALIBABA_EXPLICIT_CACHE_SLUGS):
return True
return False
def _has_cache_control(obj: Any) -> bool:
"""Recursively detect any client-supplied ``cache_control`` marker."""
if isinstance(obj, dict):
if "cache_control" in obj:
return True
return any(_has_cache_control(v) for v in obj.values())
if isinstance(obj, list):
return any(_has_cache_control(v) for v in obj)
return False
def body_has_cache_control(data: dict) -> bool:
"""True when the request already carries cache_control on messages/tools."""
return _has_cache_control(data.get("messages")) or _has_cache_control(
data.get("tools")
)
def _stamp_text_content(message: dict) -> bool:
"""Add the ephemeral marker to a message's last text block.
A string content is promoted to the array form Anthropic requires for
cache markers; an existing array gets the marker on its last text part.
Returns True when a marker was placed.
"""
content = message.get("content")
if isinstance(content, str):
if not content:
return False
message["content"] = [
{
"type": "text",
"text": content,
"cache_control": dict(EPHEMERAL_CACHE_CONTROL),
}
]
return True
if isinstance(content, list):
for part in reversed(content):
if isinstance(part, dict) and part.get("type") == "text":
part["cache_control"] = dict(EPHEMERAL_CACHE_CONTROL)
return True
return False
def _stamp_system_prompt(messages: list) -> None:
for message in messages:
if isinstance(message, dict) and message.get("role") in (
"system",
"developer",
):
_stamp_text_content(message)
return
def _stamp_last_tool(tools: Any) -> None:
if isinstance(tools, list) and tools and isinstance(tools[-1], dict):
tools[-1]["cache_control"] = dict(EPHEMERAL_CACHE_CONTROL)
def _stamp_last_conversation_message(messages: list) -> None:
for message in reversed(messages):
if isinstance(message, dict) and message.get("role") in ("user", "assistant"):
if _stamp_text_content(message):
return
def inject_anthropic_cache_breakpoints(data: dict) -> bool:
"""Stamp ephemeral cache breakpoints onto an OpenAI-shaped chat body.
Mutates ``data`` in place (the established convention in
``prepare_request_body``) and returns True when anything changed. No-ops
when the body isn't chat-shaped or the client already set cache_control.
"""
messages = data.get("messages")
if not isinstance(messages, list) or not messages:
return False
if body_has_cache_control(data):
return False
_stamp_system_prompt(messages)
_stamp_last_tool(data.get("tools"))
_stamp_last_conversation_message(messages)
return True

View File

@@ -0,0 +1,73 @@
"""TEMPORARY: local DeepSeek V4 pricing shim.
litellm's bundled cost map does not yet ship ``deepseek-v4-flash`` /
``deepseek-v4-pro``. Without an entry, ``backfill_cache_pricing`` cannot find a
``cache_read_input_token_cost`` and cache reads fall back to the full input
rate — a large overcharge on cache hits (DeepSeek V4 hits are ~0.008-0.02x
input, i.e. cached tokens cost 50-120x less than regular input).
This module injects the missing entries into ``litellm.model_cost`` at startup
so the existing backfill path resolves them. Rates mirror the canonical
``deepseek`` provider entries now in litellm's ``model_prices`` map
(``input_cost_per_token`` is the cache-*miss* rate;
``cache_read_input_token_cost`` is the cache-*hit* rate), sourced from
https://api-docs.deepseek.com/quick_start/pricing via
https://github.com/BerriAI/litellm/pull/26380 (issue
https://github.com/BerriAI/litellm/issues/30430).
=== REMOVAL (once litellm ships these models) ===
Delete this file and the single ``register_deepseek_v4_pricing()`` call in
``routstr/core/main.py``. Nothing else depends on it. Entries are only added
when absent, so a stale shim is harmless after upstream lands — but remove it.
"""
import litellm
from ..core import get_logger
logger = get_logger(__name__)
# USD per token. Mirrors the canonical ``deepseek`` provider entries in
# litellm's model_prices map (source: DeepSeek API pricing docs). Keep these in
# sync with ``litellm.model_cost["deepseek/deepseek-v4-*"]``.
_DEEPSEEK_V4_RATES: dict[str, dict[str, float]] = {
"deepseek-v4-flash": {
"input_cost_per_token": 1.4e-07,
"output_cost_per_token": 2.8e-07,
"cache_read_input_token_cost": 2.8e-09,
"cache_creation_input_token_cost": 0.0,
"input_cost_per_token_cache_hit": 2.8e-09,
},
"deepseek-v4-pro": {
"input_cost_per_token": 4.35e-07,
"output_cost_per_token": 8.7e-07,
"cache_read_input_token_cost": 3.625e-09,
"cache_creation_input_token_cost": 0.0,
"input_cost_per_token_cache_hit": 3.625e-09,
},
}
def register_deepseek_v4_pricing() -> None:
"""Inject DeepSeek V4 pricing into ``litellm.model_cost`` if absent.
Idempotent and non-destructive: a key already present in the cost map
(e.g. once litellm ships it) is left untouched. Registers both the bare
(``deepseek-v4-flash``) and prefixed (``deepseek/deepseek-v4-flash``)
spellings since ``backfill_cache_pricing`` tries both.
"""
added = []
for bare, rates in _DEEPSEEK_V4_RATES.items():
for key in (bare, f"deepseek/{bare}"):
if key in litellm.model_cost:
continue
entry: dict[str, object] = dict(rates)
entry["litellm_provider"] = "deepseek"
entry["mode"] = "chat"
litellm.model_cost[key] = entry
added.append(key)
if added:
logger.info(
"Registered temporary DeepSeek V4 pricing shim",
extra={"models": added},
)

View File

@@ -20,7 +20,7 @@ class FireworksUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "FireworksUpstreamProvider":
return cls(

View File

@@ -51,7 +51,7 @@ class GeminiUpstreamProvider(BaseUpstreamProvider):
return self._client
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "GeminiUpstreamProvider":
return cls(

View File

@@ -5,6 +5,12 @@ from typing import TYPE_CHECKING
import httpx
from .base import BaseUpstreamProvider
from .pricing_resolver import (
FallbackPricingResolver,
ResolvedPricing,
_as_float,
estimate_context_length,
)
if TYPE_CHECKING:
from ..core.db import UpstreamProviderRow
@@ -45,7 +51,7 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "GenericUpstreamProvider":
return cls(
@@ -64,6 +70,40 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
"platform_url": cls.platform_url,
}
def _native_pricing(
self, model_id: str, model_spec: dict
) -> ResolvedPricing | None:
"""Read pricing/metadata from Venice's bespoke ``model_spec`` schema.
Returns ``None`` when the upstream reported no *usable* native price —
absent, non-numeric, negative, or both-zero — so the caller falls
through to the shared resolution chain instead of fabricating a number
or trusting a bogus one. This mirrors the money-safety guards the
litellm and OpenRouter rungs already apply: a both-zero price would
serve the model free, a negative one would credit the caller, and a
non-numeric string would otherwise throw and drop the whole catalog.
"""
pricing_info = model_spec.get("pricing", {})
input_usd = _as_float(pricing_info.get("input", {}).get("usd"))
output_usd = _as_float(pricing_info.get("output", {}).get("usd"))
if input_usd is None or output_usd is None:
return None
if input_usd < 0 or output_usd < 0 or (input_usd == 0 and output_usd == 0):
return None
capabilities = model_spec.get("capabilities", {})
input_modalities = ["text"]
if capabilities.get("supportsVision", False):
input_modalities.append("image")
return ResolvedPricing(
prompt=input_usd / 1_000_000,
completion=output_usd / 1_000_000,
context_length=model_spec.get("availableContextTokens"),
source="native",
input_modalities=input_modalities,
)
async def fetch_models(self) -> list[Model]:
"""Fetch models from upstream API using /models endpoint."""
from ..payment.models import Architecture, Model, Pricing, TopProvider
@@ -78,6 +118,7 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
response.raise_for_status()
data = response.json()
resolver = FallbackPricingResolver()
models_list = []
for model_data in data.get("data", []):
model_id = model_data.get("id", "")
@@ -89,41 +130,44 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
owned_by = model_data.get("owned_by", "unknown")
model_spec = model_data.get("model_spec", {})
context_length = 4096
if model_spec.get("availableContextTokens"):
context_length = model_spec["availableContextTokens"]
elif any(
pattern in model_id.lower() for pattern in ["32k", "32000"]
):
context_length = 32768
elif any(
pattern in model_id.lower() for pattern in ["16k", "16000"]
):
context_length = 16384
elif any(pattern in model_id.lower() for pattern in ["8k", "8000"]):
context_length = 8192
elif "gpt-4" in model_id.lower():
context_length = 8192
elif "claude" in model_id.lower():
context_length = 200000
resolved = self._native_pricing(model_id, model_spec)
if resolved is None:
resolved = await resolver.resolve(model_id)
pricing_info = model_spec.get("pricing", {})
input_pricing = pricing_info.get("input", {})
output_pricing = pricing_info.get("output", {})
if resolved is None:
# Fail closed: never invent a price. Import the model
# disabled with a warning so the operator can price it
# (the admin UI surfaces disabled remote models).
logger.warning(
f"No pricing source resolved for '{model_id}' from "
f"{self.upstream_name}; importing it disabled",
extra={"model_id": model_id, "base_url": self.base_url},
)
resolved = ResolvedPricing(
prompt=0.0,
completion=0.0,
context_length=None,
source="unresolved",
)
enabled = False
else:
enabled = True
prompt_price = input_pricing.get("usd", 0.001) / 1000000
completion_price = output_pricing.get("usd", 0.001) / 1000000
# Prefer the source's own modality string (OpenRouter ships
# one, e.g. "text+image->text"); otherwise derive it from the
# captured input/output modalities in the same "in->out" shape
# rather than flattening vision models to "text->text".
modality = resolved.modality or (
f"{'+'.join(resolved.input_modalities)}"
f"->{'+'.join(resolved.output_modalities)}"
)
capabilities = model_spec.get("capabilities", {})
input_modalities = ["text"]
output_modalities = ["text"]
if capabilities.get("supportsVision", False):
input_modalities.append("image")
modality = "text"
if capabilities.get("supportsVision", False):
modality = "text->text"
# A source can carry a price but no context (e.g. a litellm
# entry missing max_input_tokens); fall back to an id-based
# estimate so we never persist a zero-length window.
context_length = resolved.context_length or estimate_context_length(
model_id
)
spec_name = model_spec.get("name", model_name)
description = f"{spec_name}"
@@ -139,30 +183,33 @@ class GenericUpstreamProvider(BaseUpstreamProvider):
context_length=context_length,
architecture=Architecture(
modality=modality,
input_modalities=input_modalities,
output_modalities=output_modalities,
tokenizer="unknown",
instruct_type=None,
input_modalities=resolved.input_modalities,
output_modalities=resolved.output_modalities,
tokenizer=resolved.tokenizer,
instruct_type=resolved.instruct_type,
),
pricing=Pricing(
prompt=prompt_price,
completion=completion_price,
prompt=resolved.prompt,
completion=resolved.completion,
request=0.0,
image=0.0,
web_search=0.0,
internal_reasoning=0.0,
max_prompt_cost=0.001,
max_completion_cost=0.001,
max_cost=0.001,
input_cache_read=resolved.input_cache_read,
input_cache_write=resolved.input_cache_write,
),
sats_pricing=None,
per_request_limits=None,
top_provider=TopProvider(
context_length=context_length,
max_completion_tokens=context_length // 2,
is_moderated=False,
max_completion_tokens=(
resolved.max_completion_tokens
if resolved.max_completion_tokens is not None
else context_length // 2
),
is_moderated=bool(resolved.is_moderated),
),
enabled=True,
enabled=enabled,
upstream_provider_id=None,
canonical_slug=None,
)

View File

@@ -20,7 +20,7 @@ class GroqUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "GroqUpstreamProvider":
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "GroqUpstreamProvider":
return cls(
api_key=provider_row.api_key,
provider_fee=provider_row.provider_fee,

View File

@@ -12,6 +12,7 @@ from sqlmodel import select
from ..core import get_logger
from ..core.db import AsyncSession, ModelRow, UpstreamProviderRow, create_session
from ..core.provider_slugs import allocate_unique_provider_slug
from ..payment.models import Model
from .base import BaseUpstreamProvider
@@ -215,9 +216,6 @@ async def init_upstreams() -> list[BaseUpstreamProvider]:
provider = _instantiate_provider(provider_row)
if provider:
# Keep provider DB id on runtime instance so model mapping can
# bind DB overrides to the correct upstream.
setattr(provider, "db_id", provider_row.id)
await provider.refresh_models_cache()
logger.debug(
f"Initialized {provider_row.provider_type} provider",
@@ -250,6 +248,7 @@ async def _seed_providers_from_settings(
providers_to_add: list[UpstreamProviderRow] = []
seeded_provider_keys: set[tuple[str, str]] = set()
reserved_slugs: set[str] = set()
provider_classes_by_type = {
cls.provider_type: cls
@@ -279,8 +278,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, provider_type, reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type=provider_type,
base_url=base_url,
api_key=api_key,
@@ -299,8 +303,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "ollama", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="ollama",
base_url=ollama_base_url,
api_key=ollama_api_key,
@@ -320,8 +329,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "azure", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="azure",
base_url=base_url,
api_key=api_key,
@@ -342,8 +356,13 @@ async def _seed_providers_from_settings(
)
)
if not result.first():
slug = await allocate_unique_provider_slug(
session, "custom", reserved_slugs
)
reserved_slugs.add(slug)
providers_to_add.append(
UpstreamProviderRow(
slug=slug,
provider_type="custom",
base_url=base_url,
api_key=api_key,
@@ -356,7 +375,7 @@ async def _seed_providers_from_settings(
session.add(provider)
logger.info(
f"Seeding {provider.provider_type} provider", # type: ignore[str-format]
extra={"base_url": provider.base_url},
extra={"base_url": provider.base_url, "slug": provider.slug},
)
@@ -391,9 +410,7 @@ def _instantiate_provider(
return provider
if provider_row.provider_type == "custom":
return BaseUpstreamProvider(
provider_row.base_url, provider_row.api_key, provider_row.provider_fee
)
return BaseUpstreamProvider.from_db_row(provider_row)
logger.error(
f"Unknown provider type: {provider_row.provider_type}",

View File

@@ -29,7 +29,9 @@ import litellm
from ..core import get_logger
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
from ..payment.models import Model
from .rate_limit import classify_rate_limit
logger = get_logger(__name__)
@@ -505,23 +507,33 @@ async def dispatch_anthropic_messages(
try:
result = await litellm.anthropic.messages.acreate(**kwargs)
except Exception as exc:
exc_message = getattr(exc, "message", None) or str(exc) or repr(exc)
raw_message = getattr(exc, "message", None) or str(exc) or repr(exc)
# Redact provider account identifiers before the message reaches logs
# or the surfaced error.
exc_message = redact_org_ids(raw_message)
exc_status = getattr(exc, "status_code", None)
exc_response = getattr(exc, "response", None)
response_text = None
if exc_response is not None:
try:
response_text = getattr(exc_response, "text", str(exc_response))
response_text = redact_org_ids(
getattr(exc_response, "text", str(exc_response))
)
except Exception:
response_text = "<unreadable>"
status_for_classify = exc_status if isinstance(exc_status, int) else 502
rate_limit = classify_rate_limit(
status_for_classify, exc_message, getattr(exc, "headers", None)
)
logger.error(
"litellm dispatch failed",
extra={
"error": exc_message,
"error_type": type(exc).__name__,
"status_code": exc_status,
"error_code": rate_limit.code if rate_limit else None,
"llm_provider": getattr(exc, "llm_provider", None),
"body": getattr(exc, "body", None),
"body": redact_org_ids(str(getattr(exc, "body", "") or "")) or None,
"response_text": response_text,
"model": litellm_model,
"api_base": base_url,
@@ -529,7 +541,9 @@ async def dispatch_anthropic_messages(
)
raise UpstreamError(
f"Upstream error via litellm: {exc_message}",
status_code=exc_status if isinstance(exc_status, int) else 502,
status_code=status_for_classify,
code=rate_limit.code if rate_limit else None,
details=rate_limit.as_details() if rate_limit else None,
) from exc
if not client_stream and hasattr(result, "__aiter__"):

View File

@@ -43,7 +43,7 @@ class OllamaUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OllamaUpstreamProvider":
return cls(

View File

@@ -20,7 +20,7 @@ class OpenAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OpenAIUpstreamProvider":
return cls(

View File

@@ -1,3 +1,4 @@
import json
from typing import TYPE_CHECKING
import httpx
@@ -18,6 +19,38 @@ class OpenRouterUpstreamProvider(BaseUpstreamProvider):
supports_anthropic_messages = True
litellm_provider_prefix = "openrouter/"
def prepare_request_body(
self, body: bytes | None, model_obj: Model
) -> bytes | None:
"""Set provider.require_parameters on tool-use requests.
Without it OpenRouter can route a tool call to an endpoint that doesn't
support function calling and 404 with "No endpoints found that support
tool use". We leave a client-supplied value untouched.
"""
body = super().prepare_request_body(body, model_obj)
if not body:
return body
try:
data = json.loads(body)
except json.JSONDecodeError:
return body
if not isinstance(data, dict) or not data.get("tools"):
return body
provider = data.get("provider")
if not isinstance(provider, dict):
provider = {}
if "require_parameters" in provider:
return body
provider["require_parameters"] = True
data["provider"] = provider
return json.dumps(data).encode()
def _apply_provider_field(self, response_json: object) -> None:
"""Stamp the ``provider`` field for OpenRouter responses.
@@ -57,7 +90,7 @@ class OpenRouterUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "OpenRouterUpstreamProvider":
return cls(

View File

@@ -23,7 +23,7 @@ class PerplexityUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "PerplexityUpstreamProvider":
return cls(

View File

@@ -46,7 +46,7 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "PPQAIUpstreamProvider":
return cls(
@@ -229,17 +229,14 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
f"Disabling PPQ.AI provider ({self.base_url}) due to insufficient balance",
extra={"error": error_message},
)
from sqlmodel import select
from ..core.db import UpstreamProviderRow, create_session
async with create_session() as session:
statement = select(UpstreamProviderRow).where(
UpstreamProviderRow.base_url == self.base_url,
UpstreamProviderRow.api_key == self.api_key,
provider = (
await session.get(UpstreamProviderRow, self.db_id)
if self.db_id is not None
else None
)
result = await session.exec(statement)
provider = result.first()
if provider:
provider.enabled = False

View File

@@ -0,0 +1,203 @@
"""Shared price/metadata resolution chain for upstream model discovery.
Most OpenAI-compatible ``/models`` responses carry no pricing. Rather than let
a provider fabricate one, this module resolves a model through decreasingly
trustworthy sources — litellm's bundled cost map (curated list prices, mirrors
provider docs), then the OpenRouter feed (resale prices, broader coverage) —
and returns ``None`` when none of them know the model, so the caller can fail
closed instead of inventing a number.
Provider-native pricing (a gateway's own ``/models`` schema, e.g. Venice's
``model_spec``) is authoritative and handled by the provider before this chain
is consulted; only the shared fallback lives here so a later refactor can hoist
it into the base provider unchanged.
"""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass
class ResolvedPricing:
"""Per-token pricing plus whatever metadata the answering source carried.
Prices are USD per token. ``source`` records provenance
(``native``/``litellm``/``openrouter``/``unresolved``) so later work can
surface where each price came from.
"""
prompt: float
completion: float
context_length: int | None
source: str
modality: str | None = None
max_completion_tokens: int | None = None
input_cache_read: float = 0.0
input_cache_write: float = 0.0
input_modalities: list[str] = field(default_factory=lambda: ["text"])
output_modalities: list[str] = field(default_factory=lambda: ["text"])
tokenizer: str = "unknown"
instruct_type: str | None = None
is_moderated: bool | None = None
def estimate_context_length(model_id: str) -> int:
"""Best-effort context window from a model id when no source reports one.
The last rung of the fallback chain, reached only for a model whose price
resolved but whose context did not (or that imported disabled). Context is
not a billing input, so a rough id-based guess is acceptable here where a
guessed *price* never would be.
"""
lowered = model_id.lower()
if any(pattern in lowered for pattern in ["32k", "32000"]):
return 32768
if any(pattern in lowered for pattern in ["16k", "16000"]):
return 16384
if any(pattern in lowered for pattern in ["8k", "8000"]):
return 8192
if "gpt-4" in lowered:
return 8192
if "claude" in lowered:
return 200000
return 4096
def _as_float(value: object) -> float | None:
"""OpenRouter reports prices as strings; coerce, ``None`` if unparseable."""
try:
return float(value) # type: ignore[arg-type]
except (TypeError, ValueError):
return None
def _as_int(value: object) -> int | None:
"""Coerce an already-numeric token count to ``int``, else ``None``."""
return int(value) if isinstance(value, (int, float)) else None
def _from_litellm(model_id: str) -> ResolvedPricing | None:
# Lazy import so the resolver stays import-light and shares the exact
# lookup semantics used by cache-rate backfill.
from ..payment.models import litellm_cost_entry
info = litellm_cost_entry(model_id)
if info is None:
return None
prompt = info.get("input_cost_per_token")
completion = info.get("output_cost_per_token")
if not isinstance(prompt, (int, float)) or not isinstance(completion, (int, float)):
return None
# A both-zero entry is litellm listing a model without a real price (free
# moderation/rerank tiers do this) — treating 0/0 as resolved would serve
# the model for free. Reject it (and any negative) so the caller falls
# through, mirroring async_fetch_openrouter_models' _has_valid_pricing.
if prompt < 0 or completion < 0 or (prompt == 0 and completion == 0):
return None
input_modalities = ["text"]
if info.get("supports_vision"):
input_modalities.append("image")
return ResolvedPricing(
prompt=float(prompt),
completion=float(completion),
# max_input_tokens is the context window; max_tokens is litellm's
# completion cap (it tracks max_output_tokens for ~94% of models), so
# it is never a context source. A missing window falls to the id-based
# estimate downstream rather than borrowing the output cap.
context_length=_as_int(info.get("max_input_tokens")),
source="litellm",
max_completion_tokens=_as_int(info.get("max_output_tokens")),
input_cache_read=float(info.get("cache_read_input_token_cost") or 0.0),
input_cache_write=float(info.get("cache_creation_input_token_cost") or 0.0),
input_modalities=input_modalities,
)
def _match_openrouter(model_id: str, feed: list[dict]) -> dict | None:
"""Find ``model_id`` in the OpenRouter feed, exact id before bare tail.
Bare-tail matching (``deepseek-chat`` ↔ ``deepseek/deepseek-chat``) is a
looser, lower-trust match — OpenRouter fans a model out across resellers —
so an exact id match always wins first. When several entries share the bare
tail, the one with the highest *combined* (prompt + completion) per-token
cost wins: the choice must be deterministic (not feed-order-dependent) and
money-safe whichever way traffic leans, since undercharging is the hazard.
Ranking on prompt alone could pick an entry that is cheap on input but dear
on output. The live feed has no such collisions today; this only governs
the latent case.
"""
bare = model_id.split("/", 1)[-1]
exact = next((m for m in feed if m.get("id") == model_id), None)
if exact is not None:
return exact
matches = [m for m in feed if m.get("id", "").split("/", 1)[-1] == bare]
if not matches:
return None
def _combined_cost(m: dict) -> float:
pricing = m.get("pricing", {})
return (_as_float(pricing.get("prompt")) or 0.0) + (
_as_float(pricing.get("completion")) or 0.0
)
return max(matches, key=_combined_cost)
def _from_openrouter(model_id: str, feed: list[dict]) -> ResolvedPricing | None:
entry = _match_openrouter(model_id, feed)
if entry is None:
return None
pricing = entry.get("pricing", {})
prompt = _as_float(pricing.get("prompt"))
completion = _as_float(pricing.get("completion"))
if prompt is None or completion is None:
return None
architecture = entry.get("architecture", {})
top_provider = entry.get("top_provider", {})
return ResolvedPricing(
prompt=prompt,
completion=completion,
context_length=_as_int(entry.get("context_length")),
source="openrouter",
modality=architecture.get("modality"),
max_completion_tokens=_as_int(top_provider.get("max_completion_tokens")),
input_cache_read=_as_float(pricing.get("input_cache_read")) or 0.0,
input_cache_write=_as_float(pricing.get("input_cache_write")) or 0.0,
input_modalities=architecture.get("input_modalities") or ["text"],
output_modalities=architecture.get("output_modalities") or ["text"],
tokenizer=architecture.get("tokenizer") or "unknown",
instruct_type=architecture.get("instruct_type"),
is_moderated=top_provider.get("is_moderated"),
)
class FallbackPricingResolver:
"""Resolves models via litellm → OpenRouter for one discovery pass.
The OpenRouter catalog is fetched at most once and only when a model
actually misses litellm, so a provider full of litellm-known models never
touches the network. Instantiate one per ``fetch_models`` call.
"""
def __init__(self) -> None:
self._openrouter_feed: list[dict] | None = None
async def resolve(self, model_id: str) -> ResolvedPricing | None:
"""Resolve ``model_id``; ``None`` if no source knows it."""
resolved = _from_litellm(model_id)
if resolved is not None:
return resolved
if self._openrouter_feed is None:
# Lazy import so tests can patch the feed at its source.
from ..payment.models import async_fetch_openrouter_models
self._openrouter_feed = await async_fetch_openrouter_models()
return _from_openrouter(model_id, self._openrouter_feed)

View File

@@ -0,0 +1,136 @@
"""Detection and parsing of upstream provider rate-limit errors.
Upstream OpenAI-compatible providers signal rate limits via HTTP 429 and/or a
human-readable message such as::
Rate limit reached for gpt-5.5-2026-04-23 (for limit gpt-5.5) in organization
org-XXXX on tokens per min (TPM): Limit 180000000, Used 180000000,
Requested 8929. Please try again in 2ms.
This module classifies those failures into a stable :data:`UPSTREAM_RATE_LIMIT`
code and extracts useful debugging fields. All retained text is redacted of
organization IDs first.
"""
from __future__ import annotations
import re
from dataclasses import asdict, dataclass
from ..core.redaction import redact_org_ids
# Stable error code callers can switch on to distinguish upstream rate limits
# from generic request failures. The literal value matches the identifier named
# in issue #555 ("UPSTREAM_RATE_LIMIT") so the public API contract is exact.
UPSTREAM_RATE_LIMIT = "UPSTREAM_RATE_LIMIT"
# Message fragments that indicate a rate-limit even when the status code is not
# 429 (some providers wrap it in a 400/500 envelope).
_RATE_LIMIT_MARKERS = (
"rate limit reached",
"rate_limit_exceeded",
"rate limit exceeded",
"too many requests",
)
_MODEL_RE = re.compile(r"Rate limit reached for ([^\s(]+)", re.IGNORECASE)
_LIMIT_NAME_RE = re.compile(r"\(for limit ([^)]+)\)", re.IGNORECASE)
_METRIC_RE = re.compile(r"on ([a-z ]+\((?:TPM|RPM|TPD|RPD|IPM)\))", re.IGNORECASE)
_LIMIT_RE = re.compile(r"Limit (\d+)", re.IGNORECASE)
_USED_RE = re.compile(r"Used (\d+)", re.IGNORECASE)
_REQUESTED_RE = re.compile(r"Requested (\d+)", re.IGNORECASE)
_RETRY_RE = re.compile(r"try again in ([\d.]+)\s*(ms|s)", re.IGNORECASE)
@dataclass
class RateLimitInfo:
"""Structured, redaction-safe view of an upstream rate-limit error."""
code: str
message: str
model: str | None = None
limit_name: str | None = None
metric: str | None = None
limit: int | None = None
used: int | None = None
requested: int | None = None
retry_after_seconds: float | None = None
def as_details(self) -> dict[str, object]:
"""Return a JSON-serialisable dict for embedding in an error envelope."""
return {k: v for k, v in asdict(self).items() if v is not None}
def _looks_like_rate_limit(status_code: int, message: str) -> bool:
if status_code == 429:
return True
lowered = message.lower()
return any(marker in lowered for marker in _RATE_LIMIT_MARKERS)
def _parse_retry_after_header(headers: dict[str, str] | None) -> float | None:
"""Parse a ``Retry-After`` header (delta-seconds form) into seconds."""
if not headers:
return None
raw = headers.get("retry-after") or headers.get("Retry-After")
if raw is None:
return None
try:
return float(str(raw).strip())
except (TypeError, ValueError):
return None
def _int_or_none(match: re.Match[str] | None) -> int | None:
if match is None:
return None
try:
return int(match.group(1))
except (TypeError, ValueError):
return None
def classify_rate_limit(
status_code: int,
message: str,
headers: dict[str, str] | None = None,
) -> RateLimitInfo | None:
"""Classify an upstream error as a rate-limit and extract its fields.
Args:
status_code: HTTP status code from the upstream response.
message: Upstream error message (may contain sensitive identifiers).
headers: Optional upstream response headers, used for ``Retry-After``.
Returns:
A :class:`RateLimitInfo` when the error is a rate-limit, else ``None``.
"""
message = message or ""
if not _looks_like_rate_limit(status_code, message):
return None
redacted = redact_org_ids(message)
model_match = _MODEL_RE.search(redacted)
metric_match = _METRIC_RE.search(redacted)
retry_after = _parse_retry_after_header(headers)
if retry_after is None:
retry_match = _RETRY_RE.search(redacted)
if retry_match is not None:
value = float(retry_match.group(1))
retry_after = value / 1000.0 if retry_match.group(2).lower() == "ms" else value
limit_name_match = _LIMIT_NAME_RE.search(redacted)
return RateLimitInfo(
code=UPSTREAM_RATE_LIMIT,
message=redacted,
model=model_match.group(1) if model_match else None,
limit_name=limit_name_match.group(1).strip() if limit_name_match else None,
metric=metric_match.group(1).strip() if metric_match else None,
limit=_int_or_none(_LIMIT_RE.search(redacted)),
used=_int_or_none(_USED_RE.search(redacted)),
requested=_int_or_none(_REQUESTED_RE.search(redacted)),
retry_after_seconds=retry_after,
)

View File

@@ -0,0 +1,142 @@
"""Reactive request-correction layer.
When an upstream rejects a request with a recoverable 4xx error, this layer
tries to *fix* the request body and let the caller retry the same upstream
instead of failing outright. It is provider-agnostic: correctors key off the
upstream's own error wording, so the same recovery works across every provider.
The layer is a small pipeline of :data:`Corrector` callables. Each corrector
inspects the parsed request body and the upstream error message and either
returns a corrected body (plus a short label identifying the fix) or declines
by returning ``None``. Adding a new reactive fix means writing one corrector
and adding it to :data:`DEFAULT_CORRECTORS` — no changes to the proxy loop.
All corrections are immutable: a corrector never mutates the body it is given,
it returns a new ``dict``. The proxy threads an ``applied`` set of fix labels
through retries so each distinct fix is applied at most once, guaranteeing the
retry loop always terminates.
"""
from __future__ import annotations
import json
import re
from collections.abc import Callable, Sequence
from dataclasses import dataclass
from fastapi.responses import Response
from ..core import get_logger
logger = get_logger(__name__)
# Matches upstream error text that names a single rejected request parameter,
# e.g. "`temperature` is deprecated for this model." or
# "parameter 'top_p' is not supported". Keys off the upstream's own wording so
# a 400 about an unsupported sampling/option field can be recovered by stripping
# that field and retrying the same upstream.
_UNSUPPORTED_PARAM_RE = re.compile(
r"[`'\"]?(?P<param>[a-zA-Z_][a-zA-Z0-9_]*)[`'\"]?\s+is\s+"
r"(?:deprecated|not\s+supported|unsupported|no\s+longer\s+supported)",
re.IGNORECASE,
)
# A corrector inspects the parsed request body and the upstream error message
# and returns ``(new_body_dict, label)`` for a fix it can apply, or ``None`` to
# decline. ``label`` identifies the fix so it is applied at most once per request.
Corrector = Callable[[dict, str], "tuple[dict, str] | None"]
@dataclass(frozen=True)
class Correction:
"""A successful request correction ready to retry.
``body`` is the corrected JSON body (encoded), ``label`` identifies the fix
that was applied (e.g. the stripped param name) so the caller can guard
against applying the same fix twice.
"""
body: bytes
label: str
def extract_error_message(response: Response) -> str:
"""Best-effort extraction of an error message string from a proxy Response."""
body_bytes = getattr(response, "body", None)
if not body_bytes:
return ""
try:
data = json.loads(body_bytes)
except Exception:
return body_bytes.decode("utf-8", errors="ignore")[:500]
if isinstance(data, dict):
err = data.get("error")
if isinstance(err, dict):
msg = err.get("message") or err.get("detail")
if isinstance(msg, str):
return msg
elif isinstance(err, str):
return err
if isinstance(data.get("message"), str):
return data["message"]
return ""
def strip_unsupported_param(
body: dict, error_message: str
) -> tuple[dict, str] | None:
"""Drop a top-level param the upstream named as unsupported/deprecated.
Returns ``(new_body, param)`` (a new dict, original untouched) when the
error names a top-level param present in the body, otherwise ``None``.
"""
match = _UNSUPPORTED_PARAM_RE.search(error_message)
if not match:
return None
param = match.group("param")
if param not in body:
return None
new_body = {k: v for k, v in body.items() if k != param}
return new_body, param
# Ordered pipeline of correctors tried on each recoverable rejection.
DEFAULT_CORRECTORS: tuple[Corrector, ...] = (strip_unsupported_param,)
def correct_request(
request_body: bytes,
error_message: str,
applied: set[str],
correctors: Sequence[Corrector] = DEFAULT_CORRECTORS,
) -> Correction | None:
"""Try to correct a rejected request body so it can be retried.
Runs each corrector in order against the parsed body and ``error_message``.
The first corrector that proposes a fix whose ``label`` is not already in
``applied`` wins; its result is returned as a :class:`Correction`. Returns
``None`` when nothing parses, nothing matches, or every proposed fix was
already applied — the caller then treats the response as a normal failure.
``applied`` is read-only here; the caller records the returned ``label`` to
bound retries and guarantee forward progress.
"""
if not request_body or not error_message:
return None
try:
data = json.loads(request_body)
except Exception:
return None
if not isinstance(data, dict):
return None
for corrector in correctors:
result = corrector(data, error_message)
if result is None:
continue
new_body, label = result
if label in applied:
continue
return Correction(body=json.dumps(new_body).encode(), label=label)
return None

View File

@@ -55,7 +55,7 @@ class RoutstrUpstreamProvider(BaseUpstreamProvider):
return path.lstrip("/")
@classmethod
def from_db_row(
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "RoutstrUpstreamProvider":
import json

View File

@@ -21,7 +21,7 @@ class XAIUpstreamProvider(BaseUpstreamProvider):
)
@classmethod
def from_db_row(cls, provider_row: "UpstreamProviderRow") -> "XAIUpstreamProvider":
def _build_from_row(cls, provider_row: "UpstreamProviderRow") -> "XAIUpstreamProvider":
return cls(
api_key=provider_row.api_key,
provider_fee=provider_row.provider_fee,

View File

@@ -1,8 +1,11 @@
import asyncio
import re
import socket
import time
import typing
from typing import TypedDict
import httpx
from cashu.core.base import Proof, Token
from cashu.core.mint_info import MintInfo as _CashuMintInfo
from cashu.wallet.helpers import deserialize_token_from_string
@@ -30,11 +33,172 @@ _CashuMintInfo.model_rebuild(force=True)
logger = get_logger(__name__)
class MintConnectionError(Exception):
"""The mint could not be reached (network transport failure).
Maps to a 503, not a 4xx: the token is fine, the mint is just unavailable.
"""
class TokenConsumedError(Exception):
"""A failure that happened AFTER the token's proofs were spent (melt
succeeded, or redemption already returned) — e.g. minting on the primary
mint or the DB credit then failed.
Non-retryable: the same token will not work again. Seals the cause chain so
a transport error underneath is never re-surfaced as a retryable
mint_unreachable.
"""
# httpx base classes cover their subclasses. HTTPStatusError is excluded on
# purpose — that means the mint answered, just with an error status.
_TRANSPORT_EXC_TYPES: tuple[type[BaseException], ...] = (
httpx.NetworkError,
httpx.TimeoutException,
ConnectionError, # refused/reset/aborted
socket.gaierror, # DNS failure
asyncio.TimeoutError,
)
def is_mint_connection_error(error: BaseException) -> bool:
"""True if ``error`` (or anything in its cause/context chain) is a mint
transport failure. Walks the chain because some sites re-raise transport
errors wrapped in ValueError/MintConnectionError; matches on TYPE, not text.
"""
seen: set[int] = set()
current: BaseException | None = error
while current is not None and id(current) not in seen:
seen.add(id(current))
if isinstance(current, TokenConsumedError):
# Sealed: the token was already spent, so whatever transport error
# sits underneath must not make this look retryable.
return False
if isinstance(current, MintConnectionError):
return True
if isinstance(current, _TRANSPORT_EXC_TYPES):
return True
current = current.__cause__ or current.__context__
return False
# Redemption ``code`` values whose token is spent/consumed/unusable — the
# X-Cashu path must NOT echo the original token for these (echoing invites a
# retry with a token that can never succeed again).
SPENT_TOKEN_CODES: frozenset[str] = frozenset(
{
"cashu_token_already_spent",
"cashu_token_consumed",
"cashu_token_zero_value",
"internal_error",
}
)
def classify_redemption_error(
error: Exception,
) -> tuple[str, int, str, str] | None:
"""Map a token-redemption failure to ``(type, status, message, code)``.
Single source of truth for every endpoint that redeems a token (bearer,
X-Cashu, top-up) so the same failure yields the same taxonomy everywhere.
``type`` and ``code`` are stable client contract; ``message`` is sanitized
(raw error text stays in logs). Returns None for an unclassified internal
fault — the caller emits a generic 500.
"""
if isinstance(error, TokenConsumedError):
return (
"token_consumed",
500,
"Token was redeemed but could not be credited; do not retry",
"cashu_token_consumed",
)
if is_mint_connection_error(error):
return (
"mint_unreachable",
503,
"Cashu mint is unreachable",
"cashu_mint_unreachable",
)
lowered = str(error).lower()
if "already spent" in lowered:
return (
"token_already_spent",
400,
"Cashu token already spent",
"cashu_token_already_spent",
)
if (
"insufficient" in lowered
or "melt fee" in lowered
or "exceed token amount" in lowered
or "estimate fees" in lowered
):
return (
"mint_error",
422,
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
)
if "failed to melt" in lowered:
return (
"mint_error",
422,
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
)
if ("invalid" in lowered or "decode" in lowered) and "token" in lowered:
# Anchored to "token" so internal faults whose text merely contains
# "invalid"/"decode" fall through to the 500 branch, not a token error.
return (
"invalid_token",
400,
"Invalid Cashu token",
"invalid_cashu_token",
)
if "must be positive" in lowered or "yielded no value" in lowered:
# Redeemed to <= 0 (empty/dust token, or value fully consumed by fees).
# Consumed, so non-retryable, but its own code — not the generic bucket.
return (
"cashu_error",
400,
"Failed to redeem Cashu token: token yielded no value",
"cashu_token_zero_value",
)
if isinstance(error, ValueError):
return (
"cashu_error",
400,
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
)
return None
async def get_balance(unit: str) -> int:
wallet = await get_wallet(settings.primary_mint, unit)
return wallet.available_balance.amount
async def _redeem_same_mint(
wallet: Wallet, token_obj: Token
) -> tuple[int, str, str]: # amount, unit, mint_url
"""Redeem proofs at their own issuing mint (no cross-mint swap).
split() re-mints the incoming proofs into fresh ones we own so the sender
can't double-spend them. With include_fees=True the mint deducts its NUT-02
per-proof input fee, so we end up holding only `amount - input_fees`. Credit
that, not the face value, or routstr over-credits the user and its wallet
drifts insolvent.
"""
await wallet.load_mint(keyset_id=token_obj.keysets[0])
wallet.verify_proofs_dleq(token_obj.proofs)
input_fees = wallet.get_fees_for_proofs(token_obj.proofs)
await wallet.split(proofs=token_obj.proofs, amount=0, include_fees=True)
return int(token_obj.amount) - input_fees, token_obj.unit, token_obj.mint
async def recieve_token(
token: str,
) -> tuple[int, str, str]: # amount, unit, mint_url
@@ -48,12 +212,7 @@ async def recieve_token(
if token_obj.mint not in settings.cashu_mints:
return await swap_to_primary_mint(token_obj, wallet)
await wallet.load_mint(keyset_id=token_obj.keysets[0])
wallet.verify_proofs_dleq(token_obj.proofs)
await wallet.split(proofs=token_obj.proofs, amount=0, include_fees=True)
return token_obj.amount, token_obj.unit, token_obj.mint
return await _redeem_same_mint(wallet, token_obj)
async def send(amount: int, unit: str, mint_url: str | None = None) -> tuple[int, str]:
@@ -116,6 +275,72 @@ async def send_token(amount: int, unit: str, mint_url: str | None = None) -> str
return token
# A foreign mint's fee_reserve is a non-binding estimate (NUT-05): the mint may
# demand more when re-quoting or at melt execution. Instead of padding the
# estimate with a safety buffer (which strands the margin at the foreign mint
# on every swap), the swap retries with the amount recomputed from the fees the
# mint actually demands, up to this many attempts.
_MAX_SWAP_ATTEMPTS = 3
_MINT_ERROR_CODE_RE = re.compile(r"\(Code: (\d+)\)")
_MELT_SHORTFALL_RE = re.compile(r"Provided: (\d+), needed: (\d+)")
# Insufficient-melt-inputs failures differ across mint implementations. 11005 is
# the registered "Transaction is not balanced" code (cdk), specific enough to
# trust on the code alone. 11000 is nutshell's generic, unregistered
# TransactionError covering many unrelated failures, so it only counts as a fee
# shortfall alongside the "not enough inputs" detail text. With no code suffix at
# all, that same text is the only signal.
def _net_minted_amount(amount_msat: int, token_unit: str, fees: int) -> int:
"""
Convert the token value minus fees (given in the token unit) into an
amount in the primary mint's unit.
"""
fee_msat = fees * 1000 if token_unit == "sat" else fees
remaining_msat = amount_msat - fee_msat
if settings.primary_mint_unit == "sat":
return int(remaining_msat // 1000)
return int(remaining_msat)
def _melt_insufficient_shortfall(error: Exception) -> int | None:
"""
Classify a melt failure: return the observed shortfall (in the token unit)
when the mint rejected the inputs as insufficient, or None when the failure
is unrelated to fees and must not be retried (e.g. a Lightning payment
failure, where a smaller invoice would not help).
Cashu errors carry no structured amounts (NUT-00 defines only detail/code,
flattened to "Mint Error: <detail> (Code: <code>)" by cashu-py), so the
classification uses the code and the shortfall must be inferred: the
"Provided: X, needed: Y" amounts are nutshell-specific free text and only
refine the shortfall when present; otherwise shrink one unit at a time.
"""
message = str(error)
code_match = _MINT_ERROR_CODE_RE.search(message)
code = code_match.group(1) if code_match is not None else None
has_shortfall_text = "not enough inputs" in message.lower()
match code:
case "11005": # registered TransactionUnbalanced: trust the code
pass
case "11000" if has_shortfall_text: # generic nutshell error: needs the text
pass
case None if has_shortfall_text: # no code suffix: text is the only signal
pass
case _: # other codes, a bare 11000, or no signal: must not retry
return None
amounts = _MELT_SHORTFALL_RE.search(message)
if amounts is not None:
provided, needed = int(amounts.group(1)), int(amounts.group(2))
if needed > provided:
return needed - provided
return 1
async def _calculate_swap_amount(
amount_msat: int,
token_unit: str,
@@ -154,26 +379,18 @@ async def _calculate_swap_amount(
fee_reserve = dummy_melt_quote.fee_reserve
input_fees = token_wallet.get_fees_for_proofs(proofs)
if token_unit == "sat":
fee_msat = (fee_reserve + input_fees) * 1000
else:
fee_msat = fee_reserve + input_fees
amount_msat_after_fee = amount_msat - fee_msat
if settings.primary_mint_unit == "sat":
minted_amount = int(amount_msat_after_fee // 1000)
else:
minted_amount = int(amount_msat_after_fee)
total_fees = fee_reserve + input_fees
minted_amount = _net_minted_amount(amount_msat, token_unit, total_fees)
if minted_amount <= 0:
raise ValueError(f"Fees ({fee_reserve + input_fees} {token_unit}) exceed token amount")
raise ValueError(f"Fees ({total_fees} {token_unit}) exceed token amount")
logger.info(
"swap_to_primary_mint: fee estimation result",
extra={
"token_amount_sat": amount_msat // 1000,
"estimated_fee_sat": fee_msat // 1000,
"estimated_fee": total_fees,
"estimated_fee_unit": token_unit,
"input_fees": input_fees,
"minted_amount": minted_amount,
"minted_unit": settings.primary_mint_unit,
@@ -186,6 +403,8 @@ async def _calculate_swap_amount(
"swap_to_primary_mint: fee estimation failed",
extra={"error": str(e)},
)
if is_mint_connection_error(e):
raise MintConnectionError("Cashu mint is unreachable") from e
raise ValueError(f"Failed to estimate fees: {e}") from e
@@ -213,10 +432,9 @@ async def swap_to_primary_mint(
amount_msat = token_amount
else:
raise ValueError("Invalid unit")
primary_wallet = await get_wallet(settings.primary_mint, settings.primary_mint_unit)
# If the token is already from the primary mint, we don't need to swap
# and we definitely don't want to calculate or pay fees.
# If the token is already from the primary mint, we don't need a cross-mint
# swap — redeem it same-mint. There's no melt/Lightning fee, but the mint's
# NUT-02 input fee still applies; _redeem_same_mint accounts for it.
if token_obj.mint == settings.primary_mint:
logger.info(
"swap_to_primary_mint: token already on primary mint, skipping swap",
@@ -226,8 +444,9 @@ async def swap_to_primary_mint(
"unit": token_obj.unit,
},
)
await token_wallet.split(proofs=token_obj.proofs, amount=0, include_fees=True)
return token_amount, token_obj.unit, token_obj.mint
return await _redeem_same_mint(token_wallet, token_obj)
primary_wallet = await get_wallet(settings.primary_mint, settings.primary_mint_unit)
minted_amount = await _calculate_swap_amount(
amount_msat,
@@ -238,67 +457,123 @@ async def swap_to_primary_mint(
token_obj.proofs,
)
mint_quote = await primary_wallet.request_mint(minted_amount)
logger.info(
"swap_to_primary_mint: mint quote received",
extra={"mint_quote_id": mint_quote.quote},
)
# The estimate above is non-binding: the mint may demand a higher fee on the
# real quote or reject the melt outright. Retry the quote/melt cycle with the
# amount recomputed from the fees the mint actually demands.
observed_extra_fee = 0
attempt = 0
while True:
attempt += 1
mint_quote = await primary_wallet.request_mint(minted_amount)
logger.info(
"swap_to_primary_mint: mint quote received",
extra={"mint_quote_id": mint_quote.quote, "attempt": attempt},
)
melt_quote = await token_wallet.melt_quote(mint_quote.request)
input_fees = token_wallet.get_fees_for_proofs(token_obj.proofs)
total_needed = melt_quote.amount + melt_quote.fee_reserve + input_fees
logger.info(
"swap_to_primary_mint: melt quote received",
extra={
"melt_quote_id": melt_quote.quote,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"token_amount": token_amount,
},
)
if total_needed > token_amount:
logger.warning(
"swap_to_primary_mint: insufficient token amount for melt fees",
melt_quote = await token_wallet.melt_quote(mint_quote.request)
input_fees = token_wallet.get_fees_for_proofs(token_obj.proofs)
total_needed = melt_quote.amount + melt_quote.fee_reserve + input_fees
logger.info(
"swap_to_primary_mint: melt quote received",
extra={
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"shortfall": total_needed - token_amount,
"token_amount": token_amount,
"attempt": attempt,
},
)
raise ValueError(
f"Token amount ({token_amount} {token_obj.unit}) is insufficient to cover "
f"melt fees. Needed: {total_needed} {token_obj.unit} "
f"(amount: {melt_quote.amount} + fee: {melt_quote.fee_reserve} + input_fees: {input_fees})"
)
try:
_ = await token_wallet.melt(
proofs=token_obj.proofs,
invoice=mint_quote.request,
fee_reserve_sat=melt_quote.fee_reserve,
quote_id=melt_quote.quote,
)
except Exception as e:
logger.error(
"swap_to_primary_mint: melt failed",
extra={
"error": str(e),
"error_type": type(e).__name__,
"foreign_mint": token_obj.mint,
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"total_needed": total_needed,
},
)
raise ValueError(
f"Failed to melt token from foreign mint {token_obj.mint}: {e}"
) from e
if total_needed > token_amount:
recomputed = _net_minted_amount(
amount_msat,
token_obj.unit,
melt_quote.fee_reserve + input_fees + observed_extra_fee,
)
if attempt >= _MAX_SWAP_ATTEMPTS or recomputed <= 0:
logger.warning(
"swap_to_primary_mint: insufficient token amount for melt fees",
extra={
"token_amount": token_amount,
"melt_amount": melt_quote.amount,
"melt_fee_reserve": melt_quote.fee_reserve,
"input_fees": input_fees,
"total_needed": total_needed,
"shortfall": total_needed - token_amount,
"attempts": attempt,
},
)
raise ValueError(
f"Token amount ({token_amount} {token_obj.unit}) is insufficient to cover "
f"melt fees. Needed: {total_needed} {token_obj.unit} "
f"(amount: {melt_quote.amount} + fee: {melt_quote.fee_reserve} + input_fees: {input_fees})"
)
logger.warning(
"swap_to_primary_mint: melt quote exceeds token amount, retrying",
extra={
"total_needed": total_needed,
"token_amount": token_amount,
"retry_minted_amount": recomputed,
"attempt": attempt,
},
)
minted_amount = recomputed
continue
try:
_ = await token_wallet.melt(
proofs=token_obj.proofs,
invoice=mint_quote.request,
fee_reserve_sat=melt_quote.fee_reserve,
quote_id=melt_quote.quote,
)
except Exception as e:
# A down mint won't fix itself by retrying with a smaller amount.
if is_mint_connection_error(e):
logger.error(
"swap_to_primary_mint: melt failed — mint unreachable",
extra={"error": str(e), "foreign_mint": token_obj.mint},
)
raise MintConnectionError("Cashu mint is unreachable") from e
shortfall = _melt_insufficient_shortfall(e)
recomputed = 0
if shortfall is not None:
observed_extra_fee += shortfall
recomputed = _net_minted_amount(
amount_msat,
token_obj.unit,
melt_quote.fee_reserve + input_fees + observed_extra_fee,
)
if shortfall is None or attempt >= _MAX_SWAP_ATTEMPTS or recomputed <= 0:
logger.error(
"swap_to_primary_mint: melt failed",
extra={
"error": str(e),
"error_type": type(e).__name__,
"foreign_mint": token_obj.mint,
"token_amount": token_amount,
"melt_quote_id": melt_quote.quote,
"total_needed": total_needed,
"attempts": attempt,
},
)
raise ValueError(
f"Failed to melt token from foreign mint {token_obj.mint}: {e}"
) from e
logger.warning(
"swap_to_primary_mint: mint demanded more than quoted at melt, retrying",
extra={
"shortfall": shortfall,
"retry_minted_amount": recomputed,
"attempt": attempt,
},
)
minted_amount = recomputed
continue
break
logger.info(
"swap_to_primary_mint: melt succeeded, minting on primary",
@@ -337,21 +612,21 @@ async def swap_to_primary_mint(
# Recovery scan ran but did NOT restore the orphaned proofs
# (mint reports them as spent — they're stuck). Refuse to
# credit the API key balance for proofs we don't actually hold.
raise ValueError(
raise TokenConsumedError(
f"Swap recovery failed: mint signed outputs but proofs are "
f"unrecoverable (mint reports them spent). "
f"Expected {minted_amount}, recovered {balance_gained}. "
f"Local wallet DB ('.wallet/') state is corrupted — "
f"the counter for keyset is stuck at a bad index range."
)
except ValueError:
except TokenConsumedError:
raise
except Exception as recovery_err:
logger.error(
"swap_to_primary_mint: recovery failed",
extra={"error": str(recovery_err)},
)
raise ValueError(
raise TokenConsumedError(
f"Mint on primary failed and recovery unsuccessful: {e}"
) from e
else:
@@ -364,7 +639,10 @@ async def swap_to_primary_mint(
"mint_quote_id": mint_quote.quote,
},
)
raise
# Foreign proofs already melted (spent) — non-retryable.
raise TokenConsumedError(
"Mint on primary failed after successful melt"
) from e
logger.info(
"swap_to_primary_mint: completed successfully",
@@ -422,15 +700,33 @@ async def credit_balance(
extra={"old_balance": key.balance, "credit_amount": amount},
)
# Use atomic SQL UPDATE to prevent race conditions during concurrent topups
stmt = (
update(db.ApiKey)
.where(col(db.ApiKey.hashed_key) == key.hashed_key)
.values(balance=(db.ApiKey.balance) + amount)
)
await session.exec(stmt) # type: ignore[call-overload]
await session.commit()
await session.refresh(key)
# The token is already redeemed (spent) here, so any crediting failure
# is post-redemption and non-retryable — surface it as TokenConsumedError
# (a key that vanished mid-flight, or an unexpected DB fault), never a
# retryable/token-error taxonomy.
try:
# Atomic UPDATE to prevent race conditions during concurrent topups.
stmt = (
update(db.ApiKey)
.where(col(db.ApiKey.hashed_key) == key.hashed_key)
.values(balance=(db.ApiKey.balance) + amount)
)
result = await session.exec(stmt) # type: ignore[call-overload]
# If pruning removed this key after redemption, do not commit a no-op
# balance update and pretend the top-up succeeded.
if (getattr(result, "rowcount", 0) or 0) == 0:
raise TokenConsumedError(
"Token redeemed but the API key disappeared before the "
"credit could be recorded"
)
await session.commit()
await session.refresh(key)
except TokenConsumedError:
raise
except Exception as db_error:
raise TokenConsumedError(
"Token redeemed but crediting the balance failed"
) from db_error
logger.info(
"credit_balance: Balance updated successfully",
@@ -567,11 +863,19 @@ async def fetch_all_balances(
}
return error_result
# Build the set of mints to inspect. Received tokens are stored against
# ``primary_mint`` (which defaults to a real mint even when ``cashu_mints``
# is empty), so include it as a fallback — otherwise a node that accepts
# payments would still report empty balances when ``cashu_mints`` is unset.
mint_urls: list[str] = list(settings.cashu_mints)
if settings.primary_mint and settings.primary_mint not in mint_urls:
mint_urls.append(settings.primary_mint)
# Create tasks for all mint/unit combinations
async with db.create_session() as session:
tasks = [
fetch_balance(session, mint_url, unit)
for mint_url in settings.cashu_mints
for mint_url in mint_urls
for unit in units
]

View File

@@ -0,0 +1,229 @@
import json
from datetime import datetime, timedelta, timezone
from unittest.mock import patch
import pytest
from httpx import AsyncClient
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.admin import admin_sessions
from routstr.core.db import ModelRow, UpstreamProviderRow
from routstr.payment.cost_calculation import CostData, calculate_cost
from routstr.proxy import get_model_instance, reinitialize_upstreams
def _admin_headers() -> dict[str, str]:
token = "test-admin-cache-pricing-token"
admin_sessions[token] = int(
(datetime.now(timezone.utc) + timedelta(minutes=5)).timestamp()
)
return {"Authorization": f"Bearer {token}"}
def _model_payload(
provider_id: int,
*,
cache_read: float,
cache_write: float,
) -> dict[str, object]:
return {
"id": "custom-cache-model",
"name": "Custom Cache Model",
"description": "custom model with explicit cache pricing",
"created": 0,
"context_length": 128000,
"architecture": {
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
},
"pricing": {
"prompt": 1.4e-7,
"completion": 2.8e-7,
"input_cache_read": cache_read,
"input_cache_write": cache_write,
"request": 0.0,
"image": 0.0,
"web_search": 0.0,
"internal_reasoning": 0.0,
},
"per_request_limits": None,
"top_provider": None,
"upstream_provider_id": provider_id,
"canonical_slug": None,
"alias_ids": [],
"enabled": True,
"forwarded_model_id": "custom-cache-model",
}
@pytest.mark.integration
@pytest.mark.asyncio
async def test_admin_provider_model_api_persists_cache_pricing_on_create_and_update(
integration_client: AsyncClient,
integration_session: AsyncSession,
) -> None:
provider = UpstreamProviderRow(
provider_type="generic",
base_url="https://custom-upstream.example/v1",
api_key="test-key",
provider_fee=1.0,
)
integration_session.add(provider)
await integration_session.commit()
await integration_session.refresh(provider)
assert provider.id is not None
await reinitialize_upstreams()
headers = _admin_headers()
create_payload = _model_payload(
provider.id,
cache_read=2.8e-9,
cache_write=3.5e-9,
)
with patch("routstr.payment.models.sats_usd_price", return_value=1e-6):
create_response = await integration_client.post(
f"/admin/api/upstream-providers/{provider.id}/models",
headers=headers,
json=create_payload,
)
assert create_response.status_code == 200
create_body = create_response.json()
assert create_body["pricing"]["input_cache_read"] == pytest.approx(2.8e-9)
assert create_body["pricing"]["input_cache_write"] == pytest.approx(3.5e-9)
row = await integration_session.get(ModelRow, ("custom-cache-model", provider.id))
assert row is not None
stored_pricing = json.loads(row.pricing)
assert stored_pricing["input_cache_read"] == pytest.approx(2.8e-9)
assert stored_pricing["input_cache_write"] == pytest.approx(3.5e-9)
update_payload = _model_payload(
provider.id,
cache_read=1.25e-9,
cache_write=4.5e-9,
)
with patch("routstr.payment.models.sats_usd_price", return_value=1e-6):
update_response = await integration_client.post(
f"/admin/api/upstream-providers/{provider.id}/models",
headers=headers,
json=update_payload,
)
assert update_response.status_code == 200
update_body = update_response.json()
assert update_body["pricing"]["input_cache_read"] == pytest.approx(1.25e-9)
assert update_body["pricing"]["input_cache_write"] == pytest.approx(4.5e-9)
await integration_session.refresh(row)
updated_pricing = json.loads(row.pricing)
assert updated_pricing["input_cache_read"] == pytest.approx(1.25e-9)
assert updated_pricing["input_cache_write"] == pytest.approx(4.5e-9)
model = get_model_instance("custom-cache-model")
assert model is not None
assert model.sats_pricing is not None
assert model.sats_pricing.input_cache_read == pytest.approx(0.00125)
assert model.sats_pricing.input_cache_write == pytest.approx(0.0045)
with patch("routstr.payment.cost_calculation.sats_usd_price", return_value=1e-6):
cost = await calculate_cost(
{
"model": "custom-cache-model",
"usage": {
"prompt_tokens": 1000,
"completion_tokens": 100,
"prompt_tokens_details": {"cached_tokens": 800},
},
},
max_cost=1_000_000,
)
assert isinstance(cost, CostData)
assert cost.input_tokens == 200
assert cost.cache_read_input_tokens == 800
assert cost.cache_read_msats == 1000
assert cost.output_msats == 28000
assert cost.input_msats == 29000
assert cost.total_msats == 57000
@pytest.mark.integration
@pytest.mark.asyncio
async def test_upstream_response_cost_uses_model_cache_pricing(
integration_session: AsyncSession,
patched_db_engine: None,
) -> None:
"""A model's configured cache price must discount upstream cached-token usage."""
provider = UpstreamProviderRow(
provider_type="generic",
base_url="https://cache-priced-upstream.example/v1",
api_key="test-key",
provider_fee=1.0,
)
integration_session.add(provider)
await integration_session.commit()
await integration_session.refresh(provider)
assert provider.id is not None
row = ModelRow(
id="cache-priced-model",
name="Cache Priced Model",
description="model seeded with explicit cache pricing",
created=0,
context_length=128000,
architecture=json.dumps(
{
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
}
),
pricing=json.dumps(
{
"prompt": 1.4e-7,
"completion": 2.8e-7,
"input_cache_read": 1.25e-9,
"input_cache_write": 4.5e-9,
}
),
upstream_provider_id=provider.id,
enabled=True,
forwarded_model_id="cache-priced-model",
)
integration_session.add(row)
await integration_session.commit()
with patch("routstr.payment.models.sats_usd_price", return_value=1e-6):
await reinitialize_upstreams()
with patch("routstr.payment.cost_calculation.sats_usd_price", return_value=1e-6):
cost = await calculate_cost(
{
"model": "cache-priced-model",
"usage": {
"prompt_tokens": 1000,
"completion_tokens": 100,
"prompt_tokens_details": {"cached_tokens": 800},
},
},
max_cost=1_000_000,
)
assert isinstance(cost, CostData)
# prompt 1.4e-7 USD/token -> 0.14 sats/token -> 140 msats/token
# cache 1.25e-9 USD/token -> 0.00125 sats/token -> 1.25 msats/token
# completion 2.8e-7 USD/token -> 0.28 sats/token -> 280 msats/token
assert cost.input_tokens == 200
assert cost.cache_read_input_tokens == 800
assert cost.cache_read_msats == 1000
assert cost.input_msats == 29000
assert cost.output_msats == 28000
assert cost.total_msats == 57000

View File

@@ -69,9 +69,14 @@ async def test_wallet_info_child_key_no_child_keys(
info_response = await integration_client.get("/v1/wallet/info")
assert info_response.status_code == 200
info_data = info_response.json()
parent_key = authenticated_client._test_api_key # type: ignore[attr-defined]
parent_key_hash = parent_key.removeprefix("sk-")
assert info_data["is_child"] is True
assert "child_keys" not in info_data
assert "parent_key" not in info_data
assert info_data["parent_key_preview"] == parent_key_hash[:8] + "..."
assert info_data["parent_key_preview"] not in {parent_key, parent_key_hash}
@pytest.mark.integration

View File

@@ -0,0 +1,146 @@
"""Regression tests for charging after stale reservation cleanup."""
import time
import uuid
from unittest.mock import patch
import pytest
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.db import ApiKey
from routstr.payment.cost_calculation import CostData
def _make_key(balance: int, reserved: int) -> ApiKey:
return ApiKey(
hashed_key=f"test_{uuid.uuid4().hex}",
balance=balance,
reserved_balance=reserved,
total_spent=0,
total_requests=1,
)
def _cost_data(total_msats: int) -> CostData:
return CostData(
base_msats=0,
input_msats=total_msats // 2,
output_msats=total_msats - total_msats // 2,
total_msats=total_msats,
total_usd=0.0,
input_tokens=100,
output_tokens=100,
)
@pytest.mark.asyncio
async def test_overrun_charges_after_reservation_swept(
integration_session: AsyncSession,
) -> None:
"""Overrun finalize must charge even when the reservation was already released."""
from routstr.auth import adjust_payment_for_tokens
deducted_max_cost = 990 # discounted reservation
actual_token_cost = 1000 # actual cost overruns the reservation
# Sweeper has zeroed reserved_balance but left balance untouched.
key = _make_key(balance=1000, reserved=0)
integration_session.add(key)
await integration_session.commit()
response_data = {
"model": "test-model",
"usage": {"prompt_tokens": 100, "completion_tokens": 100},
}
with patch(
"routstr.auth.calculate_cost",
return_value=_cost_data(actual_token_cost),
):
await adjust_payment_for_tokens(
key, response_data, integration_session, deducted_max_cost
)
await integration_session.refresh(key)
assert key.total_spent == actual_token_cost, (
f"Request was not billed (total_spent={key.total_spent}) — free response bug"
)
assert key.balance == 1000 - actual_token_cost, (
f"Balance not charged: {key.balance}"
)
assert key.balance >= 0
assert key.reserved_balance == 0
@pytest.mark.asyncio
async def test_free_response_path_closed_end_to_end(
integration_session: AsyncSession,
patched_db_engine: None,
) -> None:
"""A reservation released by the real sweeper must not yield a free response."""
from routstr.auth import adjust_payment_for_tokens, pay_for_request
from routstr.core.db import create_session, release_stale_reservations
deducted_max_cost = 990
actual_token_cost = 1000
key_hash = f"test_sweep_{uuid.uuid4().hex}"
async with create_session() as session:
session.add(
ApiKey(
hashed_key=key_hash,
balance=1000,
reserved_balance=0,
total_spent=0,
total_requests=0,
)
)
await session.commit()
# Reserve the request, then backdate reserved_at so the sweeper treats it as
# stale (simulates a stream that outlived stale_reservation_timeout_seconds).
async with create_session() as session:
key = await session.get(ApiKey, key_hash)
assert key is not None
await pay_for_request(key, deducted_max_cost, session)
await session.refresh(key)
assert key.reserved_balance == deducted_max_cost
key.reserved_at = int(time.time()) - 10_000
session.add(key)
await session.commit()
# Sweeper releases the stale reservation without charging.
async with create_session() as session:
released = await release_stale_reservations(session, max_age_seconds=300)
assert released == 1
async with create_session() as session:
key = await session.get(ApiKey, key_hash)
assert key is not None
assert key.reserved_balance == 0, "Precondition: sweeper zeroed the reservation"
response_data = {
"model": "test-model",
"usage": {"prompt_tokens": 100, "completion_tokens": 100},
}
with patch(
"routstr.auth.calculate_cost",
return_value=_cost_data(actual_token_cost),
):
await adjust_payment_for_tokens(
key, response_data, session, deducted_max_cost
)
async with create_session() as session:
final = await session.get(ApiKey, key_hash)
assert final is not None
assert final.total_spent == actual_token_cost, (
f"Free response: total_spent={final.total_spent}, expected {actual_token_cost}"
)
assert final.balance == 1000 - actual_token_cost, (
f"Balance not charged after sweep: {final.balance}"
)
assert final.balance >= 0
assert final.reserved_balance == 0

View File

@@ -1,12 +1,14 @@
import asyncio
import time
from datetime import datetime, timedelta
import pytest
from fastapi import HTTPException
from sqlmodel import select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.auth import pay_for_request
from routstr.core.db import ApiKey
from routstr.core.db import ApiKey, create_session
@pytest.mark.asyncio
@@ -120,6 +122,270 @@ async def test_periodic_key_reset_job(integration_session: AsyncSession) -> None
assert key2.total_spent == 0
@pytest.mark.asyncio
async def test_balance_limit_enforced_atomically_under_concurrency(
patched_db_engine: None,
) -> None:
parent_hash = "parent_limit_atomic"
child_hash = "child_limit_atomic"
cost = 300
async with create_session() as session:
parent = ApiKey(hashed_key=parent_hash, balance=10000)
child = ApiKey(
hashed_key=child_hash,
balance=0,
parent_key_hash=parent_hash,
balance_limit=cost,
total_spent=0,
)
session.add(parent)
session.add(child)
await session.commit()
results: list[str] = []
async def attempt() -> None:
async with create_session() as session:
fresh_child = await session.get(ApiKey, child_hash)
assert fresh_child is not None
try:
await pay_for_request(fresh_child, cost, session)
results.append("success")
except HTTPException as exc:
assert exc.status_code == 402
results.append("blocked")
await asyncio.gather(attempt(), attempt())
assert sorted(results) == ["blocked", "success"], (
f"Expected exactly one success and one 402, got: {results}"
)
async with create_session() as session:
final_child = await session.get(ApiKey, child_hash)
assert final_child is not None
assert final_child.reserved_balance == cost, (
f"Child reserved_balance should equal one reservation, "
f"got {final_child.reserved_balance}"
)
@pytest.mark.asyncio
async def test_parallel_payments_with_parent_and_child_key(
patched_db_engine: None,
) -> None:
parent_hash = "parent_parallel_mixed"
child_hash = "child_parallel_mixed"
cost = 300
async with create_session() as session:
parent = ApiKey(hashed_key=parent_hash, balance=10000)
child = ApiKey(
hashed_key=child_hash,
balance=0,
parent_key_hash=parent_hash,
balance_limit=2 * cost,
)
session.add(parent)
session.add(child)
await session.commit()
async def attempt(key_hash: str) -> str:
async with create_session() as session:
fresh_key = await session.get(ApiKey, key_hash)
assert fresh_key is not None
try:
await pay_for_request(fresh_key, cost, session)
return "success"
except HTTPException as exc:
assert exc.status_code == 402
return "blocked"
results = await asyncio.gather(attempt(parent_hash), attempt(child_hash))
assert results == ["success", "success"], (
f"Both parent and child payments should succeed, got: {results}"
)
async with create_session() as session:
final_parent = await session.get(ApiKey, parent_hash)
final_child = await session.get(ApiKey, child_hash)
assert final_parent is not None
assert final_child is not None
# Both requests bill the parent; only the child request reserves on the child.
assert final_parent.reserved_balance == 2 * cost
assert final_parent.total_requests == 2
assert final_child.reserved_balance == cost
assert final_child.total_requests == 1
@pytest.mark.asyncio
async def test_balance_limit_with_existing_total_spent_under_concurrency(
patched_db_engine: None,
) -> None:
parent_hash = "parent_total_spent"
child_hash = "child_total_spent"
cost = 300
async with create_session() as session:
parent = ApiKey(hashed_key=parent_hash, balance=10000)
# 700 already spent against a 1000 limit: only one more 300 request fits.
child = ApiKey(
hashed_key=child_hash,
balance=0,
parent_key_hash=parent_hash,
balance_limit=1000,
total_spent=700,
)
session.add(parent)
session.add(child)
await session.commit()
results: list[str] = []
async def attempt() -> None:
async with create_session() as session:
fresh_child = await session.get(ApiKey, child_hash)
assert fresh_child is not None
try:
await pay_for_request(fresh_child, cost, session)
results.append("success")
except HTTPException as exc:
assert exc.status_code == 402
results.append("blocked")
await asyncio.gather(attempt(), attempt())
assert sorted(results) == ["blocked", "success"], (
f"Expected exactly one success and one 402, got: {results}"
)
async with create_session() as session:
final_child = await session.get(ApiKey, child_hash)
assert final_child is not None
assert final_child.reserved_balance == cost
assert final_child.total_spent == 700
@pytest.mark.asyncio
async def test_balance_limit_with_existing_reserved_balance(
patched_db_engine: None,
) -> None:
parent_hash = "parent_reserved_set"
blocked_hash = "child_reserved_blocked"
allowed_hash = "child_reserved_allowed"
cost = 300
async with create_session() as session:
parent = ApiKey(hashed_key=parent_hash, balance=10000)
# 800 already reserved against a 1000 limit: another 300 must be rejected.
blocked_child = ApiKey(
hashed_key=blocked_hash,
balance=0,
parent_key_hash=parent_hash,
balance_limit=1000,
reserved_balance=800,
)
# 500 reserved against a 1000 limit: another 300 still fits.
allowed_child = ApiKey(
hashed_key=allowed_hash,
balance=0,
parent_key_hash=parent_hash,
balance_limit=1000,
reserved_balance=500,
)
session.add(parent)
session.add(blocked_child)
session.add(allowed_child)
await session.commit()
async with create_session() as session:
fresh_blocked = await session.get(ApiKey, blocked_hash)
assert fresh_blocked is not None
with pytest.raises(HTTPException) as exc_info:
await pay_for_request(fresh_blocked, cost, session)
assert exc_info.value.status_code == 402
async with create_session() as session:
fresh_allowed = await session.get(ApiKey, allowed_hash)
assert fresh_allowed is not None
await pay_for_request(fresh_allowed, cost, session)
async with create_session() as session:
final_blocked = await session.get(ApiKey, blocked_hash)
final_allowed = await session.get(ApiKey, allowed_hash)
final_parent = await session.get(ApiKey, parent_hash)
assert final_blocked is not None
assert final_allowed is not None
assert final_parent is not None
assert final_blocked.reserved_balance == 800, "Rejected request must not reserve"
assert final_blocked.total_requests == 0
assert final_allowed.reserved_balance == 500 + cost
assert final_allowed.total_requests == 1
# Only the allowed request should have billed the parent.
assert final_parent.reserved_balance == cost
assert final_parent.total_requests == 1
@pytest.mark.asyncio
async def test_child_reservation_discarded_when_parent_balance_depleted(
patched_db_engine: None,
) -> None:
parent_hash = "parent_depleted"
child_hash = "child_depleted"
cost = 300
async with create_session() as session:
# Parent can only afford one request; child has no balance_limit.
parent = ApiKey(hashed_key=parent_hash, balance=cost)
child = ApiKey(
hashed_key=child_hash,
balance=0,
parent_key_hash=parent_hash,
)
session.add(parent)
session.add(child)
await session.commit()
results: list[str] = []
async def attempt() -> None:
async with create_session() as session:
fresh_child = await session.get(ApiKey, child_hash)
assert fresh_child is not None
try:
await pay_for_request(fresh_child, cost, session)
results.append("success")
except HTTPException as exc:
assert exc.status_code == 402
results.append("blocked")
await asyncio.gather(attempt(), attempt())
assert sorted(results) == ["blocked", "success"], (
f"Expected exactly one success and one 402, got: {results}"
)
async with create_session() as session:
final_parent = await session.get(ApiKey, parent_hash)
final_child = await session.get(ApiKey, child_hash)
assert final_parent is not None
assert final_child is not None
assert final_parent.reserved_balance == cost
# The failed request must not leave a committed reservation on the child.
assert final_child.reserved_balance == cost, (
f"Child reserved_balance should reflect only the successful request, "
f"got {final_child.reserved_balance}"
)
assert final_child.total_requests == 1
@pytest.mark.asyncio
async def test_refund_does_not_delete_key(integration_session: AsyncSession) -> None:
# This requires mocking the router call or testing the logic in balance.py

View File

@@ -0,0 +1,224 @@
import time
from types import TracebackType
from typing import Any
from unittest.mock import patch
import pytest
from httpx import AsyncClient
from routstr.core.admin import admin_sessions
from routstr.core.db import ModelRow, UpstreamProviderRow
@pytest.mark.integration
@pytest.mark.asyncio
async def test_model_test_endpoint_requires_admin_auth(
integration_client: AsyncClient,
) -> None:
with patch("httpx.AsyncClient") as mock_async_client:
response = await integration_client.post(
"/api/models/test",
json={
"model_id": "model-a",
"endpoint_type": "chat-completions",
"request_data": {"messages": []},
},
)
assert response.status_code == 403
mock_async_client.assert_not_called()
@pytest.mark.integration
@pytest.mark.asyncio
async def test_model_test_endpoint_rejects_unsupported_endpoint_type(
integration_client: AsyncClient,
integration_session: Any,
) -> None:
admin_token = "test-admin-token-model-test"
admin_sessions[admin_token] = int(time.time()) + 3600
integration_client.headers["Authorization"] = f"Bearer {admin_token}"
provider = UpstreamProviderRow(
provider_type="custom",
base_url="https://api.example.com/v1",
api_key="sk-upstream-test",
enabled=True,
provider_fee=1.01,
)
integration_session.add(provider)
await integration_session.commit()
await integration_session.refresh(provider)
assert provider.id is not None
model = ModelRow(
id="model-a",
name="Model A",
created=1,
description="desc",
context_length=100,
architecture='{"modality": "text", "input_modalities": ["text"], "output_modalities": ["text"], "tokenizer": "tiktoken", "instruct_type": "chat"}',
pricing='{"prompt": 1.0, "completion": 1.0}',
upstream_provider_id=provider.id,
enabled=True,
)
integration_session.add(model)
await integration_session.commit()
try:
with patch("httpx.AsyncClient") as mock_async_client:
response = await integration_client.post(
"/api/models/test",
json={
"model_id": "model-a",
"endpoint_type": "../../abuse",
"request_data": {"messages": []},
},
)
assert response.status_code == 400
assert response.json()["detail"] == "Unsupported endpoint_type"
mock_async_client.assert_not_called()
finally:
admin_sessions.pop(admin_token, None)
@pytest.mark.integration
@pytest.mark.asyncio
async def test_model_test_endpoint_rejects_oversized_request_data(
integration_client: AsyncClient,
integration_session: Any,
) -> None:
admin_token = "test-admin-token-model-test-oversized"
admin_sessions[admin_token] = int(time.time()) + 3600
integration_client.headers["Authorization"] = f"Bearer {admin_token}"
provider = UpstreamProviderRow(
provider_type="custom",
base_url="https://api.example.com/v1",
api_key="sk-upstream-test",
enabled=True,
provider_fee=1.01,
)
integration_session.add(provider)
await integration_session.commit()
await integration_session.refresh(provider)
assert provider.id is not None
model = ModelRow(
id="model-a",
name="Model A",
created=1,
description="desc",
context_length=100,
architecture='{"modality": "text", "input_modalities": ["text"], "output_modalities": ["text"], "tokenizer": "tiktoken", "instruct_type": "chat"}',
pricing='{"prompt": 1.0, "completion": 1.0}',
upstream_provider_id=provider.id,
enabled=True,
)
integration_session.add(model)
await integration_session.commit()
oversized = "x" * (64 * 1024 + 1)
try:
with patch("httpx.AsyncClient") as mock_async_client:
response = await integration_client.post(
"/api/models/test",
json={
"model_id": "model-a",
"endpoint_type": "chat-completions",
"request_data": {"blob": oversized},
},
)
assert response.status_code == 413
assert response.json()["detail"] == "request_data too large"
mock_async_client.assert_not_called()
finally:
admin_sessions.pop(admin_token, None)
@pytest.mark.integration
@pytest.mark.asyncio
async def test_model_test_endpoint_admin_uses_allowed_upstream_path(
integration_client: AsyncClient,
integration_session: Any,
) -> None:
admin_token = "test-admin-token-model-test-success"
admin_sessions[admin_token] = int(time.time()) + 3600
integration_client.headers["Authorization"] = f"Bearer {admin_token}"
provider = UpstreamProviderRow(
provider_type="custom",
base_url="https://api.example.com/v1",
api_key="sk-upstream-test",
enabled=True,
provider_fee=1.01,
)
integration_session.add(provider)
await integration_session.commit()
await integration_session.refresh(provider)
assert provider.id is not None
model = ModelRow(
id="model-a",
name="Model A",
created=1,
description="desc",
context_length=100,
architecture='{"modality": "text", "input_modalities": ["text"], "output_modalities": ["text"], "tokenizer": "tiktoken", "instruct_type": "chat"}',
pricing='{"prompt": 1.0, "completion": 1.0}',
upstream_provider_id=provider.id,
enabled=True,
forwarded_model_id="upstream-model-a",
)
integration_session.add(model)
await integration_session.commit()
class MockResponse:
status_code = 200
text = '{"ok": true}'
def json(self) -> dict[str, bool]:
return {"ok": True}
class MockAsyncClient:
async def __aenter__(self) -> "MockAsyncClient":
return self
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc: BaseException | None,
tb: TracebackType | None,
) -> None:
return None
async def post(
self, url: str, json: dict[str, Any], headers: dict[str, str]
) -> MockResponse:
assert url == "https://api.example.com/v1/chat/completions"
assert json["model"] == "upstream-model-a"
assert headers["Authorization"] == "Bearer sk-upstream-test"
return MockResponse()
try:
with patch("httpx.AsyncClient", return_value=MockAsyncClient()):
response = await integration_client.post(
"/api/models/test",
json={
"model_id": "model-a",
"endpoint_type": "chat-completions",
"request_data": {"messages": []},
},
)
assert response.status_code == 200
assert response.json() == {
"success": True,
"data": {"ok": True},
"status_code": 200,
}
finally:
admin_sessions.pop(admin_token, None)

View File

@@ -0,0 +1,128 @@
"""A live upstream provider resolves its OWN database row by stable identity
(its primary key), not by its mutable/secret ``api_key``.
Today ``from_db_row`` drops ``provider_row.id`` and the two self-referential
paths — PPQ.AI's insufficient-balance self-disable and the base
``refresh_models_cache`` — re-find their own row with
``WHERE base_url == self.base_url AND api_key == self.api_key``. That uses a
rotatable secret as a self-handle: the moment the row's key changes underneath a
live object (a rotation racing an in-flight request), the object can no longer
find itself. These tests pin the invariant that a provider looks itself up by
identity, so the lookup survives a key change (and, later, key encryption).
"""
from unittest.mock import AsyncMock, patch
import pytest
from routstr.core.db import UpstreamProviderRow
from routstr.upstream.ppqai import PPQAIUpstreamProvider
@pytest.mark.integration
@pytest.mark.asyncio
async def test_provider_object_carries_its_persistent_identity(
integration_session: object,
patched_db_engine: None,
) -> None:
"""``from_db_row`` gives the in-memory object its row's identity (``db_id``)."""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
provider = PPQAIUpstreamProvider.from_db_row(row)
assert provider is not None
assert provider.db_id == row.id
@pytest.mark.integration
@pytest.mark.asyncio
async def test_self_disable_targets_own_row_after_key_rotation(
integration_session: object,
patched_db_engine: None,
) -> None:
"""PPQ.AI self-disable must disable *its* row even after the key rotated.
RED (current): the object holds the pre-rotation key, so the
``(base_url, api_key)`` lookup misses the row → the provider is never
disabled. GREEN: lookup by ``id`` finds it and disables it.
"""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
provider = PPQAIUpstreamProvider.from_db_row(row) # captures sk-original
assert provider is not None
# Key is rotated in the DB while `provider` is still live.
row.api_key = "sk-rotated"
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
with patch("routstr.proxy.reinitialize_upstreams", new=AsyncMock()):
await provider.on_upstream_error_redirect(402, "Insufficient balance")
await integration_session.refresh(row) # type: ignore[attr-defined]
assert row.enabled is False
@pytest.mark.integration
@pytest.mark.asyncio
async def test_refresh_models_cache_finds_own_row_after_key_rotation(
integration_session: object,
patched_db_engine: None,
) -> None:
"""``refresh_models_cache`` must resolve its own row after a key rotation.
``refresh_models_cache`` swallows every exception (it only logs), so the
observable proof it found its row is that it reaches ``list_models`` — which
is called with the row's ``id`` only *after* the row is resolved. RED
(current): the stale-key ``(base_url, api_key)`` lookup returns nothing, the
method raises ``404`` internally and returns before ``list_models`` is ever
called. GREEN: lookup by ``id`` finds the row and ``list_models`` runs for
that ``id``.
"""
row = UpstreamProviderRow(
provider_type="ppqai",
base_url="https://api.ppq.ai",
api_key="sk-original",
enabled=True,
provider_fee=1.0,
)
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
await integration_session.refresh(row) # type: ignore[attr-defined]
row_id = row.id
provider = PPQAIUpstreamProvider.from_db_row(row) # captures sk-original
assert provider is not None
row.api_key = "sk-rotated"
integration_session.add(row) # type: ignore[attr-defined]
await integration_session.commit() # type: ignore[attr-defined]
list_models_mock = AsyncMock(return_value=[])
with (
patch.object(provider, "fetch_models", new=AsyncMock(return_value=[])),
patch("routstr.upstream.base.list_models", new=list_models_mock),
):
await provider.refresh_models_cache()
list_models_mock.assert_awaited_once()
assert list_models_mock.await_args is not None
assert list_models_mock.await_args.kwargs["upstream_id"] == row_id

View File

@@ -0,0 +1,283 @@
"""
Tests for prune_dead_api_keys — the janitor that removes provably-dead 0/0/0
API keys (funded keys fully refunded/expired without ever being used, plus bare
orphans), while protecting keys that are still meaningful.
"""
import time
import uuid
from collections.abc import Awaitable, Callable
from typing import Any, cast
import pytest
from sqlalchemy.sql.dml import Update
from sqlmodel import col, update
from routstr.core.db import (
ApiKey,
CashuTransaction,
LightningInvoice,
create_session,
prune_dead_api_keys,
)
OLD = 100 # min_age_seconds used by the tests
NOW = int(time.time())
LONG_AGO = NOW - 10_000 # well past the grace period
async def _exists(key_hash: str) -> bool:
async with create_session() as session:
return (await session.get(ApiKey, key_hash)) is not None
def _dead_key(created_at: int | None) -> ApiKey:
return ApiKey(
hashed_key=f"dead_{uuid.uuid4().hex}",
balance=0,
reserved_balance=0,
total_spent=0,
total_requests=0,
created_at=created_at,
)
@pytest.mark.asyncio
async def test_prunes_old_refunded_zero_key(patched_db_engine: None) -> None:
"""A funded-then-refunded key (0/0/0, NULL parent, old) is pruned."""
key = _dead_key(LONG_AGO)
async with create_session() as session:
session.add(key)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 1
assert not await _exists(key.hashed_key)
@pytest.mark.asyncio
async def test_grace_period_protects_fresh_key(patched_db_engine: None) -> None:
"""A dead-looking but recently created key is protected by the grace period."""
key = _dead_key(int(time.time()))
async with create_session() as session:
session.add(key)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 0
assert await _exists(key.hashed_key)
@pytest.mark.asyncio
async def test_used_key_never_pruned(patched_db_engine: None) -> None:
"""Keys with any spend/requests or live balance are never pruned."""
spent = _dead_key(LONG_AGO)
spent.total_spent = 1
requested = _dead_key(LONG_AGO)
requested.total_requests = 1
funded = _dead_key(LONG_AGO)
funded.balance = 1000
reserved = _dead_key(LONG_AGO)
reserved.reserved_balance = 500
async with create_session() as session:
for k in (spent, requested, funded, reserved):
session.add(k)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 0
for k in (spent, requested, funded, reserved):
assert await _exists(k.hashed_key)
@pytest.mark.asyncio
async def test_parent_and_child_keys_are_not_pruned(
patched_db_engine: None,
) -> None:
"""Pruning must not orphan child keys or delete valid children."""
parent = _dead_key(LONG_AGO)
child = ApiKey(
hashed_key=f"child_{uuid.uuid4().hex}",
balance=0,
reserved_balance=0,
total_spent=0,
total_requests=0,
created_at=LONG_AGO,
parent_key_hash=parent.hashed_key,
)
async with create_session() as session:
session.add(parent)
session.add(child)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 0
assert await _exists(parent.hashed_key)
assert await _exists(child.hashed_key)
@pytest.mark.asyncio
async def test_pending_invoice_protects_key(patched_db_engine: None) -> None:
"""A key referenced by a pending topup invoice is never pruned mid-topup."""
key = _dead_key(LONG_AGO)
invoice = LightningInvoice(
id=f"inv_{uuid.uuid4().hex}",
bolt11=f"lnbc_{uuid.uuid4().hex}",
amount_sats=10,
description="topup",
payment_hash=uuid.uuid4().hex,
status="pending",
api_key_hash=key.hashed_key,
purpose="topup",
expires_at=NOW + 10_000,
)
async with create_session() as session:
session.add(key)
session.add(invoice)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 0
assert await _exists(key.hashed_key)
@pytest.mark.asyncio
async def test_paid_invoice_does_not_protect_key(patched_db_engine: None) -> None:
"""A settled (non-pending) invoice does not keep a dead key alive."""
key = _dead_key(LONG_AGO)
invoice = LightningInvoice(
id=f"inv_{uuid.uuid4().hex}",
bolt11=f"lnbc_{uuid.uuid4().hex}",
amount_sats=10,
description="topup",
payment_hash=uuid.uuid4().hex,
status="paid",
api_key_hash=key.hashed_key,
purpose="topup",
expires_at=NOW - 1,
)
async with create_session() as session:
session.add(key)
session.add(invoice)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 1
assert not await _exists(key.hashed_key)
@pytest.mark.asyncio
async def test_key_that_becomes_meaningful_during_prune_survives(
patched_db_engine: None, monkeypatch: pytest.MonkeyPatch
) -> None:
"""Revalidate before unlink/delete so a late top-up cannot be pruned."""
key = _dead_key(LONG_AGO)
txn = CashuTransaction(
id=uuid.uuid4().hex,
token="cashuABC",
amount=21,
unit="sat",
type="in",
source="apikey",
api_key_hashed_key=key.hashed_key,
)
async with create_session() as session:
session.add(key)
session.add(txn)
await session.commit()
async with create_session() as session:
original_exec = cast(Callable[..., Awaitable[Any]], session.exec)
topped_up = False
async def exec_with_late_topup(
statement: Any, *args: Any, **kwargs: Any
) -> Any:
nonlocal topped_up
if (
not topped_up
and isinstance(statement, Update)
and getattr(statement.table, "name", None) == "cashu_transactions"
):
topped_up = True
async with create_session() as topup_session:
await topup_session.exec( # type: ignore[call-overload]
update(ApiKey)
.where(col(ApiKey.hashed_key) == key.hashed_key)
.values(balance=42)
)
await topup_session.commit()
return await original_exec(statement, *args, **kwargs)
monkeypatch.setattr(session, "exec", exec_with_late_topup)
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 0
assert await _exists(key.hashed_key)
async with create_session() as session:
surviving = await session.get(CashuTransaction, txn.id)
assert surviving is not None
assert surviving.api_key_hashed_key == key.hashed_key
@pytest.mark.asyncio
async def test_transaction_audit_trail_preserved(patched_db_engine: None) -> None:
"""Pruning a refunded key keeps its cashu_transactions, unlinked from the key."""
key = _dead_key(LONG_AGO)
txn = CashuTransaction(
id=uuid.uuid4().hex,
token="cashuABC",
amount=21,
unit="sat",
type="in",
source="apikey",
api_key_hashed_key=key.hashed_key,
)
async with create_session() as session:
session.add(key)
session.add(txn)
await session.commit()
async with create_session() as session:
pruned = await prune_dead_api_keys(session, OLD)
assert pruned == 1
assert not await _exists(key.hashed_key)
async with create_session() as session:
surviving = await session.get(CashuTransaction, txn.id)
assert surviving is not None, "Financial audit row must survive key deletion"
assert surviving.api_key_hashed_key is None, "Link must be nulled, not dangling"
assert surviving.amount == 21
@pytest.mark.asyncio
async def test_periodic_prune_disabled_returns_immediately(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Non-positive intervals disable the janitor."""
from unittest.mock import AsyncMock
from routstr import auth
from routstr.core.settings import settings
monkeypatch.setattr(settings, "dead_key_prune_interval_seconds", 0)
sleep_mock = AsyncMock()
monkeypatch.setattr(auth.asyncio, "sleep", sleep_mock)
await auth.periodic_dead_key_prune()
sleep_mock.assert_not_called()

View File

@@ -0,0 +1,194 @@
"""
Integration tests for reactive swap fee retries via the wallet topup endpoint.
Foreign-mint tokens are swapped to the primary mint using the foreign mint's
melt quote, whose fee_reserve is a non-binding estimate (NUT-05): the mint may
demand more when re-quoting or at melt execution. These tests cover the
endpoint behaviour in those cases:
1. The mint demands one sat more at melt time than every quote reported
(the mint.cubabitcoin.org incident): the swap retries with a smaller
invoice and the topup succeeds, crediting the recomputed amount.
2. The real melt quote reports a higher fee_reserve than the estimate: the
swap re-quotes from the observed fee and the topup succeeds.
3. The mint escalates its fee demands on every attempt: the retry budget is
exhausted and the endpoint returns 400 with a clear error (never 500),
without ever executing a melt.
"""
from collections.abc import Callable
from unittest.mock import AsyncMock, Mock, patch
import pytest
from httpx import AsyncClient, Response
from routstr.core.settings import settings
# Captured at collection time, before the integration_app fixture replaces it
# with the testmint stub that bypasses swapping (see conftest.py).
from routstr.wallet import recieve_token as _real_recieve_token
PRIMARY_MINT = "http://primary:3338"
def _make_swap_mocks(
token_amount: int,
fee_reserves: list[int],
input_fees: int = 0,
mint_url: str = "http://foreign-mint:3338",
) -> tuple[Mock, Mock, Mock]:
"""Return (token, token_wallet, primary_wallet) mocks that act like a mint.
Mint quotes pass the requested amount through their ``request`` field and
melt quotes echo that amount back, so the mocks stay consistent for
whatever amounts the implementation requests. ``fee_reserves`` supplies the
fee_reserve of each successive melt quote (the first serves the estimation
pass); requesting more quotes than provided fails the test.
"""
mock_token = Mock()
mock_token.mint = mint_url
mock_token.unit = "sat"
mock_token.amount = token_amount
mock_token.keysets = ["keyset1"]
mock_token.proofs = [Mock(amount=token_amount)]
mock_token_wallet = Mock()
mock_token_wallet.load_mint = AsyncMock()
mock_token_wallet.load_proofs = AsyncMock()
mock_token_wallet.get_fees_for_proofs = Mock(return_value=input_fees)
mock_primary_wallet = Mock()
mock_primary_wallet.load_mint = AsyncMock()
mock_primary_wallet.load_proofs = AsyncMock()
mock_primary_wallet.available_balance = Mock(amount=0)
mock_primary_wallet.mint = AsyncMock(return_value=Mock())
fees = iter(fee_reserves)
def _next_fee() -> int:
try:
return next(fees)
except StopIteration:
raise AssertionError(
"more melt quotes requested than fee_reserves provided"
) from None
mock_primary_wallet.request_mint = AsyncMock(
side_effect=lambda amount: Mock(quote=f"mint_quote_{amount}", request=amount)
)
mock_token_wallet.melt_quote = AsyncMock(
side_effect=lambda invoice: Mock(
quote=f"melt_quote_{invoice}", amount=invoice, fee_reserve=_next_fee()
)
)
mock_token_wallet.melt = AsyncMock(return_value=Mock())
return mock_token, mock_token_wallet, mock_primary_wallet
def _wallet_router(primary_wallet: Mock, token_wallet: Mock) -> Callable[..., Mock]:
"""Route get_wallet calls to the primary or foreign wallet mock by URL."""
def fake_get_wallet(mint_url: str, unit: str = "sat", load: bool = True) -> Mock:
return primary_wallet if mint_url == PRIMARY_MINT else token_wallet
return fake_get_wallet
async def _post_topup(
client: AsyncClient,
mock_token: Mock,
token_wallet: Mock,
primary_wallet: Mock,
) -> Response:
"""POST /v1/wallet/topup with the swap layer mocked at the mint boundary.
The conftest's testmint stub for recieve_token is swapped back for the
real implementation so the request exercises the actual swap path.
"""
with patch("routstr.wallet.recieve_token", _real_recieve_token):
with patch(
"routstr.wallet.deserialize_token_from_string", return_value=mock_token
):
with patch(
"routstr.wallet.get_wallet",
side_effect=_wallet_router(primary_wallet, token_wallet),
):
with patch.object(settings, "primary_mint", PRIMARY_MINT):
with patch.object(settings, "primary_mint_unit", "sat"):
with patch.object(settings, "cashu_mints", [PRIMARY_MINT]):
return await client.post(
"/v1/wallet/topup",
params={"cashu_token": "cashuAtest_foreign_token"},
)
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_retries_when_melt_demands_more_than_quoted(
authenticated_client: AsyncClient,
) -> None:
"""A 179-sat token where every quote reports fee_reserve=1 but the mint
rejects the first melt demanding 180. The retry shrinks the invoice to 177
and the topup credits 177 sats (177_000 msats)."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
179, fee_reserves=[1, 1, 1], mint_url="http://mint.cubabitcoin.org"
)
token_wallet.melt.side_effect = [
Exception(
"Mint Error: not enough inputs provided for melt. "
"Provided: 179, needed: 180 (Code: 11000)"
),
Mock(),
]
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 200
assert response.json()["msats"] == 177_000
assert token_wallet.melt.call_count == 2
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_retries_when_quote_fee_exceeds_estimate(
authenticated_client: AsyncClient,
) -> None:
"""A 1000-sat token estimated at fee 20, but the real quote demands 23.
The retry recomputes 1000 - 23 = 977, which fits, and the topup credits
977 sats (977_000 msats) with a single melt."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
1000, fee_reserves=[20, 23, 23]
)
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 200
assert response.json()["msats"] == 977_000
assert token_wallet.melt.call_count == 1
@pytest.mark.integration
@pytest.mark.asyncio
async def test_topup_returns_422_when_retries_exhausted(
authenticated_client: AsyncClient,
) -> None:
"""A mint that escalates fee_reserve on every re-quote (1 → 10 → 25 → 50)
exhausts the retry budget: clean 422 mint_error/too-small taxonomy, melt
never executed."""
mock_token, token_wallet, primary_wallet = _make_swap_mocks(
1000, fee_reserves=[1, 10, 25, 50]
)
response = await _post_topup(
authenticated_client, mock_token, token_wallet, primary_wallet
)
assert response.status_code == 422
assert "too small to cover swap fees" in response.json()["detail"]
assert token_wallet.melt_quote.call_count == 4 # estimation + 3 attempts
token_wallet.melt.assert_not_called()

View File

@@ -12,10 +12,7 @@ from sqlmodel import select
from routstr.core.db import ApiKey
from .utils import (
CashuTokenGenerator,
ResponseValidator,
)
from .utils import ResponseValidator
@pytest.mark.integration
@@ -79,29 +76,31 @@ async def test_api_key_generation_invalid_token(
# Capture initial state
await db_snapshot.capture()
# Test various invalid tokens
invalid_tokens = [
CashuTokenGenerator.generate_invalid_token(), # Malformed token
"not-a-cashu-token", # Wrong format
"cashuA", # Empty token
"cashuA" + "x" * 1000, # Invalid base64
# Non-Cashu bearer values are invalid API keys (401). Malformed values that
# look like Cashu tokens use the shared Cashu taxonomy (400 invalid_token).
invalid_tokens: list[tuple[str, int, str | None]] = [
("not-a-cashu-token", 401, None),
("sk-not-a-real-api-key", 401, None),
("cashuA", 400, "invalid_cashu_token"),
("cashuA" + "x" * 1000, 400, "invalid_cashu_token"),
]
for invalid_token in invalid_tokens:
for invalid_token, expected_status, expected_code in invalid_tokens:
integration_client.headers["Authorization"] = f"Bearer {invalid_token}"
response = await integration_client.get("/v1/wallet/info")
# Should fail with 401
assert response.status_code == 401, (
assert response.status_code == expected_status, (
f"Token {invalid_token[:20]}... should be invalid"
)
# Validate error response
validator = ResponseValidator()
error_validation = validator.validate_error_response(
response, expected_status=401, expected_error_key="detail"
response, expected_status=expected_status, expected_error_key="detail"
)
assert error_validation["valid"]
if expected_code is not None:
assert response.json()["detail"]["error"]["code"] == expected_code
# Verify no database changes
diff = await db_snapshot.diff()

View File

@@ -14,6 +14,7 @@ from httpx import AsyncClient
from sqlmodel import select
from routstr.core.db import ApiKey, CashuTransaction
from routstr.wallet import MintConnectionError
@pytest.mark.integration
@@ -503,26 +504,19 @@ async def test_mint_unavailability_handling(
# The global mock in conftest.py is already in place,
# so we need to temporarily modify it
from unittest.mock import patch
raw_error = "Mint unavailable: Connection refused"
# Make the send_token method raise an exception
# Make the send_token method raise a typed mint connection exception.
with patch(
"routstr.balance.send_token",
side_effect=Exception("Mint unavailable: Connection refused"),
side_effect=MintConnectionError(raw_error),
):
# The exception should propagate as a 503 error (Service Unavailable)
# But we need to handle it properly
try:
response = await authenticated_client.post("/v1/wallet/refund")
# If we get here, check the status code
assert response.status_code == 503
assert "Mint service unavailable" in response.json()["detail"]
except Exception as e:
# If the exception propagates, that's also a failure scenario
assert "Mint unavailable" in str(e)
response = await authenticated_client.post("/v1/wallet/refund")
assert response.status_code == 503
assert response.json()["detail"] == "Mint service unavailable"
assert raw_error not in response.text
# Balance should remain unchanged (transaction should roll back)
# Note: Current implementation might not handle this perfectly
wallet_response = await authenticated_client.get("/v1/wallet/")
assert wallet_response.status_code == 200
assert wallet_response.json()["balance"] == 10_000_000

View File

@@ -0,0 +1,293 @@
import hashlib
from types import SimpleNamespace
from typing import AsyncGenerator, cast
from unittest.mock import AsyncMock, patch
import httpx
import pytest
from fastapi import HTTPException
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlalchemy.pool import StaticPool
from sqlmodel import SQLModel
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.auth import validate_bearer_key
from routstr.core.db import ApiKey
from routstr.wallet import MintConnectionError
def _value_error_wrapping_transport() -> ValueError:
"""A ValueError re-raised ``from`` a real httpx transport error, mirroring
``wallet.py`` wrapping a connection failure. The sanitized classifier must
still see the mint-unreachable signal through the ``__cause__`` chain."""
try:
raise httpx.ConnectError("All connection attempts failed")
except httpx.ConnectError as exc:
err = ValueError("Failed to estimate fees: connection failed")
err.__cause__ = exc
return err
def _make_engine() -> AsyncEngine:
return create_async_engine(
"sqlite+aiosqlite://",
poolclass=StaticPool,
connect_args={"check_same_thread": False},
)
@pytest.fixture
async def session() -> AsyncGenerator[AsyncSession, None]:
engine = _make_engine()
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
db_session = AsyncSession(engine, expire_on_commit=False)
try:
yield db_session
finally:
await db_session.close()
await engine.dispose()
@pytest.mark.asyncio
async def test_failed_first_cashu_redemption_rolls_back_empty_api_key(
session: AsyncSession,
) -> None:
token = "cashuAfirst_seen_but_redemption_fails"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(side_effect=ValueError("token already spent")),
),
):
with pytest.raises(HTTPException):
await validate_bearer_key(token, session)
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.parametrize(
("error", "expected_status", "expected_type", "expected_message", "expected_code"),
[
(
ValueError("Mint Error: Token already spent. (Code: 11001)"),
400,
"token_already_spent",
"Cashu token already spent",
"cashu_token_already_spent",
),
(
# Raw httpx transport error propagated unwrapped from cashu.
httpx.ConnectError("All connection attempts failed"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# Typed error raised by wallet.py at a wrap site.
MintConnectionError("connect to http://mint:3338 refused"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# ValueError wrapping the httpx error in its __cause__ chain.
_value_error_wrapping_transport(),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
# asyncio.TimeoutError is builtin TimeoutError on 3.11+.
TimeoutError("Timed out connecting to Cashu mint http://mint:3338"),
503,
"mint_unreachable",
"Cashu mint is unreachable",
"cashu_mint_unreachable",
),
(
ValueError(
"Token amount (5 sat) is insufficient to cover melt fees. "
"Needed: 7 sat (amount: 5 + fee: 1 + input_fees: 1)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError(
"Failed to melt token from foreign mint http://foreign:3338: boom"
),
422,
"mint_error",
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
),
(
ValueError("could not decode token"),
400,
"invalid_token",
"Invalid Cashu token",
"invalid_cashu_token",
),
(
ValueError("some unexpected wallet condition"),
400,
"cashu_error",
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
),
(
ValueError("Redeemed token amount must be positive, got 0 msats"),
400,
"cashu_error",
"Failed to redeem Cashu token: token yielded no value",
"cashu_token_zero_value",
),
],
)
@pytest.mark.asyncio
async def test_redemption_failure_returns_sanitized_error(
session: AsyncSession,
error: Exception,
expected_status: int,
expected_type: str,
expected_message: str,
expected_code: str,
) -> None:
"""Redemption failures reuse the shared X-Cashu taxonomy (carried in
``type``), expose stable sanitized messages and granular machine-readable
``code`` values, and leave no orphan ApiKey row."""
token = "cashuAredemption_fails_with_specific_error"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(side_effect=error),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == expected_status
detail = cast(dict[str, dict[str, object]], exc_info.value.detail)
error_detail = detail["error"]
assert error_detail["type"] == expected_type
assert error_detail["code"] == expected_code
assert error_detail["message"] == expected_message
assert str(error) not in cast(str, error_detail["message"])
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_unexpected_redemption_error_returns_internal_error(
session: AsyncSession,
) -> None:
"""Unexpected (non-wallet) failures surface as generic 500s without
leaking internal details, instead of masquerading as token errors."""
token = "cashuAredemption_fails_with_internal_error"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(side_effect=RuntimeError("db exploded at /var/lib/secret")),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 500
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
error_detail = detail["error"]
assert error_detail["code"] == "internal_error"
assert "/var/lib/secret" not in error_detail["message"]
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_internal_error_with_invalid_keyword_does_not_masquerade(
session: AsyncSession,
) -> None:
"""A non-wallet fault whose text merely contains "invalid" (but not
"token") must fall through to a generic 500, not a 401 token error.
Guards the anchored `"invalid"/"decode"` + `"token"` gate against stdlib/
driver strings like "Invalid isoformat string" leaking as token errors."""
token = "cashuAinternal_fault_mentions_invalid"
hashed_key = hashlib.sha256(token.encode()).hexdigest()
token_obj = SimpleNamespace(mint="http://mint:3338", unit="sat")
from routstr.core.settings import settings
with (
patch.object(settings, "cashu_mints", ["http://mint:3338"]),
patch("routstr.auth.deserialize_token_from_string", return_value=token_obj),
patch(
"routstr.auth.credit_balance",
new=AsyncMock(
side_effect=RuntimeError("Invalid isoformat string: '2020-13-99'")
),
),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 500
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
assert detail["error"]["code"] == "internal_error"
assert await session.get(ApiKey, hashed_key) is None
@pytest.mark.asyncio
async def test_malformed_cashu_token_returns_400_invalid_token(
session: AsyncSession,
) -> None:
"""A malformed 'cashu...' token that fails to decode maps to 400
invalid_cashu_token (shared taxonomy), not the generic 401 invalid_api_key."""
token = "cashuAthis_is_not_a_valid_token"
with patch(
"routstr.auth.deserialize_token_from_string",
side_effect=ValueError("unable to decode token: bad base64"),
):
with pytest.raises(HTTPException) as exc_info:
await validate_bearer_key(token, session)
assert exc_info.value.status_code == 400
detail = cast(dict[str, dict[str, str]], exc_info.value.detail)
assert detail["error"]["type"] == "invalid_token"
assert detail["error"]["code"] == "invalid_cashu_token"
# Raw decoder text must not leak to the client.
assert "base64" not in detail["error"]["message"]

View File

@@ -1,12 +1,13 @@
import json
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from fastapi.responses import JSONResponse
from routstr.balance import refund_wallet_endpoint
from routstr.balance import refund_wallet_endpoint, topup_wallet_endpoint
from routstr.core.db import ApiKey, CashuTransaction
from routstr.wallet import credit_balance
from routstr.wallet import MintConnectionError, credit_balance
def _make_cashu_tx(
@@ -103,6 +104,64 @@ async def test_refund_x_cashu_not_found_raises_404() -> None:
assert exc_info.value.status_code == 404
@pytest.mark.asyncio
async def test_refund_x_cashu_pending_raises_425() -> None:
"""in row exists with a request_id but out row not yet created → 425.
This is the race condition where /v1/wallet/refund is polled while the
upstream request is still in flight. The endpoint must signal "retry"
rather than a permanent 404.
"""
from fastapi import HTTPException
x_cashu_token = "cashuApending_token"
in_tx = _make_cashu_tx(
token=x_cashu_token, amount=0, unit="msat", type="in", request_id="req-pending"
)
session = MagicMock()
session.exec = AsyncMock(side_effect=[_exec_result(in_tx), _exec_result(None)])
session.add = MagicMock()
session.commit = AsyncMock()
with pytest.raises(HTTPException) as exc_info:
await refund_wallet_endpoint(
authorization="Bearer sk-somekey",
x_cashu=x_cashu_token,
session=session,
)
assert exc_info.value.status_code == 425
assert exc_info.value.headers == {"Retry-After": "2"}
@pytest.mark.asyncio
async def test_refund_x_cashu_in_tx_without_request_id_raises_404() -> None:
"""in row exists but has no request_id (cannot link to a refund) → 404.
This is a genuine "no refund will ever exist" case, distinct from the
pending 425 path.
"""
from fastapi import HTTPException
x_cashu_token = "cashuAnoreqid_token"
in_tx = _make_cashu_tx(
token=x_cashu_token, amount=0, unit="msat", type="in", request_id=None
)
session = MagicMock()
session.exec = AsyncMock(side_effect=[_exec_result(in_tx)])
with pytest.raises(HTTPException) as exc_info:
await refund_wallet_endpoint(
authorization="Bearer sk-somekey",
x_cashu=x_cashu_token,
session=session,
)
assert exc_info.value.status_code == 404
@pytest.mark.asyncio
async def test_refund_x_cashu_swept_raises_410() -> None:
from fastapi import HTTPException
@@ -339,7 +398,10 @@ async def test_apikey_refund_restores_balance_on_mint_failure() -> None:
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.send_token", AsyncMock(side_effect=Exception("mint down"))),
patch(
"routstr.balance.send_token",
AsyncMock(side_effect=MintConnectionError("raw mint outage detail")),
),
patch("routstr.balance.store_cashu_transaction", AsyncMock()),
patch("routstr.balance._refund_cache_get", AsyncMock(return_value=None)),
patch("routstr.balance._refund_cache_set", AsyncMock()),
@@ -353,10 +415,46 @@ async def test_apikey_refund_restores_balance_on_mint_failure() -> None:
)
assert exc_info.value.status_code == 503
assert exc_info.value.detail == "Mint service unavailable"
assert "raw mint outage detail" not in exc_info.value.detail
# Verify two exec calls: debit + restore
assert session.exec.await_count == 2
@pytest.mark.asyncio
async def test_apikey_refund_generic_failure_is_sanitized_500() -> None:
"""Unexpected send-side failures restore balance without leaking exception text."""
from fastapi import HTTPException
key = _make_api_key(balance=5000, refund_currency="sat")
raw_error = "database secret token raw-mint-response"
session = MagicMock()
session.get = AsyncMock(return_value=key)
session.exec = AsyncMock(side_effect=[_update_result(1), _update_result(1)])
session.commit = AsyncMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.send_token", AsyncMock(side_effect=RuntimeError(raw_error))),
patch("routstr.balance.store_cashu_transaction", AsyncMock()),
patch("routstr.balance._refund_cache_get", AsyncMock(return_value=None)),
patch("routstr.balance._refund_cache_set", AsyncMock()),
patch("routstr.balance.logger"),
):
with pytest.raises(HTTPException) as exc_info:
await refund_wallet_endpoint(
authorization="Bearer sk-testhash",
x_cashu=None,
session=session,
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == "Refund failed"
assert raw_error not in exc_info.value.detail
assert session.exec.await_count == 2
# ---------------------------------------------------------------------------
# no-create guarantee: fresh Cashu/unknown sk- tokens must not create API keys
# ---------------------------------------------------------------------------
@@ -401,3 +499,190 @@ async def test_refund_unknown_sk_bearer_returns_401() -> None:
assert exc_info.value.status_code == 401
session.get.assert_awaited_once()
# --- Topup redemption error taxonomy (POST /v1/wallet/topup) ------------------
@pytest.mark.asyncio
@pytest.mark.parametrize(
"error",
[
httpx.ConnectError("All connection attempts failed"),
MintConnectionError("connect to mint refused"),
TimeoutError("timed out connecting to mint"),
],
)
async def test_topup_mint_unreachable_returns_503(error: Exception) -> None:
"""A down mint must surface 503 (retryable), not 400 or 500 — the token is
fine, so the client should retry once the mint recovers."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.credit_balance", AsyncMock(side_effect=error)),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 503
assert exc_info.value.detail == "Cashu mint is unreachable"
@pytest.mark.asyncio
async def test_topup_already_spent_still_returns_400() -> None:
"""Regression: the mint-unreachable short-circuit must not swallow the
existing ValueError substring buckets."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=ValueError("Token already spent")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 400
assert exc_info.value.detail == "Cashu token already spent"
@pytest.mark.asyncio
async def test_topup_zero_value_returns_400_zero_value_message() -> None:
"""A dust/zero redemption maps to the documented zero-value message, not the
generic redemption-failed one."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(
side_effect=ValueError("Redeemed token amount must be positive, got 0 msats")
),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 400
assert exc_info.value.detail == "Failed to redeem Cashu token: token yielded no value"
@pytest.mark.asyncio
async def test_topup_token_consumed_returns_500() -> None:
"""A post-redemption crediting failure (token spent) is a non-retryable 500,
not a 4xx that invites a retry."""
from fastapi import HTTPException
from routstr.wallet import TokenConsumedError
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=TokenConsumedError("credit failed")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == (
"Token was redeemed but could not be credited; do not retry"
)
@pytest.mark.asyncio
@pytest.mark.parametrize(
("error", "expected_status", "expected_detail"),
[
(
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"Token value is too small to cover swap fees",
),
(
ValueError(
"Token amount (5 sat) is insufficient to cover melt fees."
),
422,
"Token value is too small to cover swap fees",
),
(
ValueError("Failed to melt token from foreign mint http://m: boom"),
422,
"Failed to swap token from foreign mint",
),
],
)
async def test_topup_fee_and_swap_failures_return_422(
error: Exception, expected_status: int, expected_detail: str
) -> None:
"""Fee/swap failures map to 422 (shared taxonomy), matching the bearer and
X-Cashu paths — previously top-up flattened these to 400."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch("routstr.balance.credit_balance", AsyncMock(side_effect=error)),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == expected_status
assert exc_info.value.detail == expected_detail
@pytest.mark.asyncio
async def test_topup_unexpected_non_valueerror_returns_500() -> None:
"""A non-ValueError, non-transport fault is an internal error (500), not a
sanitized 400 — the merged except must preserve this."""
from fastapi import HTTPException
key = _make_api_key(balance=1000)
session = MagicMock()
with (
patch("routstr.balance.get_billing_key", AsyncMock(return_value=key)),
patch(
"routstr.balance.credit_balance",
AsyncMock(side_effect=RuntimeError("db exploded")),
),
):
with pytest.raises(HTTPException) as exc_info:
await topup_wallet_endpoint(
cashu_token="cashuAtoken", key=key, session=session
)
assert exc_info.value.status_code == 500
assert exc_info.value.detail == "Internal server error"

View File

@@ -0,0 +1,189 @@
"""Tests for Anthropic cache-breakpoint injection on forwarded requests.
Anthropic prompt caching is explicit; a client that doesn't recognise a routstr
URL as Anthropic-backed never sends ``cache_control`` markers, so caching never
engages over routstr. ``prepare_request_body`` must stamp the standard
breakpoints for Anthropic-family models while always deferring to client-set
markers and never touching automatic-cache providers.
"""
import json
import os
import pytest
os.environ.setdefault("UPSTREAM_BASE_URL", "http://test")
os.environ.setdefault("UPSTREAM_API_KEY", "test")
os.environ.setdefault("LIGHTNING_ADDRESS", "test@stm.to")
from routstr.upstream import GenericUpstreamProvider
from routstr.upstream.cache_breakpoints import (
body_has_cache_control,
inject_anthropic_cache_breakpoints,
is_explicit_cache_model,
)
def _chat_body() -> dict:
return {
"model": "anthropic/claude-sonnet-4.5",
"stream": True,
"messages": [
{"role": "system", "content": "You are concise."},
{"role": "user", "content": "Hello"},
],
"tools": [
{"type": "function", "function": {"name": "a"}},
{"type": "function", "function": {"name": "b"}},
],
}
# ---------------------------------------------------------------------------
# Model detection
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"model_id,expected",
[
("anthropic/claude-sonnet-4.5", True),
("claude-haiku-4-5-20251001", True),
# Alibaba explicit-cache models share Anthropic's wire format
("qwen/qwen3-max", True),
("qwen/qwen3-coder-plus", True),
("deepseek/deepseek-v3.2", True),
# Automatic-cache providers need no markers
("openai/gpt-4o", False),
("google/gemini-2.5-flash", False),
("deepseek/deepseek-chat", False),
("qwen/qwen3.5-plus-02-15", False), # snapshot, no explicit caching
(None, False),
],
)
def test_is_explicit_cache_model(model_id: str | None, expected: bool) -> None:
assert is_explicit_cache_model(model_id) is expected
def test_is_explicit_cache_model_uses_fallbacks() -> None:
# routstr id is opaque but a forwarded/canonical alias reveals the family.
assert is_explicit_cache_model("model-xyz", None, "anthropic/claude-opus-4.1")
# ---------------------------------------------------------------------------
# Breakpoint placement
# ---------------------------------------------------------------------------
def test_injects_three_breakpoints() -> None:
data = _chat_body()
assert inject_anthropic_cache_breakpoints(data) is True
# system prompt promoted to array form with a marker
system = data["messages"][0]["content"]
assert system == [
{
"type": "text",
"text": "You are concise.",
"cache_control": {"type": "ephemeral"},
}
]
# last tool marked
assert data["tools"][-1]["cache_control"] == {"type": "ephemeral"}
assert "cache_control" not in data["tools"][0]
# last user message marked
user = data["messages"][1]["content"]
assert user[-1]["cache_control"] == {"type": "ephemeral"}
def test_defers_to_client_supplied_cache_control() -> None:
data = _chat_body()
data["messages"][1]["content"] = [
{"type": "text", "text": "Hello", "cache_control": {"type": "ephemeral"}}
]
assert body_has_cache_control(data) is True
# No additional stamping when the client already controls caching.
assert inject_anthropic_cache_breakpoints(data) is False
assert "cache_control" not in data["tools"][-1]
def test_marks_last_text_part_of_array_content() -> None:
data = _chat_body()
data["messages"][1]["content"] = [
{"type": "text", "text": "first"},
{"type": "image_url", "image_url": {"url": "x"}},
{"type": "text", "text": "last"},
]
inject_anthropic_cache_breakpoints(data)
parts = data["messages"][1]["content"]
assert parts[2]["cache_control"] == {"type": "ephemeral"}
assert "cache_control" not in parts[0]
def test_noop_without_messages() -> None:
assert inject_anthropic_cache_breakpoints({"prompt": "x"}) is False
# ---------------------------------------------------------------------------
# prepare_request_body integration
# ---------------------------------------------------------------------------
def _model(model_id: str): # type: ignore[no-untyped-def]
from routstr.payment.models import Architecture, Model, Pricing
return Model(
id=model_id,
name=model_id,
created=0,
description="",
context_length=200000,
architecture=Architecture(
modality="text->text",
input_modalities=["text"],
output_modalities=["text"],
tokenizer="Claude",
instruct_type=None,
),
pricing=Pricing(prompt=0.0, completion=0.0),
)
def _openrouter_provider() -> "GenericUpstreamProvider":
# OpenRouter endpoint via the generic provider — recognised by base URL.
return GenericUpstreamProvider(base_url="https://openrouter.ai/api/v1")
@pytest.mark.parametrize(
"model_id", ["anthropic/claude-sonnet-4.5", "qwen/qwen3-max", "deepseek/deepseek-v3.2"]
)
def test_prepare_request_body_injects_for_explicit_models(model_id: str) -> None:
provider = _openrouter_provider()
body = json.dumps(_chat_body()).encode()
out = provider.prepare_request_body(body, _model(model_id))
assert out is not None
data = json.loads(out)
assert body_has_cache_control(data) is True
assert data["tools"][-1]["cache_control"] == {"type": "ephemeral"}
def test_prepare_request_body_skips_for_automatic_provider_model() -> None:
provider = _openrouter_provider()
body = json.dumps(_chat_body()).encode()
out = provider.prepare_request_body(body, _model("openai/gpt-4o"))
assert out is not None
data = json.loads(out)
assert body_has_cache_control(data) is False
def test_prepare_request_body_skips_when_upstream_rejects_markers() -> None:
# Claude id but a non-OpenRouter/Anthropic upstream → must NOT inject,
# since the markers could be rejected by an upstream that doesn't accept them.
from routstr.upstream import GenericUpstreamProvider
provider = GenericUpstreamProvider(base_url="https://some-gateway.example/v1")
body = json.dumps(_chat_body()).encode()
out = provider.prepare_request_body(body, _model("anthropic/claude-sonnet-4.5"))
assert out is not None
data = json.loads(out)
assert body_has_cache_control(data) is False

View File

@@ -0,0 +1,347 @@
"""Tests for cache-aware pricing of cached input tokens.
Specifies two things:
1. ``backfill_cache_pricing`` — when the OpenRouter model feed omits cache
rates (it does for most DeepSeek models and e.g. openai/gpt-4o), they are
filled from litellm's bundled cost map instead of silently billing cache
reads at the full input rate. Existing OpenRouter values are never
overwritten, and provider fees apply to backfilled rates like any other.
2. ``calculate_cost`` — cached tokens are billed at the cache rates from the
model's sats_pricing; the full input rate remains only as the documented
last resort when no cache rate could be resolved anywhere.
"""
import os
from unittest.mock import Mock, patch
import litellm
import pytest
os.environ.setdefault("UPSTREAM_BASE_URL", "http://test")
os.environ.setdefault("UPSTREAM_API_KEY", "test")
os.environ.setdefault("LIGHTNING_ADDRESS", "test@stm.to")
from routstr.core.settings import settings
from routstr.payment.cost_calculation import CostData, calculate_cost
from routstr.payment.models import (
Architecture,
Model,
Pricing,
backfill_cache_pricing,
)
from routstr.upstream import GenericUpstreamProvider
def _make_model(model_id: str, pricing: Pricing) -> Model:
return Model(
id=model_id,
name=model_id,
created=0,
description="",
context_length=64000,
architecture=Architecture(
modality="text->text",
input_modalities=["text"],
output_modalities=["text"],
tokenizer="Other",
instruct_type=None,
),
pricing=pricing,
)
# ============================================================================
# backfill_cache_pricing — litellm as fallback source for missing cache rates
# ============================================================================
def test_backfill_deepseek_cache_read_from_litellm() -> None:
"""deepseek/deepseek-chat has no input_cache_read on OpenRouter; litellm
knows the real rate (10x cheaper than input)."""
pricing = Pricing(prompt=2.8e-07, completion=4.2e-07)
result = backfill_cache_pricing("deepseek/deepseek-chat", pricing)
expected = litellm.model_cost["deepseek/deepseek-chat"][
"cache_read_input_token_cost"
]
assert result.input_cache_read == expected
assert result.input_cache_read < pricing.prompt # sanity: it's a discount
def test_backfill_strips_vendor_prefix_for_litellm_lookup() -> None:
"""OpenRouter ids are vendor-prefixed (openai/gpt-4o); litellm keys most
non-DeepSeek models without the prefix (gpt-4o)."""
pricing = Pricing(prompt=2.5e-06, completion=1e-05)
result = backfill_cache_pricing("openai/gpt-4o", pricing)
expected = litellm.model_cost["gpt-4o"]["cache_read_input_token_cost"]
assert result.input_cache_read == expected
def test_backfill_case_insensitive_lookup() -> None:
"""A generic upstream may report a mixed-case id
(deepseek-ai/DeepSeek-V4-Flash); litellm keys are lowercase. The
case-insensitive fallback still resolves the cache rate."""
pricing = Pricing(prompt=1.4e-07, completion=2.8e-07)
result = backfill_cache_pricing("deepseek-ai/DeepSeek-V4-Flash", pricing)
expected = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert result.input_cache_read == expected
assert result.input_cache_read < pricing.prompt # sanity: it's a discount
def test_backfill_fills_cache_write_rate() -> None:
"""Anthropic cache writes cost more than input (1.25x); billing them at
the input rate undercharges. litellm carries the write rate."""
pricing = Pricing(prompt=3e-06, completion=1.5e-05)
result = backfill_cache_pricing("anthropic/claude-sonnet-4-5", pricing)
expected = litellm.model_cost["claude-sonnet-4-5"][
"cache_creation_input_token_cost"
]
assert result.input_cache_write == expected
assert result.input_cache_write > pricing.prompt # sanity: write premium
def test_backfill_never_overwrites_openrouter_rates() -> None:
"""When OpenRouter provides a cache rate, it is authoritative."""
pricing = Pricing(
prompt=2.1e-07, completion=7.9e-07, input_cache_read=1.3e-07
)
result = backfill_cache_pricing("deepseek/deepseek-chat", pricing)
assert result.input_cache_read == 1.3e-07
def test_backfill_unknown_model_unchanged() -> None:
"""Models litellm doesn't know stay untouched (last-resort fallback to
the input rate happens later, at billing time)."""
pricing = Pricing(prompt=1e-06, completion=2e-06)
result = backfill_cache_pricing("artificial-dumbness/dumb-1", pricing)
assert result.input_cache_read == 0.0
assert result.input_cache_write == 0.0
def test_provider_fee_applies_to_backfilled_cache_rates() -> None:
"""Backfill happens before the provider fee, so cache rates carry the
same markup as every other price component."""
provider = GenericUpstreamProvider(
base_url="http://upstream.example", provider_fee=2.0
)
model = _make_model(
"deepseek/deepseek-chat", Pricing(prompt=2.8e-07, completion=4.2e-07)
)
adjusted = provider._apply_provider_fee_to_model(model)
litellm_read = litellm.model_cost["deepseek/deepseek-chat"][
"cache_read_input_token_cost"
]
assert adjusted.pricing.input_cache_read == pytest.approx(litellm_read * 2.0)
assert adjusted.pricing.prompt == pytest.approx(2.8e-07 * 2.0)
def test_row_to_model_backfills_cache_rate() -> None:
"""The DB-override path (admin-configured providers, e.g. a generic
upstream) stores pricing without cache rates. ``_row_to_model`` must
backfill them from litellm just like ``_apply_provider_fee_to_model``,
otherwise cache reads bill at the full input rate."""
import json
from routstr.core.db import ModelRow
from routstr.payment.models import _row_to_model
row = ModelRow(
id="deepseek-v4-flash",
name="deepseek-v4-flash",
created=0,
description="",
context_length=1000000,
architecture=json.dumps(
{
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
}
),
# Stored pricing omits input_cache_read (generic provider never sets it).
pricing=json.dumps({"prompt": 1.4e-07, "completion": 2.8e-07}),
enabled=True,
upstream_provider_id=1,
)
with patch(
"routstr.payment.models.sats_usd_price", return_value=5.0e-5
):
model = _row_to_model(row, apply_provider_fee=True, provider_fee=1.0)
litellm_read = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert model.pricing.input_cache_read == pytest.approx(litellm_read)
assert model.pricing.input_cache_read < model.pricing.prompt # a discount
assert model.sats_pricing is not None
assert model.sats_pricing.input_cache_read > 0
def test_row_to_model_backfills_via_forwarded_model_id() -> None:
"""An alias row (id != forwarded_model_id) must backfill cache rates from
the *forwarded* model name — the real upstream model litellm prices —
not the alias id, which litellm doesn't know."""
import json
from routstr.core.db import ModelRow
from routstr.payment.models import _row_to_model
row = ModelRow(
id="local-alias", # litellm has no such key
name="local-alias",
created=0,
description="",
context_length=1000000,
architecture=json.dumps(
{
"modality": "text",
"input_modalities": ["text"],
"output_modalities": ["text"],
"tokenizer": "unknown",
"instruct_type": None,
}
),
pricing=json.dumps({"prompt": 1.4e-07, "completion": 2.8e-07}),
enabled=True,
upstream_provider_id=1,
forwarded_model_id="deepseek-v4-flash",
)
with patch(
"routstr.payment.models.sats_usd_price", return_value=5.0e-5
):
model = _row_to_model(row, apply_provider_fee=True, provider_fee=1.0)
litellm_read = litellm.model_cost["deepseek-v4-flash"][
"cache_read_input_token_cost"
]
assert model.pricing.input_cache_read == pytest.approx(litellm_read)
assert model.pricing.input_cache_read < model.pricing.prompt
# ============================================================================
# calculate_cost — cached tokens billed at cache rates
# ============================================================================
@pytest.fixture(autouse=True)
def patch_sats_usd_price() -> None: # type: ignore[misc]
with patch("routstr.payment.cost_calculation.sats_usd_price", return_value=5.0e-5):
yield
@pytest.fixture
def model_pricing(monkeypatch: pytest.MonkeyPatch) -> Mock:
"""Model-based pricing: 1 msat per input token, 2 per output token,
0.1 per cache-read token, 1.25 per cache-write token."""
monkeypatch.setattr(settings, "fixed_pricing", False)
model = Mock()
model.sats_pricing = Pricing(
prompt=0.001,
completion=0.002,
input_cache_read=0.0001,
input_cache_write=0.00125,
)
return model
@pytest.mark.asyncio
async def test_deepseek_cache_hits_billed_at_cache_rate(model_pricing: Mock) -> None:
"""The reported overcharge scenario: a 10k-token prompt with 90% cache
hits costs 2900 msats at honest rates, not the 11000 msats that billing
every prompt token at the full input rate would charge."""
response = {
"model": "deepseek-chat",
"usage": {
"prompt_tokens": 10000,
"completion_tokens": 500,
"prompt_cache_hit_tokens": 9000,
"prompt_cache_miss_tokens": 1000,
},
}
with patch("routstr.proxy.get_model_instance", return_value=model_pricing):
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# 1000 input @ 1 msat + 9000 cache reads @ 0.1 msat + 500 output @ 2 msat.
# input_msats folds the cache-read cost in (1000 + 900) so a dashboard
# rendering I/O/T sees input + output == total; the cache portion stays
# visible in cache_read_msats.
assert result.cache_read_msats == 900
assert result.output_msats == 1000
assert result.input_msats == 1900
assert result.input_msats + result.output_msats == result.total_msats
assert result.total_msats == 2900
@pytest.mark.asyncio
async def test_anthropic_cache_write_billed_at_write_rate(model_pricing: Mock) -> None:
"""Cache writes carry their premium rate (1.25x input here), instead of
being silently billed at the plain input rate."""
response = {
"model": "claude-sonnet-4-5",
"usage": {
"input_tokens": 300,
"output_tokens": 100,
"cache_read_input_tokens": 500,
"cache_creation_input_tokens": 2000,
},
}
with patch("routstr.proxy.get_model_instance", return_value=model_pricing):
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# 300 @ 1 + 500 @ 0.1 + 2000 @ 1.25 + 100 @ 2
assert result.cache_read_msats == 50
assert result.cache_creation_msats == 2500
assert result.total_msats == 3050
@pytest.mark.asyncio
async def test_missing_cache_rate_falls_back_to_input_rate(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Documented last resort: when no cache rate could be resolved anywhere
(OpenRouter and litellm both silent), cache reads bill at the input rate —
never cheaper, never free."""
monkeypatch.setattr(settings, "fixed_pricing", False)
model = Mock()
model.sats_pricing = Pricing(prompt=0.001, completion=0.002)
response = {
"model": "dumb-1",
"usage": {
"prompt_tokens": 10000,
"completion_tokens": 500,
"prompt_cache_hit_tokens": 9000,
"prompt_cache_miss_tokens": 1000,
},
}
with patch("routstr.proxy.get_model_instance", return_value=model):
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# 1000 @ 1 + 9000 @ 1 (fallback) + 500 @ 2
assert result.total_msats == 11000

View File

@@ -1,10 +1,11 @@
"""Tests for cache token handling in cost calculation.
Covers OpenAI vs Anthropic caching formats, edge cases, and billing accuracy.
Covers OpenAI, Anthropic and DeepSeek caching formats, dialect precedence,
edge cases, and billing accuracy.
"""
import os
from unittest.mock import AsyncMock, patch
from unittest.mock import patch
import pytest
@@ -16,12 +17,6 @@ from routstr.core.settings import settings
from routstr.payment.cost_calculation import CostData, MaxCostData, calculate_cost
@pytest.fixture
def mock_session() -> AsyncMock:
"""Mock AsyncSession for cost calculation tests."""
return AsyncMock()
@pytest.fixture(autouse=True)
def mock_fixed_pricing(monkeypatch: pytest.MonkeyPatch) -> None:
"""Mock settings and price to use fixed pricing."""
@@ -41,7 +36,7 @@ def patch_sats_usd_price() -> None: # type: ignore[misc]
# Test 1: OpenAI Cache Format
# ============================================================================
@pytest.mark.asyncio
async def test_openai_cache_subtraction(mock_session: AsyncMock) -> None:
async def test_openai_cache_subtraction() -> None:
"""OpenAI includes cached_tokens in prompt_tokens, subtract them."""
response = {
"model": "gpt-4",
@@ -53,7 +48,7 @@ async def test_openai_cache_subtraction(mock_session: AsyncMock) -> None:
}
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 1000 # 2000 - 1000
@@ -65,7 +60,7 @@ async def test_openai_cache_subtraction(mock_session: AsyncMock) -> None:
# Test 2: Anthropic Cache Format
# ============================================================================
@pytest.mark.asyncio
async def test_anthropic_cache_additive(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_anthropic_cache_additive(mock_fixed_pricing: None) -> None:
"""Anthropic cache tokens are separate (additive) from input_tokens."""
response = {
"model": "claude-3-5-sonnet",
@@ -76,7 +71,7 @@ async def test_anthropic_cache_additive(mock_session: AsyncMock, mock_fixed_pric
"cache_read_input_tokens": 0,
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 500
@@ -89,7 +84,7 @@ async def test_anthropic_cache_additive(mock_session: AsyncMock, mock_fixed_pric
# Test 3: Invalid Cache (Edge Case)
# ============================================================================
@pytest.mark.asyncio
async def test_cache_read_exceeds_prompt_tokens(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_cache_read_exceeds_prompt_tokens(mock_fixed_pricing: None) -> None:
"""Handle buggy upstream reporting cached > prompt_tokens."""
response = {
"model": "gpt-4",
@@ -101,7 +96,7 @@ async def test_cache_read_exceeds_prompt_tokens(mock_session: AsyncMock, mock_fi
}
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
# Should not go negative
assert isinstance(result, CostData)
@@ -114,7 +109,7 @@ async def test_cache_read_exceeds_prompt_tokens(mock_session: AsyncMock, mock_fi
# Test 4: Malformed Token Values
# ============================================================================
@pytest.mark.asyncio
async def test_malformed_cache_tokens_coerce_to_zero(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_malformed_cache_tokens_coerce_to_zero(mock_fixed_pricing: None) -> None:
"""Handle non-numeric cache token values."""
response = {
"model": "gpt-4",
@@ -127,7 +122,7 @@ async def test_malformed_cache_tokens_coerce_to_zero(mock_session: AsyncMock, mo
}
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
# Both should coerce to 0
assert isinstance(result, CostData)
@@ -139,7 +134,7 @@ async def test_malformed_cache_tokens_coerce_to_zero(mock_session: AsyncMock, mo
# Test 5: Anthropic Cache Not Subtracted
# ============================================================================
@pytest.mark.asyncio
async def test_anthropic_cache_not_subtracted(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_anthropic_cache_not_subtracted(mock_fixed_pricing: None) -> None:
"""Anthropic cache fields should NOT be subtracted from input_tokens."""
response = {
"model": "claude-3-5-sonnet",
@@ -149,7 +144,7 @@ async def test_anthropic_cache_not_subtracted(mock_session: AsyncMock, mock_fixe
"cache_read_input_tokens": 200, # ← Additive, don't subtract
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
# Anthropic: input_tokens stays as-is
assert isinstance(result, CostData)
@@ -161,7 +156,7 @@ async def test_anthropic_cache_not_subtracted(mock_session: AsyncMock, mock_fixe
# Test 6: Only Cache Read, No Regular Input
# ============================================================================
@pytest.mark.asyncio
async def test_only_cache_read_tokens(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_only_cache_read_tokens(mock_fixed_pricing: None) -> None:
"""Handle response with only cache read tokens."""
response = {
"model": "gpt-4",
@@ -173,7 +168,7 @@ async def test_only_cache_read_tokens(mock_session: AsyncMock, mock_fixed_pricin
}
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 0 # max(0, 0 - 1000)
@@ -185,7 +180,7 @@ async def test_only_cache_read_tokens(mock_session: AsyncMock, mock_fixed_pricin
# Test 7: Only Cache Creation
# ============================================================================
@pytest.mark.asyncio
async def test_only_cache_creation_tokens(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_only_cache_creation_tokens(mock_fixed_pricing: None) -> None:
"""Handle response with only cache creation tokens (Anthropic)."""
response = {
"model": "claude-3-5-sonnet",
@@ -196,7 +191,7 @@ async def test_only_cache_creation_tokens(mock_session: AsyncMock, mock_fixed_pr
"cache_read_input_tokens": 0,
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 500
@@ -209,7 +204,7 @@ async def test_only_cache_creation_tokens(mock_session: AsyncMock, mock_fixed_pr
# Test 8: Both Cache Read and Creation
# ============================================================================
@pytest.mark.asyncio
async def test_both_cache_read_and_creation(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_both_cache_read_and_creation(mock_fixed_pricing: None) -> None:
"""Handle response with both cache read and creation."""
response = {
"model": "claude-3-5-sonnet",
@@ -220,7 +215,7 @@ async def test_both_cache_read_and_creation(mock_session: AsyncMock, mock_fixed_
"cache_read_input_tokens": 500,
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 300
@@ -233,7 +228,7 @@ async def test_both_cache_read_and_creation(mock_session: AsyncMock, mock_fixed_
# Test 9: Token Field Fallback
# ============================================================================
@pytest.mark.asyncio
async def test_token_field_fallback_order(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_token_field_fallback_order(mock_fixed_pricing: None) -> None:
"""Verify fallback order for token extraction."""
# When prompt_tokens is not present, fall back to input_tokens
response = {
@@ -243,7 +238,7 @@ async def test_token_field_fallback_order(mock_session: AsyncMock, mock_fixed_pr
"completion_tokens": 50,
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 250
@@ -254,20 +249,21 @@ async def test_token_field_fallback_order(mock_session: AsyncMock, mock_fixed_pr
# Test 10: Float Token Values
# ============================================================================
@pytest.mark.asyncio
async def test_float_token_values_coerced_to_int(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_float_token_values_coerced_to_int(mock_fixed_pricing: None) -> None:
"""Handle float token values by converting to int."""
response = {
"model": "gpt-4",
"usage": {
"prompt_tokens": 100.7, # Float
"completion_tokens": 50.3, # Float
"cache_read_input_tokens": 25.9, # Float
"prompt_tokens_details": {"cached_tokens": 25.9}, # Float
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 100 # Floored
# cached_tokens are part of prompt_tokens (OpenAI dialect) → subtracted: 100 - 25
assert result.input_tokens == 75 # Floored
assert result.output_tokens == 50 # Floored
assert result.cache_read_input_tokens == 25 # Floored
@@ -276,7 +272,7 @@ async def test_float_token_values_coerced_to_int(mock_session: AsyncMock, mock_f
# Test 11: Boolean Cache Tokens
# ============================================================================
@pytest.mark.asyncio
async def test_boolean_cache_tokens_coerced_to_zero(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_boolean_cache_tokens_coerced_to_zero(mock_fixed_pricing: None) -> None:
"""Handle boolean cache token values by coercing to zero."""
response = {
"model": "gpt-4",
@@ -286,7 +282,7 @@ async def test_boolean_cache_tokens_coerced_to_zero(mock_session: AsyncMock, moc
"cache_read_input_tokens": True, # Boolean
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.cache_read_input_tokens == 0 # Boolean coerced to 0
@@ -297,7 +293,7 @@ async def test_boolean_cache_tokens_coerced_to_zero(mock_session: AsyncMock, moc
# Test 12: Zero Cache Tokens
# ============================================================================
@pytest.mark.asyncio
async def test_zero_cache_tokens(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_zero_cache_tokens(mock_fixed_pricing: None) -> None:
"""Handle explicit zero cache tokens."""
response = {
"model": "gpt-4",
@@ -309,21 +305,174 @@ async def test_zero_cache_tokens(mock_session: AsyncMock, mock_fixed_pricing: No
}
}
}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.cache_read_input_tokens == 0
assert result.input_tokens == 100
# ============================================================================
# DeepSeek Cache Format
# DeepSeek emits neither OpenAI's prompt_tokens_details nor Anthropic's
# cache_read_input_tokens — only prompt_cache_hit_tokens and
# prompt_cache_miss_tokens, with the documented guarantee
# prompt_tokens = hit + miss. Hits are ~10x cheaper upstream, so billing
# them as regular input is a large overcharge.
# ============================================================================
@pytest.mark.asyncio
async def test_deepseek_cache_hit_tokens_extracted() -> None:
"""DeepSeek cache hits are extracted and removed from regular input.
Payload shape verbatim from the DeepSeek API reference (usage object).
"""
response = {
"model": "deepseek-chat",
"usage": {
"prompt_tokens": 10000, # = hit + miss
"completion_tokens": 500,
"total_tokens": 10500,
"prompt_cache_hit_tokens": 9000,
"prompt_cache_miss_tokens": 1000,
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 1000 # only the cache misses
assert result.cache_read_input_tokens == 9000
assert result.output_tokens == 500
@pytest.mark.asyncio
async def test_deepseek_all_tokens_cached() -> None:
"""A fully cached DeepSeek prompt bills zero regular input tokens."""
response = {
"model": "deepseek-chat",
"usage": {
"prompt_tokens": 5000,
"completion_tokens": 100,
"prompt_cache_hit_tokens": 5000,
"prompt_cache_miss_tokens": 0,
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 0
assert result.cache_read_input_tokens == 5000
@pytest.mark.asyncio
async def test_dialect_precedence_never_double_subtracts() -> None:
"""If a vendor emits both OpenAI-style and DeepSeek-style cache fields for
the same cached tokens, they are counted once, not subtracted twice."""
response = {
"model": "deepseek-chat",
"usage": {
"prompt_tokens": 10000,
"completion_tokens": 500,
"prompt_tokens_details": {"cached_tokens": 9000},
"prompt_cache_hit_tokens": 9000,
"prompt_cache_miss_tokens": 1000,
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 1000 # 10000 - 9000, applied exactly once
assert result.cache_read_input_tokens == 9000
@pytest.mark.asyncio
async def test_deepseek_malformed_hit_tokens_coerce_to_zero() -> None:
"""Malformed DeepSeek cache fields degrade to billing all input at full
rate instead of crashing or going negative."""
response = {
"model": "deepseek-chat",
"usage": {
"prompt_tokens": 1000,
"completion_tokens": 50,
"prompt_cache_hit_tokens": "garbage",
"prompt_cache_miss_tokens": -5,
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.input_tokens == 1000
assert result.cache_read_input_tokens == 0
# ============================================================================
# Truly-empty response with a non-zero USD cost → full refund
#
# When an upstream reports a USD cost but the response carries NO tokens at all
# (input, output, cache-read and cache-creation all zero), billing the
# USD-derived cost charges the user for nothing. Refund in full. The gate is
# tightened relative to PR #489: a cache-read/-creation-only turn legitimately
# reports zero prompt/completion tokens with a real cost and must still bill.
# ============================================================================
@pytest.mark.asyncio
async def test_truly_empty_usd_cost_response_is_refunded(
mock_fixed_pricing: None,
) -> None:
"""0 input + 0 output + 0 cache tokens with a non-zero USD cost → refund."""
response = {
"model": "gpt-4",
"usage": {
"prompt_tokens": 0,
"completion_tokens": 0,
"total_cost": 0.01, # non-zero USD cost despite no tokens
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
assert result.total_msats == 0 # full refund
assert result.input_msats == 0
assert result.output_msats == 0
assert result.total_usd == 0.0
assert result.input_tokens == 0
assert result.output_tokens == 0
assert result.cache_read_input_tokens == 0
assert result.cache_creation_input_tokens == 0
@pytest.mark.asyncio
async def test_cache_read_only_usd_cost_response_is_billed(
mock_fixed_pricing: None,
) -> None:
"""Cache-read-only turn (0 prompt/completion, non-zero cost) still bills."""
response = {
"model": "claude-3-5-sonnet",
"usage": {
"input_tokens": 0,
"output_tokens": 0,
"cache_read_input_tokens": 1000, # real cached usage
"cache_creation_input_tokens": 0,
"total_cost": 0.01, # non-zero USD cost
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# NOT refunded — the USD cost is billed in full. Pinning the exact value
# guards against any future regression that would over-refund a cache-only
# turn (the bug in PR #489, which refunded whenever prompt+completion == 0).
assert result.total_msats == 200000
assert result.total_usd == 0.01
assert result.cache_read_input_tokens == 1000
# ============================================================================
# Test 13: Missing Usage Block
# ============================================================================
@pytest.mark.asyncio
async def test_missing_usage_block(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_missing_usage_block(mock_fixed_pricing: None) -> None:
"""When usage is missing, return MaxCostData with zero tokens."""
response = {"model": "gpt-4", "choices": [{"message": {"content": "test"}}]}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, MaxCostData)
assert result.input_tokens == 0
@@ -335,10 +484,10 @@ async def test_missing_usage_block(mock_session: AsyncMock, mock_fixed_pricing:
# Test 14: Null Usage Block
# ============================================================================
@pytest.mark.asyncio
async def test_null_usage_block(mock_session: AsyncMock, mock_fixed_pricing: None) -> None:
async def test_null_usage_block(mock_fixed_pricing: None) -> None:
"""When usage is null, return MaxCostData with zero tokens."""
response = {"model": "gpt-4", "usage": None}
result = await calculate_cost(response, max_cost=100000, session=mock_session)
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, MaxCostData)
assert result.input_tokens == 0

View File

@@ -0,0 +1,73 @@
from contextlib import asynccontextmanager
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from routstr.wallet import fetch_all_balances
@asynccontextmanager
async def _fake_session(): # type: ignore[no-untyped-def]
yield MagicMock()
def _patches(proof_amount: int = 1000): # type: ignore[no-untyped-def]
proof = MagicMock(amount=proof_amount)
return [
patch("routstr.wallet.get_wallet", AsyncMock(return_value=MagicMock())),
patch(
"routstr.wallet.get_proofs_per_mint_and_unit",
MagicMock(return_value=[proof]),
),
patch(
"routstr.wallet.slow_filter_spend_proofs",
AsyncMock(side_effect=lambda proofs, wallet: proofs),
),
patch(
"routstr.wallet.db.balances_for_mint_and_unit",
AsyncMock(return_value=0),
),
patch("routstr.wallet.db.create_session", _fake_session),
]
@pytest.mark.asyncio
async def test_fetch_all_balances_falls_back_to_primary_mint() -> None:
"""With empty cashu_mints, balances are still fetched for primary_mint."""
from routstr.core.settings import settings
with patch.object(settings, "cashu_mints", []), patch.object(
settings, "primary_mint", "http://primary:3338"
):
for p in _patches(proof_amount=1000):
p.start()
try:
details, total_wallet, total_user, owner = await fetch_all_balances(
units=["sat"]
)
finally:
patch.stopall()
assert [d["mint_url"] for d in details] == ["http://primary:3338"]
assert total_wallet == 1000
@pytest.mark.asyncio
async def test_fetch_all_balances_no_duplicate_primary_mint() -> None:
"""primary_mint already in cashu_mints is not inspected twice."""
from routstr.core.settings import settings
with patch.object(
settings, "cashu_mints", ["http://primary:3338"]
), patch.object(settings, "primary_mint", "http://primary:3338"):
for p in _patches(proof_amount=1000):
p.start()
try:
details, total_wallet, _total_user, _owner = await fetch_all_balances(
units=["sat"]
)
finally:
patch.stopall()
assert [d["mint_url"] for d in details] == ["http://primary:3338"]
assert total_wallet == 1000

View File

@@ -11,6 +11,7 @@ import os
from typing import Any, AsyncIterator
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from fastapi.responses import Response, StreamingResponse
@@ -21,6 +22,7 @@ from routstr.core.db import ApiKey # noqa: E402
from routstr.payment.cost_calculation import CostData # noqa: E402
from routstr.payment.models import Architecture, Model, Pricing # noqa: E402
from routstr.upstream.base import BaseUpstreamProvider # noqa: E402
from routstr.wallet import MintConnectionError, TokenConsumedError # noqa: E402
# ---------------------------------------------------------------------------
# Fixtures
@@ -500,7 +502,7 @@ async def test_streaming_emits_sse_and_reconciles_cost_at_end() -> None:
captured_cost_call: dict[str, Any] = {}
async def fake_adjust(
fresh_key: Any, combined_data: Any, sess: Any, max_cost: int
fresh_key: Any, combined_data: Any, sess: Any, max_cost: int, usage: Any = None
) -> dict:
captured_cost_call["combined_data"] = combined_data
captured_cost_call["max_cost"] = max_cost
@@ -589,7 +591,7 @@ async def test_streaming_handles_iterator_yielding_raw_sse_bytes() -> None:
captured: dict[str, Any] = {}
async def fake_adjust(
fresh_key: Any, combined_data: Any, sess: Any, max_cost: int
fresh_key: Any, combined_data: Any, sess: Any, max_cost: int, usage: Any = None
) -> dict:
captured["combined_data"] = combined_data
return fake_cost
@@ -1308,3 +1310,320 @@ async def test_dispatch_uses_url_detected_prefix_for_fireworks_custom_row() -> N
"fireworks_ai/accounts/fireworks/models/glm-5"
)
assert captured_kwargs["api_base"] == "https://api.fireworks.ai/inference/v1"
# ---------------------------------------------------------------------------
# X-Cashu redemption error taxonomy (unreachable mint + string codes)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
@pytest.mark.parametrize(
"error",
[
httpx.ConnectError("All connection attempts failed"),
MintConnectionError("Cashu mint is unreachable"),
TimeoutError("timed out connecting to mint"),
],
)
async def test_x_cashu_mint_unreachable_returns_503(
handler_name: str, error: Exception
) -> None:
"""Both X-Cashu entrypoints classify a down mint as 503 mint_unreachable,
not a generic 400 cashu_error."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 503
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "mint_unreachable"
assert body["error"]["message"] == "Cashu mint is unreachable"
assert body["error"]["code"] == "cashu_mint_unreachable"
if str(error) != body["error"]["message"]:
assert str(error) not in body["error"]["message"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
@pytest.mark.parametrize(
(
"error",
"expected_status",
"expected_type",
"expected_message",
"expected_code",
),
[
(
ValueError("Mint Error: Token already spent"),
400,
"token_already_spent",
"Cashu token already spent",
"cashu_token_already_spent",
),
(
ValueError("invalid token: could not decode"),
400,
"invalid_token",
"Invalid Cashu token",
"invalid_cashu_token",
),
(
# Fee/swap failures now map to a granular 422 on the X-Cashu path,
# matching the bearer path (previously flattened to 400).
ValueError(
"Failed to estimate fees: Fees (7 sat) exceed token amount (5 sat)"
),
422,
"mint_error",
"Token value is too small to cover swap fees",
"cashu_token_swap_fees_exceed_amount",
),
(
ValueError("Failed to melt token from foreign mint http://m: boom"),
422,
"mint_error",
"Failed to swap token from foreign mint",
"cashu_foreign_mint_swap_failed",
),
(
ValueError("some unexpected wallet condition"),
400,
"cashu_error",
"Failed to redeem Cashu token",
"cashu_token_redemption_failed",
),
(
# Non-ValueError faults are internal errors (500), not token errors.
RuntimeError("db exploded"),
500,
"api_error",
"Internal error during token redemption",
"internal_error",
),
],
)
async def test_x_cashu_error_code_is_stable_string(
handler_name: str,
error: Exception,
expected_status: int,
expected_type: str,
expected_message: str,
expected_code: str,
) -> None:
"""X-Cashu emits a stable string ``code`` on every branch, matching the
bearer path's taxonomy instead of an int HTTP status."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == expected_status
body = json.loads(bytes(response.body))
assert body["error"]["type"] == expected_type
assert body["error"]["message"] == expected_message
assert body["error"]["code"] == expected_code
assert isinstance(body["error"]["code"], str)
if str(error) != expected_message:
assert str(error) not in body["error"]["message"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "forward_attr"),
[
("handle_x_cashu", "forward_x_cashu_request"),
("handle_x_cashu_responses", "forward_x_cashu_responses_request"),
],
)
async def test_x_cashu_transport_error_after_redemption_is_not_retryable(
handler_name: str, forward_attr: str
) -> None:
"""A transport failure while forwarding (after the token is spent) maps to
502 upstream_error, never a retryable cashu_mint_unreachable."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with (
patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(return_value=(5_000, "sat", "https://mint")),
),
patch("routstr.upstream.base.store_cashu_transaction", new=AsyncMock()),
patch.object(
provider,
forward_attr,
new=AsyncMock(side_effect=httpx.ConnectError("upstream down")),
),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 502
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "upstream_error"
assert body["error"]["code"] != "cashu_mint_unreachable"
@pytest.mark.asyncio
@pytest.mark.parametrize(
"handler_name",
["handle_x_cashu", "handle_x_cashu_responses"],
)
async def test_x_cashu_token_consumed_returns_500_and_no_echo(
handler_name: str,
) -> None:
"""A post-redemption failure (token spent, crediting/minting failed) is a
non-retryable 500 token_consumed and must NOT echo the spent token back."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(side_effect=TokenConsumedError("credit failed")),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 500
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "token_consumed"
assert body["error"]["code"] == "cashu_token_consumed"
assert "X-Cashu" not in response.headers
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "error", "echoed"),
[
# Spent token: must NOT be echoed.
("handle_x_cashu", ValueError("Token already spent"), False),
("handle_x_cashu_responses", ValueError("Token already spent"), False),
# Unspent but unreachable mint: echo so the client can retry the token.
("handle_x_cashu", MintConnectionError("mint down"), True),
("handle_x_cashu_responses", MintConnectionError("mint down"), True),
# Consumed token (post-redemption): must NOT be echoed.
("handle_x_cashu", TokenConsumedError("credit failed"), False),
("handle_x_cashu_responses", TokenConsumedError("credit failed"), False),
],
)
async def test_x_cashu_echoes_token_only_when_recoverable(
handler_name: str, error: Exception, echoed: bool
) -> None:
provider = _make_provider()
model = _make_model()
request = _make_request()
with patch(
"routstr.upstream.base.recieve_token", new=AsyncMock(side_effect=error)
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
if echoed:
assert response.headers.get("X-Cashu") == "cashuAtoken"
else:
assert "X-Cashu" not in response.headers
@pytest.mark.asyncio
@pytest.mark.parametrize(
("handler_name", "forward_attr"),
[
("handle_x_cashu", "forward_x_cashu_request"),
("handle_x_cashu_responses", "forward_x_cashu_responses_request"),
],
)
@pytest.mark.parametrize("amount", [0, -5])
async def test_x_cashu_zero_value_rejected_not_forwarded(
handler_name: str, forward_attr: str, amount: int
) -> None:
"""A token that redeems to <= 0 must be rejected as cashu_token_zero_value
(400) and NEVER forwarded as a free request — the X-Cashu path lacked the
guard that credit_balance has."""
provider = _make_provider()
model = _make_model()
request = _make_request()
with (
patch(
"routstr.upstream.base.recieve_token",
new=AsyncMock(return_value=(amount, "sat", "https://mint")),
),
patch("routstr.upstream.base.store_cashu_transaction", new=AsyncMock()),
patch.object(
provider,
forward_attr,
new=AsyncMock(side_effect=AssertionError("must not forward a zero-value token")),
),
):
handler = getattr(provider, handler_name)
response = await handler(
request=request,
x_cashu_token="cashuAtoken",
path="v1/chat/completions",
max_cost_for_model=10_000,
model_obj=model,
)
assert response.status_code == 400
body = json.loads(bytes(response.body))
assert body["error"]["type"] == "cashu_error"
assert (
body["error"]["message"]
== "Failed to redeem Cashu token: token yielded no value"
)
assert body["error"]["code"] == "cashu_token_zero_value"
# Spent-to-zero token must not be echoed back for retry.
assert "X-Cashu" not in response.headers

View File

@@ -0,0 +1,55 @@
from __future__ import annotations
import importlib.util
from pathlib import Path
import sqlalchemy as sa
_MIGRATION_PATH = (
Path(__file__).resolve().parents[2]
/ "migrations"
/ "versions"
/ "c6d7e8f9a0b1_add_slug_to_upstream_providers.py"
)
_spec = importlib.util.spec_from_file_location("provider_slug_migration", _MIGRATION_PATH)
assert _spec is not None and _spec.loader is not None
migration = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(migration)
def test_slug_migration_backfill_uses_api_safe_deterministic_slugs() -> None:
engine = sa.create_engine("sqlite:///:memory:")
with engine.begin() as conn:
conn.execute(
sa.text(
"CREATE TABLE upstream_providers ("
"id INTEGER PRIMARY KEY, "
"provider_type VARCHAR NOT NULL, "
"slug VARCHAR NULL"
")"
)
)
conn.execute(
sa.text(
"INSERT INTO upstream_providers (id, provider_type, slug) VALUES "
"(1, 'OpenAI Compatible', NULL), "
"(2, 'OpenAI Compatible', ''), "
"(3, '123', NULL), "
"(4, 'x', NULL), "
"(5, 'anthropic', 'anthropic')"
)
)
migration._backfill_provider_slugs(conn)
rows = conn.execute(
sa.text("SELECT id, slug FROM upstream_providers ORDER BY id")
).all()
assert rows == [
(1, "openai-compatible"),
(2, "openai-compatible-2"),
(3, "provider-123"),
(4, "x-provider"),
(5, "anthropic"),
]

View File

@@ -0,0 +1,144 @@
from __future__ import annotations
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.admin import _get_upstream_provider_by_ref
from routstr.core.db import UpstreamProviderRow
from routstr.core.provider_slugs import (
allocate_unique_provider_slug,
provider_slug_base,
)
from routstr.upstream.helpers import _seed_providers_from_settings
@pytest.mark.asyncio
async def test_allocate_unique_provider_slug_is_deterministic_with_suffixes() -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
async with AsyncSession(engine) as session:
session.add(
UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://api.openai.com/v1",
api_key="key-1",
)
)
await session.commit()
assert await allocate_unique_provider_slug(session, "openai") == "openai-2"
assert (
await allocate_unique_provider_slug(session, "openai", {"openai-2"})
== "openai-3"
)
await engine.dispose()
def test_provider_slug_base_sanitizes_provider_type() -> None:
assert provider_slug_base("OpenAI Compatible") == "openai-compatible"
assert provider_slug_base("!!!") == "provider"
assert provider_slug_base("AI") == "ai-provider"
assert provider_slug_base("123") == "provider-123"
@pytest.mark.asyncio
async def test_provider_ref_lookup_accepts_existing_numeric_ids_and_slugs() -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
async with AsyncSession(engine) as session:
provider = UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://api.openai.com/v1",
api_key="key-1",
)
session.add(provider)
await session.commit()
await session.refresh(provider)
by_id = await _get_upstream_provider_by_ref(session, str(provider.id))
by_slug = await _get_upstream_provider_by_ref(session, "openai")
assert by_id.id == provider.id
assert by_slug.id == provider.id
await engine.dispose()
@pytest.mark.asyncio
async def test_seed_providers_from_settings_sets_deterministic_slug(
monkeypatch: pytest.MonkeyPatch,
) -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
monkeypatch.setenv("OPENAI_API_KEY", "seeded-openai-key")
class SettingsStub:
chat_completions_api_version: str | None = None
upstream_base_url: str | None = None
upstream_api_key: str = ""
async with AsyncSession(engine) as session:
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
result = await session.exec(select(UpstreamProviderRow))
providers: list[UpstreamProviderRow] = list(result.all())
assert [(p.provider_type, p.slug) for p in providers] == [("openai", "openai")]
await engine.dispose()
@pytest.mark.asyncio
async def test_seed_providers_from_settings_keeps_slug_stable_on_reseed(
monkeypatch: pytest.MonkeyPatch,
) -> None:
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
monkeypatch.setenv("OPENAI_API_KEY", "seeded-openai-key")
class SettingsStub:
chat_completions_api_version: str | None = None
upstream_base_url: str | None = None
upstream_api_key: str = ""
async with AsyncSession(engine) as session:
session.add(
UpstreamProviderRow(
slug="openai",
provider_type="openai",
base_url="https://example.invalid/v1",
api_key="other-key",
)
)
await session.commit()
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
await _seed_providers_from_settings(session, SettingsStub()) # type: ignore[arg-type]
await session.commit()
result = await session.exec(
select(UpstreamProviderRow).order_by(UpstreamProviderRow.slug)
)
providers: list[UpstreamProviderRow] = list(result.all())
assert [(p.provider_type, p.slug) for p in providers] == [
("openai", "openai"),
("openai", "openai-2"),
]
await engine.dispose()

View File

@@ -0,0 +1,156 @@
"""Unit tests for the reactive request-correction layer.
Covers the recovery path that lets a request survive a 400 where the upstream
names a single unsupported request param (e.g. newer Anthropic models
deprecating ``temperature``): the param is stripped from the JSON body and the
same upstream is retried, provider-agnostically, keyed off the error text.
"""
from __future__ import annotations
import json
from fastapi.responses import Response
from routstr.upstream.request_correction import (
Correction,
correct_request,
extract_error_message,
strip_unsupported_param,
)
def _body(**kwargs: object) -> bytes:
return json.dumps(kwargs).encode()
class TestCorrectRequest:
def test_strips_deprecated_temperature(self) -> None:
body = _body(model="claude-opus-4-8", temperature=1, messages=[])
result = correct_request(
body, "`temperature` is deprecated for this model.", set()
)
assert isinstance(result, Correction)
assert result.label == "temperature"
decoded = json.loads(result.body)
assert "temperature" not in decoded
assert decoded["model"] == "claude-opus-4-8"
def test_strips_not_supported_param(self) -> None:
body = _body(model="m", top_p=0.9, messages=[])
result = correct_request(body, "Parameter 'top_p' is not supported", set())
assert result is not None
assert result.label == "top_p"
assert "top_p" not in json.loads(result.body)
def test_returns_none_when_label_already_applied(self) -> None:
body = _body(model="m", temperature=1)
assert (
correct_request(body, "`temperature` is deprecated", {"temperature"})
is None
)
def test_returns_none_when_param_absent_from_body(self) -> None:
body = _body(model="m", messages=[])
assert correct_request(body, "`temperature` is deprecated", set()) is None
def test_returns_none_when_message_does_not_match(self) -> None:
body = _body(model="m", temperature=1)
assert correct_request(body, "Insufficient balance", set()) is None
def test_returns_none_on_empty_inputs(self) -> None:
assert correct_request(b"", "`temperature` is deprecated", set()) is None
assert correct_request(_body(temperature=1), "", set()) is None
def test_returns_none_on_non_object_body(self) -> None:
assert correct_request(b"[1, 2, 3]", "`temperature` is deprecated", set()) is None
def test_deprecated_model_name_is_not_stripped_as_param(self) -> None:
"""A 'model is deprecated' error must not strip an unrelated body field.
The regex matches the ``<token> is deprecated`` wording, but the
``param not in body`` guard means a deprecated *model* name (not a
request param) yields no correction rather than a false strip.
"""
body = _body(model="gpt-3", temperature=1, messages=[])
assert correct_request(body, "`gpt-3` is deprecated, use gpt-4", set()) is None
def test_streaming_400_buffered_error_is_correctable(self) -> None:
"""Streaming 400s funnel through a buffered JSON Response, so the same
correction path applies as for non-streaming requests."""
# Mirrors forward_upstream_error_response's buffered JSON envelope.
resp = Response(
content=json.dumps(
{"error": {"message": "`temperature` is deprecated for this model"}}
).encode(),
status_code=400,
)
body = _body(model="claude-opus-4-8", temperature=1, messages=[])
result = correct_request(body, extract_error_message(resp), set())
assert isinstance(result, Correction)
assert result.label == "temperature"
assert "temperature" not in json.loads(result.body)
class TestStripUnsupportedParam:
def test_does_not_mutate_input(self) -> None:
body = {"model": "m", "temperature": 1}
result = strip_unsupported_param(body, "`temperature` is deprecated")
assert result is not None
new_body, param = result
assert param == "temperature"
assert "temperature" not in new_body
# original untouched (immutability)
assert body == {"model": "m", "temperature": 1}
def test_declines_when_no_match(self) -> None:
assert strip_unsupported_param({"temperature": 1}, "nope") is None
class TestExtractErrorMessage:
def test_extracts_nested_error_message(self) -> None:
resp = Response(
content=json.dumps(
{"error": {"message": "`temperature` is deprecated", "type": "x"}}
).encode(),
status_code=400,
)
assert extract_error_message(resp) == "`temperature` is deprecated"
def test_extracts_string_error(self) -> None:
resp = Response(
content=json.dumps({"error": "bad request"}).encode(), status_code=400
)
assert extract_error_message(resp) == "bad request"
def test_extracts_top_level_message(self) -> None:
resp = Response(
content=json.dumps({"message": "nope"}).encode(), status_code=400
)
assert extract_error_message(resp) == "nope"
def test_empty_body_returns_empty_string(self) -> None:
assert extract_error_message(Response(status_code=400)) == ""
def test_non_json_body_returns_preview(self) -> None:
resp = Response(content=b"plain text error", status_code=400)
assert extract_error_message(resp) == "plain text error"
class TestEndToEndChaining:
def test_two_distinct_params_corrected_sequentially(self) -> None:
"""Simulates the proxy loop: each 400 fixes one param, set guards reuse."""
body = _body(model="m", temperature=1, top_p=0.5, messages=[])
applied: set[str] = set()
first = correct_request(body, "`temperature` is deprecated", applied)
assert first is not None
body, applied = first.body, applied | {first.label}
second = correct_request(body, "`top_p` is not supported", applied)
assert second is not None
body, applied = second.body, applied | {second.label}
decoded = json.loads(body)
assert "temperature" not in decoded and "top_p" not in decoded
assert applied == {"temperature", "top_p"}

View File

@@ -0,0 +1,95 @@
import json
import os
os.environ.setdefault("UPSTREAM_BASE_URL", "http://test")
os.environ.setdefault("UPSTREAM_API_KEY", "test")
os.environ.setdefault("LIGHTNING_ADDRESS", "test@stm.to")
from routstr.upstream import GenericUpstreamProvider
from routstr.upstream.openrouter import OpenRouterUpstreamProvider
def _model(model_id: str = "openai/gpt-4o"): # type: ignore[no-untyped-def]
from routstr.payment.models import Architecture, Model, Pricing
return Model(
id=model_id,
name=model_id,
created=0,
description="",
context_length=128000,
architecture=Architecture(
modality="text->text",
input_modalities=["text"],
output_modalities=["text"],
tokenizer="GPT",
instruct_type=None,
),
pricing=Pricing(prompt=0.0, completion=0.0),
)
def _tool_body() -> dict:
return {
"model": "openai/gpt-4o",
"messages": [{"role": "user", "content": "What's the weather?"}],
"tools": [
{
"type": "function",
"function": {"name": "get_weather", "parameters": {}},
}
],
}
def _prepare(provider, body: dict) -> dict: # type: ignore[no-untyped-def]
out = provider.prepare_request_body(json.dumps(body).encode(), _model())
assert out is not None
return json.loads(out)
def test_injects_require_parameters_for_tool_request() -> None:
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), _tool_body())
assert data["provider"]["require_parameters"] is True
def test_generic_provider_on_openrouter_url_is_left_alone() -> None:
# Only OpenRouterUpstreamProvider injects; a generic provider pointed at the
# same base URL doesn't.
provider = GenericUpstreamProvider(base_url="https://openrouter.ai/api/v1")
data = _prepare(provider, _tool_body())
assert "provider" not in data
def test_no_injection_without_tools() -> None:
body = {"model": "openai/gpt-4o", "messages": [{"role": "user", "content": "hi"}]}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert "provider" not in data
def test_empty_tools_list_does_not_inject() -> None:
body = _tool_body()
body["tools"] = []
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert "provider" not in data
def test_direct_provider_does_not_inject() -> None:
provider = GenericUpstreamProvider(base_url="https://api.openai.com/v1")
data = _prepare(provider, _tool_body())
assert "provider" not in data
def test_keeps_client_set_require_parameters() -> None:
body = _tool_body()
body["provider"] = {"require_parameters": False}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert data["provider"]["require_parameters"] is False
def test_preserves_other_provider_fields() -> None:
body = _tool_body()
body["provider"] = {"order": ["openai", "azure"]}
data = _prepare(OpenRouterUpstreamProvider(api_key="test"), body)
assert data["provider"]["order"] == ["openai", "azure"]
assert data["provider"]["require_parameters"] is True

View File

@@ -0,0 +1,378 @@
"""Tests for stale reserved_balance handling (issue #551).
Covers:
- pay_for_request stamping reserved_at on billing and child keys
- release_stale_reservations sweeper semantics
- reset_all_reserved_balances clearing reserved_at
- refund endpoint self-healing stale/legacy reservations
- proxy reverting the reservation when the client disconnects (CancelledError)
"""
import asyncio
import time
from typing import AsyncGenerator
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlalchemy.pool import StaticPool
from sqlmodel import SQLModel
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.auth import pay_for_request
from routstr.balance import refund_wallet_endpoint
from routstr.core.db import (
ApiKey,
release_stale_reservations,
reset_all_reserved_balances,
)
def _make_engine() -> AsyncEngine:
return create_async_engine(
"sqlite+aiosqlite://",
poolclass=StaticPool,
connect_args={"check_same_thread": False},
)
@pytest.fixture
async def session() -> "AsyncGenerator[AsyncSession, None]":
engine = _make_engine()
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
db_session = AsyncSession(engine, expire_on_commit=False)
try:
yield db_session
finally:
await db_session.close()
await engine.dispose()
# ---------------------------------------------------------------------------
# pay_for_request stamps reserved_at
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_pay_for_request_sets_reserved_at(session: AsyncSession) -> None:
key = ApiKey(hashed_key="paykey", balance=10_000)
session.add(key)
await session.commit()
before = int(time.time())
await pay_for_request(key, 1_000, session)
await session.refresh(key)
assert key.reserved_balance == 1_000
assert key.reserved_at is not None
assert key.reserved_at >= before
@pytest.mark.asyncio
async def test_pay_for_request_sets_reserved_at_on_child_key(session: AsyncSession) -> None:
parent = ApiKey(hashed_key="parentkey", balance=10_000)
child = ApiKey(hashed_key="childkey", balance=0, parent_key_hash="parentkey")
session.add(parent)
session.add(child)
await session.commit()
await pay_for_request(child, 1_000, session)
await session.refresh(parent)
await session.refresh(child)
assert parent.reserved_balance == 1_000
assert parent.reserved_at is not None
assert child.reserved_balance == 1_000
assert child.reserved_at is not None
@pytest.mark.asyncio
async def test_revert_clears_reserved_at_when_fully_released(
session: AsyncSession,
) -> None:
from routstr.auth import revert_pay_for_request
key = ApiKey(hashed_key="revertkey", balance=10_000)
session.add(key)
await session.commit()
await pay_for_request(key, 1_000, session)
reverted = await revert_pay_for_request(key, session, 1_000)
assert reverted is True
await session.refresh(key)
assert key.reserved_balance == 0
assert key.reserved_at is None
@pytest.mark.asyncio
async def test_revert_keeps_reserved_at_while_other_reservations_remain(
session: AsyncSession,
) -> None:
from routstr.auth import revert_pay_for_request
key = ApiKey(hashed_key="partialrevert", balance=10_000)
session.add(key)
await session.commit()
await pay_for_request(key, 1_000, session)
await pay_for_request(key, 1_000, session)
reverted = await revert_pay_for_request(key, session, 1_000)
assert reverted is True
await session.refresh(key)
assert key.reserved_balance == 1_000
assert key.reserved_at is not None
# ---------------------------------------------------------------------------
# release_stale_reservations sweeper
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_release_stale_reservations_releases_old(session: AsyncSession) -> None:
key = ApiKey(
hashed_key="stalekey",
balance=5_000,
reserved_balance=1_000,
reserved_at=int(time.time()) - 1_000,
)
session.add(key)
await session.commit()
released = await release_stale_reservations(session, max_age_seconds=300)
assert released == 1
await session.refresh(key)
assert key.reserved_balance == 0
assert key.reserved_at is None
@pytest.mark.asyncio
async def test_release_stale_reservations_keeps_fresh(session: AsyncSession) -> None:
key = ApiKey(
hashed_key="freshkey",
balance=5_000,
reserved_balance=1_000,
reserved_at=int(time.time()),
)
session.add(key)
await session.commit()
released = await release_stale_reservations(session, max_age_seconds=300)
assert released == 0
await session.refresh(key)
assert key.reserved_balance == 1_000
assert key.reserved_at is not None
@pytest.mark.asyncio
async def test_release_stale_reservations_skips_null_reserved_at(session: AsyncSession) -> None:
# Reservations without a timestamp may belong to instances running older
# code (rolling deploy) — the background sweeper must not touch them.
key = ApiKey(
hashed_key="legacykey",
balance=5_000,
reserved_balance=1_000,
reserved_at=None,
)
session.add(key)
await session.commit()
released = await release_stale_reservations(session, max_age_seconds=300)
assert released == 0
await session.refresh(key)
assert key.reserved_balance == 1_000
@pytest.mark.asyncio
async def test_reset_all_reserved_balances_clears_reserved_at(session: AsyncSession) -> None:
key = ApiKey(
hashed_key="resetkey",
balance=5_000,
reserved_balance=1_000,
reserved_at=int(time.time()),
)
session.add(key)
await session.commit()
await reset_all_reserved_balances(session)
await session.refresh(key)
assert key.reserved_balance == 0
assert key.reserved_at is None
# ---------------------------------------------------------------------------
# Refund endpoint self-healing
# ---------------------------------------------------------------------------
def _refund_patches(refund_token: str = "cashuArefund"): # type: ignore[no-untyped-def]
return (
patch("routstr.balance.send_token", AsyncMock(return_value=refund_token)),
patch("routstr.balance.store_cashu_transaction", AsyncMock()),
patch("routstr.balance._refund_cache_get", AsyncMock(return_value=None)),
patch("routstr.balance._refund_cache_set", AsyncMock()),
)
async def _add_key(session: AsyncSession, **kwargs) -> ApiKey: # type: ignore[no-untyped-def]
key = ApiKey(refund_currency="sat", **kwargs)
session.add(key)
await session.commit()
return key
@pytest.mark.asyncio
async def test_refund_self_heals_stale_reservation(session: AsyncSession) -> None:
key = await _add_key(
session,
hashed_key="stalerefund",
balance=5_000,
reserved_balance=2_000,
reserved_at=int(time.time()) - 10_000,
)
p1, p2, p3, p4 = _refund_patches()
with p1, p2, p3, p4:
result = await refund_wallet_endpoint(
authorization="Bearer sk-stalerefund",
x_cashu=None,
session=session,
)
assert isinstance(result, dict)
assert result["token"] == "cashuArefund"
# Full balance refunded (5000 msats -> 5 sats), reservation healed
assert result["sats"] == "5"
await session.refresh(key)
assert key.balance == 0
assert key.reserved_balance == 0
assert key.reserved_at is None
@pytest.mark.asyncio
async def test_refund_self_heals_legacy_null_reserved_at(session: AsyncSession) -> None:
# Keys stuck from before reserved_at existed must be refundable.
key = await _add_key(
session,
hashed_key="legacyrefund",
balance=5_000,
reserved_balance=2_000,
reserved_at=None,
)
p1, p2, p3, p4 = _refund_patches()
with p1, p2, p3, p4:
result = await refund_wallet_endpoint(
authorization="Bearer sk-legacyrefund",
x_cashu=None,
session=session,
)
assert isinstance(result, dict)
assert result["token"] == "cashuArefund"
await session.refresh(key)
assert key.balance == 0
assert key.reserved_balance == 0
@pytest.mark.asyncio
async def test_refund_rejects_recent_reservation(session: AsyncSession) -> None:
from fastapi import HTTPException
await _add_key(
session,
hashed_key="activerefund",
balance=5_000,
reserved_balance=2_000,
reserved_at=int(time.time()),
)
p1, p2, p3, p4 = _refund_patches()
with p1, p2, p3, p4:
with pytest.raises(HTTPException) as exc_info:
await refund_wallet_endpoint(
authorization="Bearer sk-activerefund",
x_cashu=None,
session=session,
)
assert exc_info.value.status_code == 400
assert "ongoing requests" in exc_info.value.detail
@pytest.mark.asyncio
async def test_refund_without_reservation_still_works(session: AsyncSession) -> None:
key = await _add_key(
session,
hashed_key="plainrefund",
balance=5_000,
reserved_balance=0,
)
p1, p2, p3, p4 = _refund_patches()
with p1, p2, p3, p4:
result = await refund_wallet_endpoint(
authorization="Bearer sk-plainrefund",
x_cashu=None,
session=session,
)
assert isinstance(result, dict)
assert result["token"] == "cashuArefund"
await session.refresh(key)
assert key.balance == 0
# ---------------------------------------------------------------------------
# Proxy reverts reservation on client disconnect
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_proxy_reverts_reservation_on_client_disconnect() -> None:
from routstr import proxy as proxy_module
key = ApiKey(hashed_key="cancelkey", balance=10_000)
request = MagicMock()
request.method = "POST"
request.headers = {"authorization": "Bearer sk-cancelkey"}
request.body = AsyncMock(return_value=b'{"model": "test-model"}')
upstream = MagicMock()
upstream.provider_type = "test"
upstream.prepare_headers = MagicMock(side_effect=lambda h: h)
upstream.forward_request = AsyncMock(side_effect=asyncio.CancelledError())
session = MagicMock()
revert_mock = AsyncMock(return_value=True)
with (
patch.object(proxy_module, "get_model_instance", return_value=MagicMock()),
patch.object(proxy_module, "get_provider_for_model", return_value=[upstream]),
patch.object(
proxy_module, "get_max_cost_for_model", AsyncMock(return_value=1_000)
),
patch.object(
proxy_module,
"calculate_discounted_max_cost",
AsyncMock(return_value=1_000),
),
patch.object(proxy_module, "check_token_balance", MagicMock()),
patch.object(
proxy_module, "get_bearer_token_key", AsyncMock(return_value=key)
),
patch.object(proxy_module, "pay_for_request", AsyncMock(return_value=1_000)),
patch.object(proxy_module, "revert_pay_for_request", revert_mock),
):
with pytest.raises(asyncio.CancelledError):
await proxy_module.proxy(request, "v1/chat/completions", session=session)
revert_mock.assert_awaited_once_with(key, session, 1_000)

View File

@@ -337,3 +337,75 @@ async def test_multiline_non_json_data_each_line_prefixed() -> None:
continue
assert line.startswith(b"data: "), f"bare line leaked to client: {line!r}"
assert b"data: line one" in blob and b"data: line two" in blob
@pytest.mark.asyncio
async def test_crlf_delimiter_split_across_chunk_boundary() -> None:
"""CRLF event delimiter straddling two TCP reads must not merge events.
Regression: a per-chunk ``replace(b"\\r\\n", b"\\n")`` left a stray ``\\r``
when a ``\\r\\n`` of the ``\\r\\n\\r\\n`` delimiter landed at the very end of
one ``aiter_bytes`` chunk and the matching ``\\n`` opened the next. The
``\\n\\n`` split then missed the boundary, glued two events into one frame
with two ``data:`` lines, and the client's ``JSON.parse`` threw on the
concatenated payload (the "unexpected token"/"Extra data" crash).
"""
e1 = b'data: {"id":"x","choices":[{"delta":{"content":"a"}}]}'
e2 = b'data: {"id":"x","choices":[{"delta":{"content":"b"}}]}'
chunks = [
e1 + b"\r\n\r", # delimiter cut mid-CRLF
b"\n" + e2 + b"\r\n\r\n",
b"data: [DONE]\r\n\r\n",
]
out = await _drive(chunks)
# Client-accurate check: a real SSE client concatenates all ``data:`` lines
# *within one event* (events are ``\n\n``-delimited) before parsing. A
# merged frame would surface here as two objects glued into one payload,
# which ``_assert_clean`` (per-line) would miss.
blob = b"".join(out)
contents: list[str] = []
for event in blob.split(b"\n\n"):
datas = [
ln[len(b"data: ") :]
for ln in event.split(b"\n")
if ln.startswith(b"data: ")
]
if not datas:
continue
payload = b"".join(datas)
if payload.strip() == b"[DONE]":
continue
obj = json.loads(payload) # raises if two events were merged into one
for c in obj.get("choices", []):
if "delta" in c:
contents.append(c["delta"]["content"])
assert contents == ["a", "b"]
@pytest.mark.asyncio
async def test_truncated_json_tail_on_connection_close() -> None:
"""A stream that drops mid-event must not emit the partial JSON downstream.
Regression: the end-of-stream flush ran ``_process_event`` on the leftover
buffer unconditionally. When the upstream connection closed mid-event the
leftover was incomplete JSON, which fell through to the raw-forward path and
handed the client a ``data: {partial`` frame -> ``Unterminated string`` parse
error. The truncated tail must be dropped instead.
"""
chunks = [
b'data: {"id":"x","choices":[{"delta":{"content":"ok"}}]}\n\n',
b'data: {"id":"x","choices":[{"delta":{"con', # connection dies here
]
out = await _drive(chunks)
objs = _assert_clean(out) # raises if the partial tail leaked as a data frame
contents = [
c["delta"]["content"]
for o in objs
for c in o.get("choices", [])
if "delta" in c
]
# The one complete chunk is delivered; the truncated fragment is dropped
# entirely (no second delta), and _assert_clean above guarantees nothing
# non-JSON ever reached the client.
assert contents == ["ok"]

View File

@@ -0,0 +1,524 @@
"""Unit tests for ``GenericUpstreamProvider.fetch_models`` price/metadata resolution.
A generic upstream is any OpenAI-compatible API. Most (DeepSeek, OpenAI,
Groq, ...) answer ``/models`` with bare ``{id, object, owned_by}`` entries that
carry *no* pricing. The provider must not fabricate a price for those: it
resolves through native ``model_spec`` (Venice's bespoke schema) → litellm's
bundled cost map → the OpenRouter feed, and only when every source misses does
it import the model **disabled** with a warning rather than invent a number.
These tests drive that behaviour through the public ``fetch_models`` API. The
``/models`` HTTP call is faked at ``httpx.AsyncClient``; the OpenRouter feed is
patched at its source (``routstr.payment.models.async_fetch_openrouter_models``)
so the resolver's lazy import picks up the stub. litellm's real bundled cost map
is used unmocked — the DeepSeek rates it ships are the assertion's ground truth.
"""
from __future__ import annotations
import logging
from typing import Any
from unittest.mock import AsyncMock, patch
import pytest
from routstr.upstream.generic import GenericUpstreamProvider
class _FakeResponse:
def __init__(self, payload: dict[str, Any]) -> None:
self._payload = payload
def raise_for_status(self) -> None:
return None
def json(self) -> dict[str, Any]:
return self._payload
class _FakeAsyncClient:
"""Stand-in for ``httpx.AsyncClient`` returning a canned ``/models`` body."""
def __init__(self, payload: dict[str, Any]) -> None:
self._payload = payload
async def __aenter__(self) -> "_FakeAsyncClient":
return self
async def __aexit__(self, *exc: object) -> bool:
return False
async def get(self, url: str, headers: dict[str, str] | None = None) -> _FakeResponse:
return _FakeResponse(self._payload)
def _patch_models_endpoint(payload: dict[str, Any]) -> Any:
"""Patch the provider's ``httpx.AsyncClient`` to serve ``payload``."""
return patch(
"routstr.upstream.generic.httpx.AsyncClient",
lambda *args, **kwargs: _FakeAsyncClient(payload),
)
def _model_by_id(models: list[Any], model_id: str) -> Any:
return next(m for m in models if m.id == model_id)
# ---------------------------------------------------------------------------
# native model_spec (Venice) — must keep resolving, and capture its metadata
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_native_model_spec_resolves_and_captures_metadata() -> None:
"""A Venice-shaped ``model_spec`` is authoritative: prices/context come
straight from it and vision capability becomes an image input modality."""
payload = {
"data": [
{
"id": "venice-llama",
"owned_by": "venice",
"model_spec": {
"name": "Venice Llama",
"availableContextTokens": 65536,
"pricing": {
"input": {"usd": 0.5},
"output": {"usd": 1.5},
},
"capabilities": {"supportsVision": True},
},
}
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "venice-llama")
assert model.enabled is True
assert model.pricing.prompt == pytest.approx(0.5 / 1_000_000)
assert model.pricing.completion == pytest.approx(1.5 / 1_000_000)
assert model.context_length == 65536
assert "image" in model.architecture.input_modalities
# Vision capability must be reflected in the combined modality string, not
# flattened to "text->text".
assert model.architecture.modality == "text+image->text"
# A native price never needs the OpenRouter feed.
or_feed.assert_not_awaited()
# ---------------------------------------------------------------------------
# native model_spec validation — a bogus native price is not authoritative
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_native_both_zero_price_falls_through_to_litellm() -> None:
"""A native ``model_spec`` that prices both tokens at 0 is not a real price
(the same free-tier trap the litellm/OpenRouter rungs already reject). It
must not be treated as authoritative and served free; the resolver falls
through, so a litellm-known model lands on litellm's real rate instead."""
payload = {
"data": [
{
"id": "deepseek-chat",
"owned_by": "deepseek",
"model_spec": {
"pricing": {"input": {"usd": 0}, "output": {"usd": 0}},
},
}
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "deepseek-chat")
assert model.enabled is True
assert model.pricing.prompt == pytest.approx(2.8e-07)
assert model.pricing.completion == pytest.approx(4.2e-07)
@pytest.mark.asyncio
async def test_native_negative_price_falls_through_to_litellm() -> None:
"""A negative native price would credit the caller's balance on every
request (a fund drain, not a discount). Reject it like any other unusable
price and fall through to the chain."""
payload = {
"data": [
{
"id": "deepseek-chat",
"owned_by": "deepseek",
"model_spec": {
"pricing": {"input": {"usd": -0.5}, "output": {"usd": -1.5}},
},
}
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "deepseek-chat")
assert model.enabled is True
assert model.pricing.prompt == pytest.approx(2.8e-07)
assert model.pricing.completion == pytest.approx(4.2e-07)
@pytest.mark.asyncio
async def test_native_non_numeric_price_does_not_break_catalog() -> None:
"""A non-numeric native price (``"free"``) must not raise while parsing —
an unguarded ``"free" / 1_000_000`` throws and the outer catch drops the
*entire* provider catalog. It has to fail closed for that one model while
every other model in the same response still resolves."""
payload = {
"data": [
{
"id": "broken-price",
"owned_by": "mystery",
"model_spec": {
"pricing": {"input": {"usd": "free"}, "output": {"usd": "free"}},
},
},
{"id": "deepseek-chat", "object": "model", "owned_by": "deepseek"},
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
# One malformed entry must not empty the catalog.
assert {m.id for m in models} == {"broken-price", "deepseek-chat"}
broken = _model_by_id(models, "broken-price")
assert broken.enabled is False
healthy = _model_by_id(models, "deepseek-chat")
assert healthy.enabled is True
assert healthy.pricing.prompt == pytest.approx(2.8e-07)
# ---------------------------------------------------------------------------
# litellm rescue — the money-critical case (DeepSeek bare /models)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_bare_deepseek_resolves_via_litellm() -> None:
"""DeepSeek's ``/models`` carries no pricing. The old code fabricated
``$0.001`` + ctx 4096; the resolver must instead pull DeepSeek's real
rates from litellm's bundled cost map (``$0.28``/``$0.42`` per 1M, ctx
131072) and keep the model enabled."""
payload = {
"data": [
{"id": "deepseek-chat", "object": "model", "owned_by": "deepseek"},
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "deepseek-chat")
assert model.enabled is True
assert model.pricing.prompt == pytest.approx(2.8e-07)
assert model.pricing.completion == pytest.approx(4.2e-07)
assert model.context_length == 131072
# Richer metadata than the two base prices is captured too.
assert model.pricing.input_cache_read == pytest.approx(2.8e-08)
assert model.top_provider is not None
assert model.top_provider.max_completion_tokens == 8192
# litellm answered, so the OpenRouter feed is never consulted.
or_feed.assert_not_awaited()
@pytest.mark.asyncio
async def test_litellm_zero_price_entry_fails_closed(
caplog: pytest.LogCaptureFixture,
) -> None:
"""A litellm entry that lists a model but prices it at 0/0 (free-tier
moderation/rerank models do this) is not a real price — treating it as one
would silently serve the model for free. The resolver must reject a both-zero
litellm hit and fall through, so the model imports disabled, not at $0."""
payload = {
"data": [
{"id": "omni-moderation-latest", "object": "model", "owned_by": "openai"},
]
}
gen_logger = logging.getLogger("routstr.upstream.generic")
gen_logger.addHandler(caplog.handler)
try:
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(
base_url="http://x"
).fetch_models()
finally:
gen_logger.removeHandler(caplog.handler)
model = _model_by_id(models, "omni-moderation-latest")
assert model.enabled is False
assert model.pricing.prompt == 0.0
assert model.pricing.completion == 0.0
assert any(
"omni-moderation-latest" in rec.getMessage()
for rec in caplog.records
if rec.levelno >= logging.WARNING
)
@pytest.mark.asyncio
async def test_litellm_output_cap_not_used_as_context() -> None:
"""litellm's ``max_tokens`` is the completion cap, not the context window
(it tracks ``max_output_tokens`` for ~94% of models). When a model reports
no ``max_input_tokens``, the resolver must not smuggle the output cap in as
the context window; it falls back to the id-based estimate instead, while
``max_tokens`` still feeds the completion limit."""
payload = {
"data": [
{
"id": "gemini/gemini-gemma-2-9b-it",
"object": "model",
"owned_by": "google",
},
]
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "gemini/gemini-gemma-2-9b-it")
assert model.enabled is True
# litellm gives this model max_input_tokens=None, max_tokens=8192 (an output
# cap). Context must come from the estimate (4096), never the 8192 cap.
assert model.context_length == 4096
# The 8192 output cap still lands where it belongs: the completion limit.
assert model.top_provider is not None
assert model.top_provider.max_completion_tokens == 8192
# ---------------------------------------------------------------------------
# OpenRouter fallback — litellm misses, OR carries a full payload
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_unknown_to_litellm_resolves_via_openrouter() -> None:
"""A model litellm has never heard of still resolves if the OpenRouter
feed lists it, pulling price + context from that entry."""
payload = {
"data": [
{"id": "exotic/model-9000", "object": "model", "owned_by": "exotic"},
]
}
or_entry = {
"id": "exotic/model-9000",
"name": "Exotic 9000",
"context_length": 65536,
"architecture": {
"modality": "text+image->text",
"input_modalities": ["text", "image"],
"output_modalities": ["text"],
"tokenizer": "Other",
"instruct_type": None,
},
"pricing": {"prompt": "0.000005", "completion": "0.000010"},
"top_provider": {
"context_length": 65536,
"max_completion_tokens": 4096,
"is_moderated": False,
},
}
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[or_entry])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "exotic/model-9000")
assert model.enabled is True
assert model.pricing.prompt == pytest.approx(5e-06)
assert model.pricing.completion == pytest.approx(1e-05)
assert model.context_length == 65536
# The feed's own modality string is carried through verbatim, not recomputed.
assert model.architecture.modality == "text+image->text"
or_feed.assert_awaited()
@pytest.mark.asyncio
async def test_openrouter_bare_tail_collision_picks_highest_price() -> None:
"""When a bare model id matches several OpenRouter entries by tail
(``model`` ↔ ``a/model``, ``b/model``), the match must be deterministic and
money-safe: pick the highest-priced candidate regardless of feed order, so
ordering can never leave the node charging below true cost. (The live feed
has zero such collisions today; this guards the latent case.)"""
payload = {
"data": [
{"id": "zzz-phantom-model", "object": "model", "owned_by": "mystery"},
]
}
# Same bare tail, different resellers; the pricier one is listed *second*
# so a first-wins match would pick the cheaper (undercharging) entry.
or_feed = AsyncMock(
return_value=[
{
"id": "cheapco/zzz-phantom-model",
"context_length": 8192,
"pricing": {"prompt": "0.000001", "completion": "0.000002"},
},
{
"id": "premiumco/zzz-phantom-model",
"context_length": 8192,
"pricing": {"prompt": "0.000009", "completion": "0.000010"},
},
]
)
with _patch_models_endpoint(payload):
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "zzz-phantom-model")
assert model.pricing.prompt == pytest.approx(9e-06)
assert model.pricing.completion == pytest.approx(1e-05)
@pytest.mark.asyncio
async def test_openrouter_bare_tail_tie_breaks_on_combined_cost() -> None:
"""The bare-tail tie-break must weigh *both* rates, not prompt alone.
Given two colliding entries where one is cheaper on prompt but far dearer
on completion, ranking by prompt would pick the entry that undercharges
output-heavy traffic. Pick the highest *combined* per-token cost so the
money-safe choice holds whichever way the traffic leans."""
payload = {
"data": [
{"id": "yyy-phantom-model", "object": "model", "owned_by": "mystery"},
]
}
# dear-overall is listed first with the *lower* prompt, so a prompt-only max
# would wrongly pick the second (cheaper-overall) entry.
or_feed = AsyncMock(
return_value=[
{
"id": "dearco/yyy-phantom-model",
"context_length": 8192,
"pricing": {"prompt": "0.000001", "completion": "0.000100"},
},
{
"id": "cheapco/yyy-phantom-model",
"context_length": 8192,
"pricing": {"prompt": "0.000009", "completion": "0.000002"},
},
]
)
with _patch_models_endpoint(payload):
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
model = _model_by_id(models, "yyy-phantom-model")
assert model.pricing.prompt == pytest.approx(1e-06)
assert model.pricing.completion == pytest.approx(1e-04)
@pytest.mark.asyncio
async def test_openrouter_feed_fetched_once_per_discovery() -> None:
"""Two models both missing litellm must share a single OpenRouter fetch —
the feed is not re-downloaded per model."""
payload = {
"data": [
{"id": "exotic/model-a", "object": "model", "owned_by": "exotic"},
{"id": "exotic/model-b", "object": "model", "owned_by": "exotic"},
]
}
or_feed = AsyncMock(
return_value=[
{
"id": "exotic/model-a",
"context_length": 8192,
"pricing": {"prompt": "0.000001", "completion": "0.000002"},
},
{
"id": "exotic/model-b",
"context_length": 8192,
"pricing": {"prompt": "0.000003", "completion": "0.000004"},
},
]
)
with _patch_models_endpoint(payload):
with patch("routstr.payment.models.async_fetch_openrouter_models", or_feed):
models = await GenericUpstreamProvider(base_url="http://x").fetch_models()
assert {m.id for m in models} == {"exotic/model-a", "exotic/model-b"}
assert or_feed.await_count == 1
# ---------------------------------------------------------------------------
# fail closed — no source resolves → disabled + warned, never fabricated
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_unresolvable_model_fails_closed(
caplog: pytest.LogCaptureFixture,
) -> None:
"""When native, litellm and OpenRouter all miss, the model is imported
disabled with a warning naming it — and no price is invented (the old
``$0.001`` placeholder is gone)."""
payload = {
"data": [
{
"id": "nobody-has-priced-this-xyz",
"object": "model",
"owned_by": "mystery",
},
]
}
# routstr loggers set propagate=False, so caplog's root handler misses
# them; attach its handler to the provider logger directly.
gen_logger = logging.getLogger("routstr.upstream.generic")
gen_logger.addHandler(caplog.handler)
try:
with _patch_models_endpoint(payload):
or_feed = AsyncMock(return_value=[])
with patch(
"routstr.payment.models.async_fetch_openrouter_models", or_feed
):
models = await GenericUpstreamProvider(
base_url="http://x"
).fetch_models()
finally:
gen_logger.removeHandler(caplog.handler)
model = _model_by_id(models, "nobody-has-priced-this-xyz")
assert model.enabled is False
assert model.pricing.prompt == 0.0
assert model.pricing.completion == 0.0
assert any(
"nobody-has-priced-this-xyz" in rec.getMessage()
for rec in caplog.records
if rec.levelno >= logging.WARNING
)

View File

@@ -0,0 +1,396 @@
"""Tests for upstream rate-limit detection, classification, and org-ID redaction.
Covers issue #555: upstream OpenAI-compatible providers return rate-limit
errors that embed a sensitive organization ID. The proxy must classify these
distinctly (``UPSTREAM_RATE_LIMIT``), preserve useful debugging fields, and
never emit a raw ``org-*`` identifier in logs, errors, or returned bodies.
"""
from __future__ import annotations
import json
from typing import Any
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import httpx
import pytest
from routstr.core.redaction import redact_org_ids
from routstr.upstream.base import BaseUpstreamProvider
from routstr.upstream.rate_limit import (
UPSTREAM_RATE_LIMIT,
RateLimitInfo,
classify_rate_limit,
)
# The exact scenario from the issue, with a realistic (fake) org identifier.
RAW_ORG_ID = "org-abc123XYZ456def"
RATE_LIMIT_MESSAGE = (
f"Rate limit reached for gpt-5.5-2026-04-23 (for limit gpt-5.5) in "
f"organization {RAW_ORG_ID} on tokens per min (TPM): Limit 180000000, "
f"Used 180000000, Requested 8929. Please try again in 2ms. Visit "
f"https://platform.openai.com/account/rate-limits to learn more."
)
def _make_request(request_id: str = "req-123") -> Mock:
request = Mock(spec=["method", "state"])
request.method = "POST"
request.state = Mock()
request.state.request_id = request_id
return request
def _make_upstream_response(
*,
body: bytes,
status_code: int = 429,
content_type: str | None = "application/json",
extra_headers: dict[str, str] | None = None,
) -> httpx.Response:
headers: dict[str, str] = {}
if content_type is not None:
headers["content-type"] = content_type
if extra_headers:
headers.update(extra_headers)
return httpx.Response(status_code=status_code, headers=headers, content=body)
@pytest.fixture
def provider() -> BaseUpstreamProvider:
return BaseUpstreamProvider(
base_url="https://privateprovider.xyz", api_key="k", provider_fee=1.0
)
# --------------------------------------------------------------------------- #
# Redaction
# --------------------------------------------------------------------------- #
def test_redact_org_ids_replaces_identifier() -> None:
assert RAW_ORG_ID not in redact_org_ids(RATE_LIMIT_MESSAGE)
assert "org-[REDACTED]" in redact_org_ids(RATE_LIMIT_MESSAGE)
def test_redact_org_ids_is_idempotent() -> None:
once = redact_org_ids(RATE_LIMIT_MESSAGE)
assert redact_org_ids(once) == once
def test_redact_org_ids_leaves_unrelated_text() -> None:
assert redact_org_ids("organize the org-chart") == "organize the org-chart"
assert redact_org_ids("") == ""
# --------------------------------------------------------------------------- #
# Classification
# --------------------------------------------------------------------------- #
def test_classify_exact_scenario() -> None:
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert isinstance(info, RateLimitInfo)
assert info.code == UPSTREAM_RATE_LIMIT
assert info.model == "gpt-5.5-2026-04-23"
assert info.limit_name == "gpt-5.5"
assert info.metric == "tokens per min (TPM)"
assert info.limit == 180000000
assert info.used == 180000000
assert info.requested == 8929
assert info.retry_after_seconds == pytest.approx(0.002)
# Redaction-safe: no raw org id survives into the structured view.
assert RAW_ORG_ID not in info.message
assert RAW_ORG_ID not in json.dumps(info.as_details())
def test_classify_by_status_code_without_marker() -> None:
info = classify_rate_limit(429, "slow down")
assert info is not None
assert info.code == UPSTREAM_RATE_LIMIT
def test_classify_by_message_marker_without_429() -> None:
info = classify_rate_limit(400, "rate_limit_exceeded for this key")
assert info is not None
def test_retry_after_header_takes_precedence() -> None:
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE, {"Retry-After": "12"})
assert info is not None
assert info.retry_after_seconds == pytest.approx(12.0)
def test_non_rate_limit_error_is_not_classified() -> None:
assert classify_rate_limit(400, "invalid request: missing field 'model'") is None
assert classify_rate_limit(500, "internal server error") is None
# --------------------------------------------------------------------------- #
# forward_upstream_error_response integration
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_json_rate_limit_body_is_redacted_and_forwarded(
provider: BaseUpstreamProvider,
) -> None:
body = json.dumps(
{"error": {"message": RATE_LIMIT_MESSAGE, "type": "rate_limit_exceeded"}}
).encode()
upstream = _make_upstream_response(body=body, status_code=429)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 429
raw = bytes(response.body).decode()
# No raw organization id may survive in the forwarded body.
assert RAW_ORG_ID not in raw
assert "org-[REDACTED]" in raw
# Body remains valid JSON; the original type is preserved while a stable
# rate-limit code is injected so callers can switch on it.
payload: dict[str, Any] = json.loads(raw)
assert payload["error"]["type"] == "rate_limit_exceeded"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["model"] == "gpt-5.5-2026-04-23"
# A retry hint extracted from the message is surfaced as a header.
assert "retry-after" in {k.lower() for k in response.headers}
@pytest.mark.asyncio
async def test_non_json_rate_limit_envelope_uses_stable_code(
provider: BaseUpstreamProvider,
) -> None:
upstream = _make_upstream_response(
body=RATE_LIMIT_MESSAGE.encode(),
status_code=429,
content_type="text/plain",
)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["model"] == "gpt-5.5-2026-04-23"
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
@pytest.mark.asyncio
async def test_non_rate_limit_json_error_unchanged(
provider: BaseUpstreamProvider,
) -> None:
body = json.dumps(
{"error": {"message": "missing field 'model'", "type": "invalid_request"}}
).encode()
upstream = _make_upstream_response(body=body, status_code=400)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 400
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "invalid_request"
assert "retry-after" not in {k.lower() for k in response.headers}
# --------------------------------------------------------------------------- #
# UpstreamError -> proxy response (preserves code/details/status)
# --------------------------------------------------------------------------- #
def test_create_upstream_error_response_preserves_structure() -> None:
from routstr.core.exceptions import UpstreamError
from routstr.payment.helpers import create_upstream_error_response
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert info is not None
err = UpstreamError(
f"Upstream error via litellm: {RATE_LIMIT_MESSAGE}",
status_code=429,
code=info.code,
details=info.as_details(),
)
response = create_upstream_error_response(err, _make_request())
# Original upstream status is preserved (not flattened to 502).
assert response.status_code == 429
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["requested"] == 8929
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
def test_generic_upstream_error_still_defaults_to_502() -> None:
from routstr.core.exceptions import UpstreamError
from routstr.payment.helpers import create_upstream_error_response
err = UpstreamError("connection refused") # status_code defaults to 502
response = create_upstream_error_response(err, _make_request())
assert response.status_code == 502
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == 502
assert "details" not in payload["error"]
# --------------------------------------------------------------------------- #
# Structured log-extra redaction
# --------------------------------------------------------------------------- #
def test_security_filter_redacts_org_id_in_extra() -> None:
import logging
from routstr.core.logging import SecurityFilter
record = logging.LogRecord(
name="test",
level=logging.ERROR,
pathname=__file__,
lineno=1,
msg="upstream failed",
args=(),
exc_info=None,
)
# Simulate an ``extra={"body_preview": ...}`` field carrying an org id.
setattr(record, "body_preview", RATE_LIMIT_MESSAGE)
assert SecurityFilter().filter(record) is True
redacted: str = getattr(record, "body_preview")
assert RAW_ORG_ID not in redacted
assert "org-[REDACTED]" in redacted
def test_security_filter_redacts_org_id_in_nested_extra() -> None:
import logging
from routstr.core.logging import SecurityFilter
record = logging.LogRecord(
name="test",
level=logging.ERROR,
pathname=__file__,
lineno=1,
msg="upstream failed",
args=(),
exc_info=None,
)
# Nested structures: dict containing a list containing the org id.
setattr(record, "body", {"error": {"messages": [RATE_LIMIT_MESSAGE]}})
assert SecurityFilter().filter(record) is True
serialized = json.dumps(getattr(record, "body"))
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# --------------------------------------------------------------------------- #
# 5xx-wrapped rate limit through forward_upstream_error_response
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_5xx_wrapped_rate_limit_is_classified(
provider: BaseUpstreamProvider,
) -> None:
# Some providers wrap a rate-limit in a 5xx envelope; classification must
# key off the message marker, not only the 429 status.
body = json.dumps({"error": {"message": RATE_LIMIT_MESSAGE}}).encode()
upstream = _make_upstream_response(body=body, status_code=500)
response = await provider.forward_upstream_error_response(
_make_request(), "v1/chat/completions", upstream
)
assert response.status_code == 500
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# --------------------------------------------------------------------------- #
# Real proxy loop: structured error surfaced + reservation reverted once
# --------------------------------------------------------------------------- #
@pytest.mark.asyncio
async def test_proxy_loop_surfaces_rate_limit_and_reverts_once() -> None:
from routstr import proxy as proxy_module
from routstr.core.db import ApiKey
from routstr.core.exceptions import UpstreamError
info = classify_rate_limit(429, RATE_LIMIT_MESSAGE)
assert info is not None
key = ApiKey(hashed_key="rlkey", balance=10_000)
request = MagicMock()
request.method = "POST"
request.headers = {"authorization": "Bearer sk-rlkey"}
request.body = AsyncMock(return_value=b'{"model": "test-model"}')
request.state = MagicMock()
request.state.request_id = "req-rl"
upstream = MagicMock()
upstream.provider_type = "test"
upstream.prepare_headers = MagicMock(side_effect=lambda h: h)
upstream.forward_request = AsyncMock(
side_effect=UpstreamError(
f"Upstream error via litellm: {RATE_LIMIT_MESSAGE}",
status_code=429,
code=info.code,
details=info.as_details(),
)
)
session = MagicMock()
revert_mock = AsyncMock(return_value=True)
with (
patch.object(proxy_module, "get_model_instance", return_value=MagicMock()),
patch.object(proxy_module, "get_provider_for_model", return_value=[upstream]),
patch.object(
proxy_module, "get_max_cost_for_model", AsyncMock(return_value=1_000)
),
patch.object(
proxy_module,
"calculate_discounted_max_cost",
AsyncMock(return_value=1_000),
),
patch.object(proxy_module, "check_token_balance", MagicMock()),
patch.object(
proxy_module, "get_bearer_token_key", AsyncMock(return_value=key)
),
patch.object(proxy_module, "pay_for_request", AsyncMock(return_value=1_000)),
patch.object(proxy_module, "revert_pay_for_request", revert_mock),
):
response = await proxy_module.proxy(
request, "v1/chat/completions", session=session
)
# Original 429 status and the stable code/details survive to the client.
assert response.status_code == 429
payload: dict[str, Any] = json.loads(bytes(response.body))
assert payload["error"]["type"] == "upstream_error"
assert payload["error"]["code"] == UPSTREAM_RATE_LIMIT
assert payload["error"]["details"]["requested"] == 8929
serialized = json.dumps(payload)
assert RAW_ORG_ID not in serialized
assert "org-[REDACTED]" in serialized
# Single upstream failed -> reservation reverted exactly once (no double-charge).
revert_mock.assert_awaited_once_with(key, session, 1_000)

View File

@@ -0,0 +1,140 @@
"""Tests for vendor-agnostic usage normalization.
Specifies the seam that keeps vendor usage dialects out of generic billing
code: a canonical ``NormalizedUsage`` shape produced by
``routstr.payment.usage.normalize_usage`` (union parser for the known,
non-colliding dialects). ``calculate_cost`` normalizes the response's usage
object with this parser and needs no vendor knowledge of its own.
"""
import os
os.environ.setdefault("UPSTREAM_BASE_URL", "http://test")
os.environ.setdefault("UPSTREAM_API_KEY", "test")
os.environ.setdefault("LIGHTNING_ADDRESS", "test@stm.to")
import pytest
from routstr.payment.usage import NormalizedUsage, normalize_usage
# ============================================================================
# The union parser: one canonical shape for all known dialects
# ============================================================================
@pytest.mark.parametrize(
"usage,expected",
[
# OpenAI: cached_tokens included in prompt_tokens → subtracted
(
{
"prompt_tokens": 2000,
"completion_tokens": 100,
"prompt_tokens_details": {"cached_tokens": 800},
},
NormalizedUsage(
input_tokens=1200,
output_tokens=100,
cache_read_tokens=800,
cache_write_tokens=0,
),
),
# DeepSeek: hit/miss fields, prompt_tokens = hit + miss → hit subtracted
(
{
"prompt_tokens": 10000,
"completion_tokens": 500,
"prompt_cache_hit_tokens": 9000,
"prompt_cache_miss_tokens": 1000,
},
NormalizedUsage(
input_tokens=1000,
output_tokens=500,
cache_read_tokens=9000,
cache_write_tokens=0,
),
),
# Anthropic: cache fields additive, input_tokens NOT reduced
(
{
"input_tokens": 300,
"output_tokens": 100,
"cache_read_input_tokens": 500,
"cache_creation_input_tokens": 2000,
},
NormalizedUsage(
input_tokens=300,
output_tokens=100,
cache_read_tokens=500,
cache_write_tokens=2000,
),
),
# Plain OpenAI without caching
(
{"prompt_tokens": 100, "completion_tokens": 50},
NormalizedUsage(input_tokens=100, output_tokens=50),
),
# OpenRouter: cache writes nested as prompt_tokens_details.cache_write_tokens,
# both reads and writes included in prompt_tokens → both subtracted
(
{
"prompt_tokens": 10000,
"completion_tokens": 60,
"prompt_tokens_details": {
"cached_tokens": 5000,
"cache_write_tokens": 2000,
},
},
NormalizedUsage(
input_tokens=3000,
output_tokens=60,
cache_read_tokens=5000,
cache_write_tokens=2000,
),
),
# litellm-normalized Anthropic: prompt_tokens is the grand total and the
# write field is named cache_creation_tokens; top-level fields mirror it.
# prompt_tokens present → both subtracted (NOT additive like native).
(
{
"prompt_tokens": 10000,
"completion_tokens": 100,
"cache_read_input_tokens": 5000,
"cache_creation_input_tokens": 2000,
"prompt_tokens_details": {
"cached_tokens": 5000,
"cache_creation_tokens": 2000,
},
},
NormalizedUsage(
input_tokens=3000,
output_tokens=100,
cache_read_tokens=5000,
cache_write_tokens=2000,
),
),
],
)
def test_normalize_usage_dialects(usage: dict, expected: NormalizedUsage) -> None:
"""Each known vendor dialect maps onto the same canonical shape."""
assert normalize_usage(usage) == expected
def test_normalize_usage_absent_usage() -> None:
"""Missing/invalid usage yields None so callers can bill at max cost."""
assert normalize_usage(None) is None
assert normalize_usage("not a dict") is None # type: ignore[arg-type]
def test_normalize_usage_never_negative() -> None:
"""Buggy upstreams reporting more cached than prompt tokens clamp to 0."""
result = normalize_usage(
{
"prompt_tokens": 100,
"completion_tokens": 50,
"prompt_cache_hit_tokens": 150,
}
)
assert result is not None
assert result.input_tokens == 0
assert result.cache_read_tokens == 150

File diff suppressed because it is too large Load Diff

View File

@@ -1,36 +1,44 @@
This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/cli/create-next-app).
# Routstr node admin UI
## Getting Started
A [Next.js](https://nextjs.org) app (App Router, **static export**) that provides the
admin dashboard for a `routstr-core` node: login, settings, providers, balances,
transactions, usage, and logs.
First, run the development server:
There is no separate web server in production. `next build` produces a fully static
export (`next.config.ts` sets `output: 'export'`), and the FastAPI backend serves it
directly from `../ui_out/` (see `routstr/core/main.py`). So the UI and the API are
served from the **same origin** in production.
```bash
npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev
```
## Developing the UI (hot reload)
Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
The everyday loop runs two processes side by side — you do **not** rebuild the static
export while developing:
You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
1. Start the backend on `:8000` — from the repo root: `make docker-up` (or
`uvicorn routstr.core.main:app --reload`).
2. Start the Next.js dev server on `:3000` — from the repo root: `make ui-dev`
(or `cd ui && pnpm dev`). Edits hot-reload instantly.
This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load [Geist](https://vercel.com/font), a new font family for Vercel.
Open http://localhost:3000. With no `NEXT_PUBLIC_API_URL` set, the UI falls back to
`http://127.0.0.1:8000` in development (see `lib/api/services/configuration.ts`), so it
talks to the local backend out of the box.
## Learn More
Because dev is cross-origin (`:3000``:8000`), it relies on the backend's CORS
allowing the UI origin. The default `cors_origins` is `["*"]`; if you tighten CORS,
keep `http://localhost:3000` allowed for development.
To learn more about Next.js, take a look at the following resources:
## Building the integrated/static UI (what production serves)
- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
To produce the bundle that FastAPI serves from `../ui_out/`:
You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!
- `make ui-build` — builds with local Node/pnpm (`scripts/build-ui.sh`), then moves
`ui/out/*` to `../ui_out/`.
- `make ui-build-docker` — same, but inside Docker (no local Node needed).
## Deploy on Vercel
`NEXT_PUBLIC_*` variables are read from the repo-root `.env` at build time and baked in.
For a same-origin deployment leave `NEXT_PUBLIC_API_URL` empty (relative paths); the UI
uses `window.location.origin` at runtime. After building, start the backend and open
http://localhost:8000 — the dashboard is served at `/` and `/admin`.
The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.
If `../ui_out/` does not exist, the backend logs a warning at startup and serves the API
only (hitting a UI route returns a small JSON fallback instead of the dashboard).

View File

@@ -297,16 +297,13 @@ export default function ProvidersPage() {
};
const toggleProviderExpansion = (providerId: number) => {
const newExpanded = new Set(expandedProviders);
if (newExpanded.has(providerId)) {
newExpanded.delete(providerId);
} else {
newExpanded.add(providerId);
}
setExpandedProviders(newExpanded);
if (!newExpanded.has(providerId)) {
if (expandedProviders.has(providerId)) {
setExpandedProviders(new Set());
setViewingModels(null);
} else {
// Accordion: only one provider open at a time so switching to another
// provider's models auto-collapses the previously expanded one.
setExpandedProviders(new Set([providerId]));
setViewingModels(providerId);
}
};

View File

@@ -68,6 +68,8 @@ const FormSchema = z.object({
upstream_provider_id: z.string().default(''),
input_cost: z.coerce.number().min(0).default(0),
output_cost: z.coerce.number().min(0).default(0),
cache_read_cost: z.coerce.number().min(0).default(0),
cache_write_cost: z.coerce.number().min(0).default(0),
request_cost: z.coerce.number().min(0).default(0),
image_cost: z.coerce.number().min(0).default(0),
web_search_cost: z.coerce.number().min(0).default(0),
@@ -125,6 +127,8 @@ export function AddProviderModelDialog({
upstream_provider_id: '',
input_cost: 0,
output_cost: 0,
cache_read_cost: 0,
cache_write_cost: 0,
request_cost: 0,
image_cost: 0,
web_search_cost: 0,
@@ -190,6 +194,8 @@ export function AddProviderModelDialog({
: initialData.upstream_provider_id?.toString() || '',
input_cost: pricing?.prompt ?? 0,
output_cost: pricing?.completion ?? 0,
cache_read_cost: pricing?.input_cache_read ?? 0,
cache_write_cost: pricing?.input_cache_write ?? 0,
request_cost: pricing?.request ?? 0,
image_cost: pricing?.image ?? 0,
web_search_cost: pricing?.web_search ?? 0,
@@ -231,6 +237,8 @@ export function AddProviderModelDialog({
upstream_provider_id: '',
input_cost: 0,
output_cost: 0,
cache_read_cost: 0,
cache_write_cost: 0,
request_cost: 0,
image_cost: 0,
web_search_cost: 0,
@@ -294,6 +302,8 @@ export function AddProviderModelDialog({
);
form.setValue('input_cost', pricing?.prompt ?? 0);
form.setValue('output_cost', pricing?.completion ?? 0);
form.setValue('cache_read_cost', pricing?.input_cache_read ?? 0);
form.setValue('cache_write_cost', pricing?.input_cache_write ?? 0);
form.setValue('request_cost', pricing?.request ?? 0);
form.setValue('image_cost', pricing?.image ?? 0);
form.setValue('web_search_cost', pricing?.web_search ?? 0);
@@ -365,6 +375,8 @@ export function AddProviderModelDialog({
pricing: {
prompt: data.input_cost,
completion: data.output_cost,
input_cache_read: data.cache_read_cost,
input_cache_write: data.cache_write_cost,
request: data.request_cost,
image: data.image_cost,
web_search: data.web_search_cost,
@@ -810,6 +822,38 @@ export function AddProviderModelDialog({
</FormItem>
)}
/>
<FormField
control={form.control}
name='cache_read_cost'
render={({ field }) => (
<FormItem>
<FormLabel>Cache Read Cost</FormLabel>
<FormControl>
<Input type='number' step='0.000001' {...field} />
</FormControl>
<FormDescription>
Discounted cached-input read price per 1M tokens.
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name='cache_write_cost'
render={({ field }) => (
<FormItem>
<FormLabel>Cache Write Cost</FormLabel>
<FormControl>
<Input type='number' step='0.000001' {...field} />
</FormControl>
<FormDescription>
Cached-input creation price per 1M tokens.
</FormDescription>
<FormMessage />
</FormItem>
)}
/>
<FormField
control={form.control}
name='request_cost'

View File

@@ -118,6 +118,26 @@ export function ProviderFormFields({
/>
)}
<div className='grid gap-2'>
<Label htmlFor={`${idPrefix}slug`}>
Slug {mode === 'create' ? '(optional, auto-generated)' : ''}
</Label>
<Input
id={`${idPrefix}slug`}
value={formData.slug || ''}
onChange={(e) =>
setFormData((prev) => ({
...prev,
slug: e.target.value || undefined,
}))
}
placeholder='e.g. openai-prod'
/>
<p className='text-muted-foreground text-xs'>
Stable external key used to update this provider via the admin API.
</p>
</div>
<div className='grid gap-2'>
<Label htmlFor={`${idPrefix}base_url`}>Base URL</Label>
<Input

View File

@@ -14,6 +14,7 @@ export const ProviderTypeSchema = z.object({
export const UpstreamProviderSchema = z.object({
id: z.number(),
slug: z.string().nullable().optional(),
provider_type: z.string(),
base_url: z.string(),
api_key: z.string().optional(),
@@ -31,6 +32,7 @@ export const CreateUpstreamProviderSchema = z.object({
enabled: z.boolean().default(true),
provider_fee: z.number().optional(),
provider_settings: z.record(z.string(), z.any()).nullable().optional(),
slug: z.string().optional(),
});
export const UpdateUpstreamProviderSchema = z.object({
@@ -41,6 +43,7 @@ export const UpdateUpstreamProviderSchema = z.object({
enabled: z.boolean().optional(),
provider_fee: z.number().optional(),
provider_settings: z.record(z.string(), z.any()).nullable().optional(),
slug: z.string().optional(),
});
export const AdminModelPricingSchema = z.object({
@@ -50,6 +53,8 @@ export const AdminModelPricingSchema = z.object({
image: z.number().optional(),
web_search: z.number().optional(),
internal_reasoning: z.number().optional(),
input_cache_read: z.number().optional(),
input_cache_write: z.number().optional(),
});
export const AdminModelArchitectureSchema = z.object({
@@ -146,7 +151,7 @@ export class AdminService {
if (!pricing) return pricing;
const result = { ...pricing };
// Only prompt and completion are per-token and need scaling to per-1M
// Token-priced fields are stored per-token by the API and shown per-1M in the UI.
const convertField = (field: string) => {
const val = result[field];
if (val !== undefined && val !== null) {
@@ -161,6 +166,8 @@ export class AdminService {
convertField('prompt');
convertField('completion');
convertField('input_cache_read');
convertField('input_cache_write');
// Other fields (request, image, etc.) are already flat fees (per item)
// so we do NOT scale them.
@@ -174,7 +181,7 @@ export class AdminService {
if (!pricing) return pricing;
const result = { ...pricing };
// Only prompt and completion are per-1M in UI and need scaling down to per-token
// Token-priced fields are per-1M in the UI and need scaling down to per-token.
const convertField = (field: string) => {
const val = result[field];
if (val !== undefined && val !== null) {
@@ -187,6 +194,8 @@ export class AdminService {
convertField('prompt');
convertField('completion');
convertField('input_cache_read');
convertField('input_cache_write');
// Other fields stay as flat fees

339
uv.lock generated
View File

@@ -17,7 +17,7 @@ wheels = [
[[package]]
name = "aiohttp"
version = "3.12.15"
version = "3.14.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "aiohappyeyeballs" },
@@ -26,61 +26,111 @@ dependencies = [
{ name = "frozenlist" },
{ name = "multidict" },
{ name = "propcache" },
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
{ name = "yarl" },
]
sdist = { url = "https://files.pythonhosted.org/packages/9b/e7/d92a237d8802ca88483906c388f7c201bbe96cd80a165ffd0ac2f6a8d59f/aiohttp-3.12.15.tar.gz", hash = "sha256:4fc61385e9c98d72fcdf47e6dd81833f47b2f77c114c29cd64a361be57a763a2", size = 7823716, upload-time = "2025-07-29T05:52:32.215Z" }
sdist = { url = "https://files.pythonhosted.org/packages/82/78/8ea7308cac6934de8c74a14f3d5f65d1c89287426688be79538d0e5c013d/aiohttp-3.14.1.tar.gz", hash = "sha256:307f2cff90a764d329e77040603fa032db89c5c24fdad50c4c15334cba744035", size = 7955794, upload-time = "2026-06-07T21:09:35.529Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/20/19/9e86722ec8e835959bd97ce8c1efa78cf361fa4531fca372551abcc9cdd6/aiohttp-3.12.15-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:d3ce17ce0220383a0f9ea07175eeaa6aa13ae5a41f30bc61d84df17f0e9b1117", size = 711246, upload-time = "2025-07-29T05:50:15.937Z" },
{ url = "https://files.pythonhosted.org/packages/71/f9/0a31fcb1a7d4629ac9d8f01f1cb9242e2f9943f47f5d03215af91c3c1a26/aiohttp-3.12.15-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:010cc9bbd06db80fe234d9003f67e97a10fe003bfbedb40da7d71c1008eda0fe", size = 483515, upload-time = "2025-07-29T05:50:17.442Z" },
{ url = "https://files.pythonhosted.org/packages/62/6c/94846f576f1d11df0c2e41d3001000527c0fdf63fce7e69b3927a731325d/aiohttp-3.12.15-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:3f9d7c55b41ed687b9d7165b17672340187f87a773c98236c987f08c858145a9", size = 471776, upload-time = "2025-07-29T05:50:19.568Z" },
{ url = "https://files.pythonhosted.org/packages/f8/6c/f766d0aaafcee0447fad0328da780d344489c042e25cd58fde566bf40aed/aiohttp-3.12.15-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:bc4fbc61bb3548d3b482f9ac7ddd0f18c67e4225aaa4e8552b9f1ac7e6bda9e5", size = 1741977, upload-time = "2025-07-29T05:50:21.665Z" },
{ url = "https://files.pythonhosted.org/packages/17/e5/fb779a05ba6ff44d7bc1e9d24c644e876bfff5abe5454f7b854cace1b9cc/aiohttp-3.12.15-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:7fbc8a7c410bb3ad5d595bb7118147dfbb6449d862cc1125cf8867cb337e8728", size = 1690645, upload-time = "2025-07-29T05:50:23.333Z" },
{ url = "https://files.pythonhosted.org/packages/37/4e/a22e799c2035f5d6a4ad2cf8e7c1d1bd0923192871dd6e367dafb158b14c/aiohttp-3.12.15-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:74dad41b3458dbb0511e760fb355bb0b6689e0630de8a22b1b62a98777136e16", size = 1789437, upload-time = "2025-07-29T05:50:25.007Z" },
{ url = "https://files.pythonhosted.org/packages/28/e5/55a33b991f6433569babb56018b2fb8fb9146424f8b3a0c8ecca80556762/aiohttp-3.12.15-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3b6f0af863cf17e6222b1735a756d664159e58855da99cfe965134a3ff63b0b0", size = 1828482, upload-time = "2025-07-29T05:50:26.693Z" },
{ url = "https://files.pythonhosted.org/packages/c6/82/1ddf0ea4f2f3afe79dffed5e8a246737cff6cbe781887a6a170299e33204/aiohttp-3.12.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b5b7fe4972d48a4da367043b8e023fb70a04d1490aa7d68800e465d1b97e493b", size = 1730944, upload-time = "2025-07-29T05:50:28.382Z" },
{ url = "https://files.pythonhosted.org/packages/1b/96/784c785674117b4cb3877522a177ba1b5e4db9ce0fd519430b5de76eec90/aiohttp-3.12.15-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6443cca89553b7a5485331bc9bedb2342b08d073fa10b8c7d1c60579c4a7b9bd", size = 1668020, upload-time = "2025-07-29T05:50:30.032Z" },
{ url = "https://files.pythonhosted.org/packages/12/8a/8b75f203ea7e5c21c0920d84dd24a5c0e971fe1e9b9ebbf29ae7e8e39790/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:6c5f40ec615e5264f44b4282ee27628cea221fcad52f27405b80abb346d9f3f8", size = 1716292, upload-time = "2025-07-29T05:50:31.983Z" },
{ url = "https://files.pythonhosted.org/packages/47/0b/a1451543475bb6b86a5cfc27861e52b14085ae232896a2654ff1231c0992/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:2abbb216a1d3a2fe86dbd2edce20cdc5e9ad0be6378455b05ec7f77361b3ab50", size = 1711451, upload-time = "2025-07-29T05:50:33.989Z" },
{ url = "https://files.pythonhosted.org/packages/55/fd/793a23a197cc2f0d29188805cfc93aa613407f07e5f9da5cd1366afd9d7c/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:db71ce547012a5420a39c1b744d485cfb823564d01d5d20805977f5ea1345676", size = 1691634, upload-time = "2025-07-29T05:50:35.846Z" },
{ url = "https://files.pythonhosted.org/packages/ca/bf/23a335a6670b5f5dfc6d268328e55a22651b440fca341a64fccf1eada0c6/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:ced339d7c9b5030abad5854aa5413a77565e5b6e6248ff927d3e174baf3badf7", size = 1785238, upload-time = "2025-07-29T05:50:37.597Z" },
{ url = "https://files.pythonhosted.org/packages/57/4f/ed60a591839a9d85d40694aba5cef86dde9ee51ce6cca0bb30d6eb1581e7/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:7c7dd29c7b5bda137464dc9bfc738d7ceea46ff70309859ffde8c022e9b08ba7", size = 1805701, upload-time = "2025-07-29T05:50:39.591Z" },
{ url = "https://files.pythonhosted.org/packages/85/e0/444747a9455c5de188c0f4a0173ee701e2e325d4b2550e9af84abb20cdba/aiohttp-3.12.15-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:421da6fd326460517873274875c6c5a18ff225b40da2616083c5a34a7570b685", size = 1718758, upload-time = "2025-07-29T05:50:41.292Z" },
{ url = "https://files.pythonhosted.org/packages/36/ab/1006278d1ffd13a698e5dd4bfa01e5878f6bddefc296c8b62649753ff249/aiohttp-3.12.15-cp311-cp311-win32.whl", hash = "sha256:4420cf9d179ec8dfe4be10e7d0fe47d6d606485512ea2265b0d8c5113372771b", size = 428868, upload-time = "2025-07-29T05:50:43.063Z" },
{ url = "https://files.pythonhosted.org/packages/10/97/ad2b18700708452400278039272032170246a1bf8ec5d832772372c71f1a/aiohttp-3.12.15-cp311-cp311-win_amd64.whl", hash = "sha256:edd533a07da85baa4b423ee8839e3e91681c7bfa19b04260a469ee94b778bf6d", size = 453273, upload-time = "2025-07-29T05:50:44.613Z" },
{ url = "https://files.pythonhosted.org/packages/63/97/77cb2450d9b35f517d6cf506256bf4f5bda3f93a66b4ad64ba7fc917899c/aiohttp-3.12.15-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:802d3868f5776e28f7bf69d349c26fc0efadb81676d0afa88ed00d98a26340b7", size = 702333, upload-time = "2025-07-29T05:50:46.507Z" },
{ url = "https://files.pythonhosted.org/packages/83/6d/0544e6b08b748682c30b9f65640d006e51f90763b41d7c546693bc22900d/aiohttp-3.12.15-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:f2800614cd560287be05e33a679638e586a2d7401f4ddf99e304d98878c29444", size = 476948, upload-time = "2025-07-29T05:50:48.067Z" },
{ url = "https://files.pythonhosted.org/packages/3a/1d/c8c40e611e5094330284b1aea8a4b02ca0858f8458614fa35754cab42b9c/aiohttp-3.12.15-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:8466151554b593909d30a0a125d638b4e5f3836e5aecde85b66b80ded1cb5b0d", size = 469787, upload-time = "2025-07-29T05:50:49.669Z" },
{ url = "https://files.pythonhosted.org/packages/38/7d/b76438e70319796bfff717f325d97ce2e9310f752a267bfdf5192ac6082b/aiohttp-3.12.15-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:2e5a495cb1be69dae4b08f35a6c4579c539e9b5706f606632102c0f855bcba7c", size = 1716590, upload-time = "2025-07-29T05:50:51.368Z" },
{ url = "https://files.pythonhosted.org/packages/79/b1/60370d70cdf8b269ee1444b390cbd72ce514f0d1cd1a715821c784d272c9/aiohttp-3.12.15-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:6404dfc8cdde35c69aaa489bb3542fb86ef215fc70277c892be8af540e5e21c0", size = 1699241, upload-time = "2025-07-29T05:50:53.628Z" },
{ url = "https://files.pythonhosted.org/packages/a3/2b/4968a7b8792437ebc12186db31523f541943e99bda8f30335c482bea6879/aiohttp-3.12.15-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3ead1c00f8521a5c9070fcb88f02967b1d8a0544e6d85c253f6968b785e1a2ab", size = 1754335, upload-time = "2025-07-29T05:50:55.394Z" },
{ url = "https://files.pythonhosted.org/packages/fb/c1/49524ed553f9a0bec1a11fac09e790f49ff669bcd14164f9fab608831c4d/aiohttp-3.12.15-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6990ef617f14450bc6b34941dba4f12d5613cbf4e33805932f853fbd1cf18bfb", size = 1800491, upload-time = "2025-07-29T05:50:57.202Z" },
{ url = "https://files.pythonhosted.org/packages/de/5e/3bf5acea47a96a28c121b167f5ef659cf71208b19e52a88cdfa5c37f1fcc/aiohttp-3.12.15-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd736ed420f4db2b8148b52b46b88ed038d0354255f9a73196b7bbce3ea97545", size = 1719929, upload-time = "2025-07-29T05:50:59.192Z" },
{ url = "https://files.pythonhosted.org/packages/39/94/8ae30b806835bcd1cba799ba35347dee6961a11bd507db634516210e91d8/aiohttp-3.12.15-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:3c5092ce14361a73086b90c6efb3948ffa5be2f5b6fbcf52e8d8c8b8848bb97c", size = 1635733, upload-time = "2025-07-29T05:51:01.394Z" },
{ url = "https://files.pythonhosted.org/packages/7a/46/06cdef71dd03acd9da7f51ab3a9107318aee12ad38d273f654e4f981583a/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:aaa2234bb60c4dbf82893e934d8ee8dea30446f0647e024074237a56a08c01bd", size = 1696790, upload-time = "2025-07-29T05:51:03.657Z" },
{ url = "https://files.pythonhosted.org/packages/02/90/6b4cfaaf92ed98d0ec4d173e78b99b4b1a7551250be8937d9d67ecb356b4/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:6d86a2fbdd14192e2f234a92d3b494dd4457e683ba07e5905a0b3ee25389ac9f", size = 1718245, upload-time = "2025-07-29T05:51:05.911Z" },
{ url = "https://files.pythonhosted.org/packages/2e/e6/2593751670fa06f080a846f37f112cbe6f873ba510d070136a6ed46117c6/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:a041e7e2612041a6ddf1c6a33b883be6a421247c7afd47e885969ee4cc58bd8d", size = 1658899, upload-time = "2025-07-29T05:51:07.753Z" },
{ url = "https://files.pythonhosted.org/packages/8f/28/c15bacbdb8b8eb5bf39b10680d129ea7410b859e379b03190f02fa104ffd/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:5015082477abeafad7203757ae44299a610e89ee82a1503e3d4184e6bafdd519", size = 1738459, upload-time = "2025-07-29T05:51:09.56Z" },
{ url = "https://files.pythonhosted.org/packages/00/de/c269cbc4faa01fb10f143b1670633a8ddd5b2e1ffd0548f7aa49cb5c70e2/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:56822ff5ddfd1b745534e658faba944012346184fbfe732e0d6134b744516eea", size = 1766434, upload-time = "2025-07-29T05:51:11.423Z" },
{ url = "https://files.pythonhosted.org/packages/52/b0/4ff3abd81aa7d929b27d2e1403722a65fc87b763e3a97b3a2a494bfc63bc/aiohttp-3.12.15-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:b2acbbfff69019d9014508c4ba0401822e8bae5a5fdc3b6814285b71231b60f3", size = 1726045, upload-time = "2025-07-29T05:51:13.689Z" },
{ url = "https://files.pythonhosted.org/packages/71/16/949225a6a2dd6efcbd855fbd90cf476052e648fb011aa538e3b15b89a57a/aiohttp-3.12.15-cp312-cp312-win32.whl", hash = "sha256:d849b0901b50f2185874b9a232f38e26b9b3d4810095a7572eacea939132d4e1", size = 423591, upload-time = "2025-07-29T05:51:15.452Z" },
{ url = "https://files.pythonhosted.org/packages/2b/d8/fa65d2a349fe938b76d309db1a56a75c4fb8cc7b17a398b698488a939903/aiohttp-3.12.15-cp312-cp312-win_amd64.whl", hash = "sha256:b390ef5f62bb508a9d67cb3bba9b8356e23b3996da7062f1a57ce1a79d2b3d34", size = 450266, upload-time = "2025-07-29T05:51:17.239Z" },
{ url = "https://files.pythonhosted.org/packages/f2/33/918091abcf102e39d15aba2476ad9e7bd35ddb190dcdd43a854000d3da0d/aiohttp-3.12.15-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:9f922ffd05034d439dde1c77a20461cf4a1b0831e6caa26151fe7aa8aaebc315", size = 696741, upload-time = "2025-07-29T05:51:19.021Z" },
{ url = "https://files.pythonhosted.org/packages/b5/2a/7495a81e39a998e400f3ecdd44a62107254803d1681d9189be5c2e4530cd/aiohttp-3.12.15-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:2ee8a8ac39ce45f3e55663891d4b1d15598c157b4d494a4613e704c8b43112cd", size = 474407, upload-time = "2025-07-29T05:51:21.165Z" },
{ url = "https://files.pythonhosted.org/packages/49/fc/a9576ab4be2dcbd0f73ee8675d16c707cfc12d5ee80ccf4015ba543480c9/aiohttp-3.12.15-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:3eae49032c29d356b94eee45a3f39fdf4b0814b397638c2f718e96cfadf4c4e4", size = 466703, upload-time = "2025-07-29T05:51:22.948Z" },
{ url = "https://files.pythonhosted.org/packages/09/2f/d4bcc8448cf536b2b54eed48f19682031ad182faa3a3fee54ebe5b156387/aiohttp-3.12.15-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b97752ff12cc12f46a9b20327104448042fce5c33a624f88c18f66f9368091c7", size = 1705532, upload-time = "2025-07-29T05:51:25.211Z" },
{ url = "https://files.pythonhosted.org/packages/f1/f3/59406396083f8b489261e3c011aa8aee9df360a96ac8fa5c2e7e1b8f0466/aiohttp-3.12.15-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:894261472691d6fe76ebb7fcf2e5870a2ac284c7406ddc95823c8598a1390f0d", size = 1686794, upload-time = "2025-07-29T05:51:27.145Z" },
{ url = "https://files.pythonhosted.org/packages/dc/71/164d194993a8d114ee5656c3b7ae9c12ceee7040d076bf7b32fb98a8c5c6/aiohttp-3.12.15-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5fa5d9eb82ce98959fc1031c28198b431b4d9396894f385cb63f1e2f3f20ca6b", size = 1738865, upload-time = "2025-07-29T05:51:29.366Z" },
{ url = "https://files.pythonhosted.org/packages/1c/00/d198461b699188a93ead39cb458554d9f0f69879b95078dce416d3209b54/aiohttp-3.12.15-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:f0fa751efb11a541f57db59c1dd821bec09031e01452b2b6217319b3a1f34f3d", size = 1788238, upload-time = "2025-07-29T05:51:31.285Z" },
{ url = "https://files.pythonhosted.org/packages/85/b8/9e7175e1fa0ac8e56baa83bf3c214823ce250d0028955dfb23f43d5e61fd/aiohttp-3.12.15-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5346b93e62ab51ee2a9d68e8f73c7cf96ffb73568a23e683f931e52450e4148d", size = 1710566, upload-time = "2025-07-29T05:51:33.219Z" },
{ url = "https://files.pythonhosted.org/packages/59/e4/16a8eac9df39b48ae102ec030fa9f726d3570732e46ba0c592aeeb507b93/aiohttp-3.12.15-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:049ec0360f939cd164ecbfd2873eaa432613d5e77d6b04535e3d1fbae5a9e645", size = 1624270, upload-time = "2025-07-29T05:51:35.195Z" },
{ url = "https://files.pythonhosted.org/packages/1f/f8/cd84dee7b6ace0740908fd0af170f9fab50c2a41ccbc3806aabcb1050141/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:b52dcf013b57464b6d1e51b627adfd69a8053e84b7103a7cd49c030f9ca44461", size = 1677294, upload-time = "2025-07-29T05:51:37.215Z" },
{ url = "https://files.pythonhosted.org/packages/ce/42/d0f1f85e50d401eccd12bf85c46ba84f947a84839c8a1c2c5f6e8ab1eb50/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:9b2af240143dd2765e0fb661fd0361a1b469cab235039ea57663cda087250ea9", size = 1708958, upload-time = "2025-07-29T05:51:39.328Z" },
{ url = "https://files.pythonhosted.org/packages/d5/6b/f6fa6c5790fb602538483aa5a1b86fcbad66244997e5230d88f9412ef24c/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:ac77f709a2cde2cc71257ab2d8c74dd157c67a0558a0d2799d5d571b4c63d44d", size = 1651553, upload-time = "2025-07-29T05:51:41.356Z" },
{ url = "https://files.pythonhosted.org/packages/04/36/a6d36ad545fa12e61d11d1932eef273928b0495e6a576eb2af04297fdd3c/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:47f6b962246f0a774fbd3b6b7be25d59b06fdb2f164cf2513097998fc6a29693", size = 1727688, upload-time = "2025-07-29T05:51:43.452Z" },
{ url = "https://files.pythonhosted.org/packages/aa/c8/f195e5e06608a97a4e52c5d41c7927301bf757a8e8bb5bbf8cef6c314961/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:760fb7db442f284996e39cf9915a94492e1896baac44f06ae551974907922b64", size = 1761157, upload-time = "2025-07-29T05:51:45.643Z" },
{ url = "https://files.pythonhosted.org/packages/05/6a/ea199e61b67f25ba688d3ce93f63b49b0a4e3b3d380f03971b4646412fc6/aiohttp-3.12.15-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:ad702e57dc385cae679c39d318def49aef754455f237499d5b99bea4ef582e51", size = 1710050, upload-time = "2025-07-29T05:51:48.203Z" },
{ url = "https://files.pythonhosted.org/packages/b4/2e/ffeb7f6256b33635c29dbed29a22a723ff2dd7401fff42ea60cf2060abfb/aiohttp-3.12.15-cp313-cp313-win32.whl", hash = "sha256:f813c3e9032331024de2eb2e32a88d86afb69291fbc37a3a3ae81cc9917fb3d0", size = 422647, upload-time = "2025-07-29T05:51:50.718Z" },
{ url = "https://files.pythonhosted.org/packages/1b/8e/78ee35774201f38d5e1ba079c9958f7629b1fd079459aea9467441dbfbf5/aiohttp-3.12.15-cp313-cp313-win_amd64.whl", hash = "sha256:1a649001580bdb37c6fdb1bebbd7e3bc688e8ec2b5c6f52edbb664662b17dc84", size = 449067, upload-time = "2025-07-29T05:51:52.549Z" },
{ url = "https://files.pythonhosted.org/packages/26/dd/bf526e6f0a1120dd6f2df2e97bacfe4d358f13d17a0ff5847301a1375a51/aiohttp-3.14.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:aa00140699487bd435fde4342d85c94cb256b7cd3a5b9c3396c67f19922afda2", size = 765225, upload-time = "2026-06-07T21:06:07.957Z" },
{ url = "https://files.pythonhosted.org/packages/8f/e1/a2872aa55495a70f61310d411541c6ee23812d9a884e000c716e1bc3edbf/aiohttp-3.14.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1c1af67559445498b502030c35c59db59966f47041ca9de5b4e707f86bd10b5f", size = 518743, upload-time = "2026-06-07T21:06:09.749Z" },
{ url = "https://files.pythonhosted.org/packages/5b/e7/c60c7b209e509cc787de3cea0550a518538cfc08003e1c1e14c1c63fff71/aiohttp-3.14.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d44ec478e713ee7f29b439f7eb8dc2b9d4079e11ae114d2c2ac3d5daf30516c8", size = 514139, upload-time = "2026-06-07T21:06:11.26Z" },
{ url = "https://files.pythonhosted.org/packages/5b/8d/614ace2f579702c9840ab1e1447fd8509e35b0b904f7196418fa2f57b25d/aiohttp-3.14.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d3b1a184a9a8f548a6b73f1e26b96b052193e4b3175ed7342aaf1151a1f00a04", size = 1784088, upload-time = "2026-06-07T21:06:12.887Z" },
{ url = "https://files.pythonhosted.org/packages/49/e0/726e90f99542bf292f81a96a12cc4847deb86f3ccf62c6f4014a201f4d33/aiohttp-3.14.1-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:5f2504bc0322437c9a1ff6d3333ca56c7477b727c995f036b976ae17b98372c8", size = 1737835, upload-time = "2026-06-07T21:06:14.564Z" },
{ url = "https://files.pythonhosted.org/packages/0b/4b/d176d5c4db9d33dacf0543102ea59503bc1d528af4cfd0b719949ca49389/aiohttp-3.14.1-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:73f05ea02013e02512c3bf42714f1208c57168c779cc6fe23516e4543089d0a6", size = 1842801, upload-time = "2026-06-07T21:06:16.228Z" },
{ url = "https://files.pythonhosted.org/packages/dc/d6/5a99b563690ea0cbed912ae94a2ce33993a5709a651a3a4fe761e7dd973a/aiohttp-3.14.1-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:797457503c2d426bee06eef808d07b31ede30b65e054444e7de64cad0061b7af", size = 1929992, upload-time = "2026-06-07T21:06:17.947Z" },
{ url = "https://files.pythonhosted.org/packages/76/7f/a987b14a3859094b3cea3f4825219c3e5536242564af6e3f9c2f6c994eb2/aiohttp-3.14.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b821a1f7dedf7e37450654e620038ac3b2e81e8fa6ea269337e97101978ec730", size = 1786989, upload-time = "2026-06-07T21:06:19.677Z" },
{ url = "https://files.pythonhosted.org/packages/f1/1a/420e5c85a3e73349372ed22ce0b6af86bfa6ce16a4b20a64a2e94608c781/aiohttp-3.14.1-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4cd96b5ba05d67ed0cf00b5b405c8cd99586d8e3481e8ee0a831057591af7621", size = 1640129, upload-time = "2026-06-07T21:06:22.558Z" },
{ url = "https://files.pythonhosted.org/packages/a7/80/18a592ed3be0a402cc03670bd72ee1f8563ddbe1d8d5542dbf868f274136/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d459b98a932296c6f0e94f87511a0b1b90a8a02c30a50e60a297619cd5a58ee", size = 1756576, upload-time = "2026-06-07T21:06:24.8Z" },
{ url = "https://files.pythonhosted.org/packages/ec/0b/8b3d5713373858ff71a617daf6e3b0e81ad63e79d09a3cf2f6b6b983939c/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:764457a7be60825fb770a644852ff717bcbb5042f189f2bd16df61a81b3f6573", size = 1754668, upload-time = "2026-06-07T21:06:26.528Z" },
{ url = "https://files.pythonhosted.org/packages/9f/49/fd564575cf225821d7ba5a117cb8bc27213d8a7e1811162afb43ae077039/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:f7a16ef45b081454ef844502d87a848876c490c4cb5c650c230f6ec79ed2c1e7", size = 1817019, upload-time = "2026-06-07T21:06:28.297Z" },
{ url = "https://files.pythonhosted.org/packages/ed/1b/e850c9ae6fc91356552ae668bb6c51e93fa29c8aef13398a10b56678557f/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:2fbc3ed048b3475b9f0cbcb9978e9d2d3511acd91ead203af26ed9f0056004cf", size = 1631638, upload-time = "2026-06-07T21:06:30.242Z" },
{ url = "https://files.pythonhosted.org/packages/eb/94/3c337ba72451a89806ace6f75bddc92bafc5b8d53d90115a512858024b63/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:bedb0cd073cc2dc035e30aeb99444389d3cd2113afe4ef9fcd23d439f5bade85", size = 1835660, upload-time = "2026-06-07T21:06:31.943Z" },
{ url = "https://files.pythonhosted.org/packages/2b/9c/9c18cf367a0498212d9ba7daf990b504a5e8ae064cda4b504e2647c89c03/aiohttp-3.14.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:b6feea921016eb3d4e04d65fc4e9ca402d1a3801f562aef94989f54694917af3", size = 1775698, upload-time = "2026-06-07T21:06:33.72Z" },
{ url = "https://files.pythonhosted.org/packages/b5/63/a251a9d2a6cb45065b2ddc0bde2b3dd10108740a9a42f632c66405a761a2/aiohttp-3.14.1-cp311-cp311-win32.whl", hash = "sha256:313701e488100074ce99850404ee36e741abf6330179fec908a1944ecf570126", size = 458386, upload-time = "2026-06-07T21:06:35.279Z" },
{ url = "https://files.pythonhosted.org/packages/17/ca/69274c51dcd6e8947d77b2806cf47a4a15f2c846e2cbeb1882547d3da283/aiohttp-3.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:03ab4530fdcb3a543a122ba4b65ac9919da9fe9f78a03d328a6e38ff962f7aa5", size = 483406, upload-time = "2026-06-07T21:06:36.824Z" },
{ url = "https://files.pythonhosted.org/packages/2c/8a/c25904f77690c3688ec140f87591ef11a0cfe36bf3d5c0f1f38056fb62b3/aiohttp-3.14.1-cp311-cp311-win_arm64.whl", hash = "sha256:486f7d16ed54c39c2cbd7ca71fd8ba2b8bb7860df65bd7b6ed640bab96a38a8b", size = 452987, upload-time = "2026-06-07T21:06:38.371Z" },
{ url = "https://files.pythonhosted.org/packages/1d/21/151624b51cd92553d95424daf4bf19f19ce9be9002d19253e7e7ce67197b/aiohttp-3.14.1-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:d35143e27778b4bb0fb189562d7f275bff79c62ab8e98459717c0ea617ff2480", size = 757402, upload-time = "2026-06-07T21:06:40.311Z" },
{ url = "https://files.pythonhosted.org/packages/c2/82/280619e0bd7bf2454987e19282616e84762255dd9c8468f62382e8c191f1/aiohttp-3.14.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:bcfb80a2cc36fba2534e5e5b5264dc7ae6fcd9bf15256da3e53d2f499e6fa29d", size = 512310, upload-time = "2026-06-07T21:06:42.207Z" },
{ url = "https://files.pythonhosted.org/packages/55/b2/2aac325583aaa1353045f96dffa586d8a34e8322e14a7ba49cffeb103ab4/aiohttp-3.14.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:27fd7c91e51729b4f7e1577865fa6d34c9adccbc39aabe9000285b48af9f0ec2", size = 512448, upload-time = "2026-06-07T21:06:43.813Z" },
{ url = "https://files.pythonhosted.org/packages/8a/72/a60607cb849faa8af8a356c9329ea2eb6f395d49e82cc82ccba1fd8deb8f/aiohttp-3.14.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:64c567bf9eaf664280116a8688f63016e6b32db2505908e2bdaca1b6438142f2", size = 1766854, upload-time = "2026-06-07T21:06:45.391Z" },
{ url = "https://files.pythonhosted.org/packages/b5/d3/d9fe1c9ec7557ab4d0d82bebaa728c6418f0b93295ec2f4ab015f7710cc7/aiohttp-3.14.1-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:f5e6ff2bdbb8f4cd3fbe41f99e25bbcd58e3bf9f13d3dd31a11e7917251cc77a", size = 1740884, upload-time = "2026-06-07T21:06:47.413Z" },
{ url = "https://files.pythonhosted.org/packages/c1/dc/f2cecfaf9337ba3e63f181500814ff502aa3d00d9c7ec93a9d23d10a27b2/aiohttp-3.14.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2f73e01dc37122325caf079982621262f96d74823c179038a82fddfc50359264", size = 1810034, upload-time = "2026-06-07T21:06:50.165Z" },
{ url = "https://files.pythonhosted.org/packages/66/d7/2ff65c5e65c0d7476daf7e15c032e0805e36811185b9623e3238ad6c763e/aiohttp-3.14.1-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bb2c0c80d431c0d03f2c7dbf125150fedd4f0de17366a7ca33f7ccb822391842", size = 1904054, upload-time = "2026-06-07T21:06:52.035Z" },
{ url = "https://files.pythonhosted.org/packages/20/9c/d445818389df371f56d141d881153ba23183c4735a03f7356ffb43f7757d/aiohttp-3.14.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3e6fc1a85fa7194a1a7d19f44e8609180f4a8eb5fa4c7ed8b4355f080fad235c", size = 1790278, upload-time = "2026-06-07T21:06:54.049Z" },
{ url = "https://files.pythonhosted.org/packages/4d/aa/bf04cb4d865fc6101c2229a294ad744973b72e513fdc5a6b791e6983d72a/aiohttp-3.14.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:686b6c0d3911ec387b444ddf5dc62fb7f7c0a7d5186a7861626496a5ab4aff95", size = 1591795, upload-time = "2026-06-07T21:06:55.911Z" },
{ url = "https://files.pythonhosted.org/packages/dc/b4/4dac0038960427ba832f6609dfb4ea5437d7fd80c72001b9e48f834f428b/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:c6fa4dc7ad6f8109c70bb1499e589f76b0b792baf39f9b017eb92c8a81d0a199", size = 1728397, upload-time = "2026-06-07T21:06:57.777Z" },
{ url = "https://files.pythonhosted.org/packages/2b/f9/7cd4e8ad7aa3b75f17d56bb5498dd604a93d4e6eece822ba0568c413fff0/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:87a5eea1b2a5e21e1ebdbb33ad4165359189327e63fc4e4894693e7f821ac817", size = 1766504, upload-time = "2026-06-07T21:07:00.009Z" },
{ url = "https://files.pythonhosted.org/packages/f9/df/fc01d9fcad0f73fed3f3d361f1f94f975947b50dff82919f6dc2bf4316cc/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1c1421eb01d4fd608d88cc8290211d177a58532b55ad94076fb349c5bf467f0a", size = 1777806, upload-time = "2026-06-07T21:07:02.064Z" },
{ url = "https://files.pythonhosted.org/packages/41/09/47e2d090bddcc8fb4ccb4c314aadc32d7c5d9bb55f50f6ad1c92fc15d501/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:34b257ec41345c1e8f2df68fa908a7952f5de932723871eb633ecbbff396c9a4", size = 1580707, upload-time = "2026-06-07T21:07:03.942Z" },
{ url = "https://files.pythonhosted.org/packages/3d/36/f1a4ce904ae0b6930cfe9afc96d0896f7ec1a620c400405d63783bb95a9c/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:de538791a80e5d862addbc183f70f0158ac9b9bb872bb147f1fd2a683691e087", size = 1798121, upload-time = "2026-06-07T21:07:05.987Z" },
{ url = "https://files.pythonhosted.org/packages/70/0a/e0075ce9ca0279ee1d4f0c0b85f54fea02ebc83c3007651a72bece658fec/aiohttp-3.14.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:6f71173be42d3241d428f760122febb748de0623f44308a6f120d0dd9ec572e3", size = 1767580, upload-time = "2026-06-07T21:07:07.873Z" },
{ url = "https://files.pythonhosted.org/packages/3e/61/a0c0a8f327a9c52095cdd8e312391b00d3ed64ab6c72bb5c33d8ec251cf7/aiohttp-3.14.1-cp312-cp312-win32.whl", hash = "sha256:ec8dc383ee57ea3e883477dcca3f11b65d58199f1080acaf4cd6ad9a99698be4", size = 452771, upload-time = "2026-06-07T21:07:09.669Z" },
{ url = "https://files.pythonhosted.org/packages/df/d9/ea367c75f16ac9c6cdc8febb25e8318fa21a2b1bc8d6514d4b2d890bface/aiohttp-3.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:2aa92c87868cd13674989f9ee83e5f9f7ea4237589b728048e1f0c8f6caa3271", size = 479873, upload-time = "2026-06-07T21:07:11.538Z" },
{ url = "https://files.pythonhosted.org/packages/03/64/8d96784a7851156db8a4c6c3f6f91042fdf39fb15a4cc38c8b3c14833c45/aiohttp-3.14.1-cp312-cp312-win_arm64.whl", hash = "sha256:2c840c90759922cb5e6dda94596e079a30fb5a5ba548e7e0dc00574703940847", size = 448073, upload-time = "2026-06-07T21:07:13.637Z" },
{ url = "https://files.pythonhosted.org/packages/bc/97/bd137012dd97e1649162b099135a80e1fd59aaa807b2430fc448d1029aff/aiohttp-3.14.1-cp313-cp313-android_21_arm64_v8a.whl", hash = "sha256:b3a03285a7f9c7b016324574a6d92a1c895da6b978cb8f1deee3ac72bc6da178", size = 506882, upload-time = "2026-06-07T21:07:15.501Z" },
{ url = "https://files.pythonhosted.org/packages/ef/79/e5cc690e9d922a66887ceeaca53a8ffd5a7b0be3816142b7abc433742d89/aiohttp-3.14.1-cp313-cp313-android_21_x86_64.whl", hash = "sha256:2a73f487ab8ef5abbb24b7aa9b73e98eaba9e9e031804ff2416f02eca315ccaf", size = 515270, upload-time = "2026-06-07T21:07:17.53Z" },
{ url = "https://files.pythonhosted.org/packages/fe/22/a73ccbf9dbd6e26dda0b24d5fd5db7da92ee3383a79f47677ffb834c5c5b/aiohttp-3.14.1-cp313-cp313-ios_13_0_arm64_iphoneos.whl", hash = "sha256:915fbb7b41b115192259f8c9ae58f3ddc444d2b5579917270211858e606a4afd", size = 485841, upload-time = "2026-06-07T21:07:19.555Z" },
{ url = "https://files.pythonhosted.org/packages/3b/b9/57ed8eaf596321c2ad747bd480fb1700dbd7177c60dfc9e4c187f629662e/aiohttp-3.14.1-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:7fb4bdf95b0561a79f259f9d28fbc109728c5ee7f27aff6391f0ca703a329abe", size = 492088, upload-time = "2026-06-07T21:07:21.581Z" },
{ url = "https://files.pythonhosted.org/packages/78/c0/5ebe5270a7c140d7c6f79dcb018640225f14d406c149e4eec04a7d82fe71/aiohttp-3.14.1-cp313-cp313-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:1b9748363260121d2927704f5d4fc498150669ca3ae93625986ee89c8f80dcd4", size = 501564, upload-time = "2026-06-07T21:07:23.388Z" },
{ url = "https://files.pythonhosted.org/packages/75/7f/8cdaa24fc7983865e0915153b96a9ac5bcdd3548d64c5a27d17cecccad2d/aiohttp-3.14.1-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:86a6dab78b0e43e2897a3bbe15745aa60dc5423ca437b7b0b164c069bf91b876", size = 751998, upload-time = "2026-06-07T21:07:25.046Z" },
{ url = "https://files.pythonhosted.org/packages/b2/f4/c4227aacfacc5cb0cc2d119b65301d177912a6842cd64e120c47af76064f/aiohttp-3.14.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:4dfd6e47d3c44c2279907607f73a4240b88c69eb8b90da7e2441a8045dfd21da", size = 510918, upload-time = "2026-06-07T21:07:27.28Z" },
{ url = "https://files.pythonhosted.org/packages/ab/01/a2d5f96cd4e74424864d30bc0a7e44d0a12dacdcfa91b5b2d1bd3dca6bf3/aiohttp-3.14.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:317acd9f8602858dc7d59679812c376c7f0b97bcbbf16e0d6237f54141d8a8a6", size = 508657, upload-time = "2026-06-07T21:07:29.252Z" },
{ url = "https://files.pythonhosted.org/packages/e8/ed/3c0fb5c500fdd8e7ebc10d1889c04384fffa1a9163eac1356088ca9da1b1/aiohttp-3.14.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:bd869c427324e5cb15195793de951295710db28be7d818247f3097b4ab5d4b96", size = 1757907, upload-time = "2026-06-07T21:07:31.03Z" },
{ url = "https://files.pythonhosted.org/packages/0b/ab/d4c924d9bd5be3050c226612413ce68cb54c70d2c31b661bfc8d9a5b6a70/aiohttp-3.14.1-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:93b032b5ec3255473c143627d21a69ac74ae12f7f33974cb587c564d11b1066f", size = 1737565, upload-time = "2026-06-07T21:07:33.031Z" },
{ url = "https://files.pythonhosted.org/packages/19/2a/37326821ff779084020cdc33224d20b19f42f4183a500ff92022a739eda7/aiohttp-3.14.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f234b4deb12f3ad59127e037bc57c40c21e45b45282df7d3a55a0f409f595296", size = 1799018, upload-time = "2026-06-07T21:07:35.003Z" },
{ url = "https://files.pythonhosted.org/packages/b3/4f/6e947ba73e4ce09070761c05ed3a8ceb7c21f5e46798671d8b2aac0e4626/aiohttp-3.14.1-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:9af6779bfb46abf124068327abcdf9ce95c9ef8287a3e8da76ccf2d0f16c28fa", size = 1894416, upload-time = "2026-06-07T21:07:36.956Z" },
{ url = "https://files.pythonhosted.org/packages/9d/6e/dbf1d0625dc711fb2851f4f3c3055c39ed58bae92082d8c627dbe6013736/aiohttp-3.14.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:faccab372e66bc76d5731525e7f1143c922271725b9d38c9f97edcc66266b451", size = 1783881, upload-time = "2026-06-07T21:07:39.063Z" },
{ url = "https://files.pythonhosted.org/packages/44/c2/5e25098a67268ed369483ae7d1a58bd0a13d03aab860d2a0e4a6eb25b046/aiohttp-3.14.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f380468b09d2a81633ee863b0ec5648d364bd17bb8ecfb8c2f387f7ac1faf42c", size = 1587572, upload-time = "2026-06-07T21:07:41.058Z" },
{ url = "https://files.pythonhosted.org/packages/2a/bd/cf9cee17e140f942a3de73e658a543aa8fbf35a5fc67a9d2538d52d77f0b/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:97e704dcd26271f5bda3fa07c3ce0fb76d6d3f8659f4baa1a24442cc9ba177ca", size = 1722137, upload-time = "2026-06-07T21:07:43.014Z" },
{ url = "https://files.pythonhosted.org/packages/89/6d/5684f8c59045c96f81a18cefbc1fbbd79d25b88f1c622f2a5c5c08fcb632/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:269b76ac5394092b95bc4a098f4fc6c191c083c3bd12775d1e30e663132f6a09", size = 1755953, upload-time = "2026-06-07T21:07:45.933Z" },
{ url = "https://files.pythonhosted.org/packages/a8/40/35caf3170f8359760740a7d9aa0fff2e344bef98e1d1186f5a0f6dec17e6/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:5c0b3e614340c889d575451696374c9d17affd54cd607ca0babed8f8c37b9397", size = 1766479, upload-time = "2026-06-07T21:07:48.047Z" },
{ url = "https://files.pythonhosted.org/packages/6d/a1/b0c61e7a137f0d81de49a82023a6df73c3c16d6fefb0f8e4a93d21639002/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:5663ee9257cfa1add7253a7da3035a02f31b6600ec48261585e1800a81533080", size = 1580077, upload-time = "2026-06-07T21:07:50.069Z" },
{ url = "https://files.pythonhosted.org/packages/0b/41/194ea4623693009fcefebef7aef63c141754f153e9cd0d39d3b9e36c175c/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:603a2c834142172ffddc054067f5ec0ca65d57a0aa98a71bc81952573208e345", size = 1791688, upload-time = "2026-06-07T21:07:52.106Z" },
{ url = "https://files.pythonhosted.org/packages/ba/45/4de841f005cfe1fd63e2a2fe011262c515e2a62aa6994b15947e7d717ac9/aiohttp-3.14.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:cb21957bb8aca671c1765e32f58164cf0c50e6bf41c0bbbd16da20732ecaf588", size = 1761094, upload-time = "2026-06-07T21:07:54.113Z" },
{ url = "https://files.pythonhosted.org/packages/e4/ae/dbce10533d3896d544d5053939ed75b7dc31a1b0973d959b1b5ae21028d6/aiohttp-3.14.1-cp313-cp313-win32.whl", hash = "sha256:e509a55f681e6158c20f70f102f9cf61fb20fbc382272bc6d94b7343f2582780", size = 452662, upload-time = "2026-06-07T21:07:56.06Z" },
{ url = "https://files.pythonhosted.org/packages/7b/d9/0bf1a19362c32f06229da5e7ddfcec91f93474d6307f7a2d3135e9c674dc/aiohttp-3.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:1ac8531b638959718e18c2207fbfe297819875da46a740b29dfa29beba64355a", size = 479748, upload-time = "2026-06-07T21:07:58.319Z" },
{ url = "https://files.pythonhosted.org/packages/22/0a/62e7232dc9484fbec112ceb32efb6a624cc7994ec6e2b019286f17c4e8f2/aiohttp-3.14.1-cp313-cp313-win_arm64.whl", hash = "sha256:250d14af67f6b6a1a4a811049b1afa69d61d617fca6bf33149b3ab1a6dbcf7b8", size = 447723, upload-time = "2026-06-07T21:08:00.154Z" },
{ url = "https://files.pythonhosted.org/packages/c4/a1/5fafa04e1ca91ddb47608699d60649c1c6db3cf41c99e78fc4056f9513db/aiohttp-3.14.1-cp314-cp314-android_24_arm64_v8a.whl", hash = "sha256:7c106c26852ca1c2047c6b80384f17100b4e439af276f21ef3d4e2f450ae7e15", size = 508531, upload-time = "2026-06-07T21:08:02.093Z" },
{ url = "https://files.pythonhosted.org/packages/fa/2e/bfa02f699d87ffc86d5959270b28f1cb410add3ccaced8ed2e0b8a5238fc/aiohttp-3.14.1-cp314-cp314-android_24_x86_64.whl", hash = "sha256:20205f7f5ade7aaec9f4b500549bbc071b046453aed72f9c06dcab87896a83e8", size = 514718, upload-time = "2026-06-07T21:08:04.476Z" },
{ url = "https://files.pythonhosted.org/packages/85/a5/9594ad6289eebbc97d167c44213d557807f90e59115caad24de21ad2c3b1/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphoneos.whl", hash = "sha256:62a759436b29e677181a9e76bab8b8f689a29cb9c535f45f7c48c9c830d3f8c3", size = 487918, upload-time = "2026-06-07T21:08:06.377Z" },
{ url = "https://files.pythonhosted.org/packages/b4/61/16a32c36c3c49edec122a3dc811f2057df2f94d3b14aa107c8017d981618/aiohttp-3.14.1-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl", hash = "sha256:2964cbf553df4d7a57348da44d961d871895fc1ee4e8c322b2a95612c7b17fba", size = 494014, upload-time = "2026-06-07T21:08:08.263Z" },
{ url = "https://files.pythonhosted.org/packages/9b/89/3ebcf96ed99c05bec9c434aaac6963fd3cbab4a786ae739908a144d9ce44/aiohttp-3.14.1-cp314-cp314-ios_13_0_x86_64_iphonesimulator.whl", hash = "sha256:237651caadc3a59badd39319c54642b5299e9cc98a3a194310e55d5bb9f5e397", size = 502398, upload-time = "2026-06-07T21:08:10.244Z" },
{ url = "https://files.pythonhosted.org/packages/fd/3d/b74870a0c2d40c355928cd5b96c7a11fa821b8a40fc41365e64479b151fb/aiohttp-3.14.1-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:896e12dfdbbab9d8f7e16d2b28c6769a60126fa92095d1ebf9473d02593a2448", size = 758018, upload-time = "2026-06-07T21:08:12.447Z" },
{ url = "https://files.pythonhosted.org/packages/d3/66/f42f5c984d99e49c6cff5f26f590750f2e2f7ef1fcfb99966ab5be1b632e/aiohttp-3.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d03f281ed22579314ba00821ce20115a7c0ac430660b4cc05704a3f818b3e004", size = 512462, upload-time = "2026-06-07T21:08:14.624Z" },
{ url = "https://files.pythonhosted.org/packages/e9/a7/248e1aebe0c7810b0271e021a0f2a5eb6e78a051885b3c9df49f42a5802d/aiohttp-3.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:07eabb979d236335fed927e137a928c9adfb7df3b9ec7aa31726f133a62be983", size = 512824, upload-time = "2026-06-07T21:08:16.572Z" },
{ url = "https://files.pythonhosted.org/packages/26/97/2aa0e5ba0727dc3bd5aaebb7ccbc510f7dfb7fb961ec87497cd496635ab1/aiohttp-3.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4fe1f1087cbadb280b5e1bb054a4f00d1423c74d6626c5e48400d871d34ecefe", size = 1749898, upload-time = "2026-06-07T21:08:18.635Z" },
{ url = "https://files.pythonhosted.org/packages/00/8d/e97f6c96c891d457c8479d92a514ba194d0412f981d72c70341ee18488ed/aiohttp-3.14.1-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:367a9314fdc79dab0fac96e216cb41dd73c85bdca85306ce8999118ba7e0f333", size = 1710114, upload-time = "2026-06-07T21:08:20.892Z" },
{ url = "https://files.pythonhosted.org/packages/6f/e6/aa8d7e863048c8fceb5cd6ce74017311cec3ead07847387e12265fb4444e/aiohttp-3.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a24f677ebe83749039e7bdf862ff0bbb16818ae4193d4ef96505e269375bcce0", size = 1802541, upload-time = "2026-06-07T21:08:23.044Z" },
{ url = "https://files.pythonhosted.org/packages/83/a8/72193137de57fda4ebfae4563182d082c8856e3b6e9871d0b46f028fb369/aiohttp-3.14.1-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c83afe0ba876be7e943d2e0ba645809ad441575d2840c895c21ee5de93b9377a", size = 1875776, upload-time = "2026-06-07T21:08:25.288Z" },
{ url = "https://files.pythonhosted.org/packages/a0/18/938441025db6769a3464596b2410af3afde0b21eb2f204c6f766f68af4bd/aiohttp-3.14.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:634e385930fb6d2d479cf3aa66515955863b77a5e3c2b5894ca259a25b308602", size = 1760329, upload-time = "2026-06-07T21:08:27.363Z" },
{ url = "https://files.pythonhosted.org/packages/60/29/bf2496b4065e76e09fe48015aaffe5ce161d8f089b06ac6982070f653076/aiohttp-3.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:eeea07c4397bbc57719c4eed8f9c284874d4f175f9b6d57f7a1546b976d455ca", size = 1587293, upload-time = "2026-06-07T21:08:29.805Z" },
{ url = "https://files.pythonhosted.org/packages/49/a2/2136674d52123b1354bd05dd5753c318db47dc0c927cc70b27bab3755456/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:335c0cc3e3545ce98dcb9cfcb836f40c3411f43fa03dab757597d80c89af8a35", size = 1714756, upload-time = "2026-06-07T21:08:32.094Z" },
{ url = "https://files.pythonhosted.org/packages/a7/b9/e5fd2e6f915503081c0f9b1e8540947037929c70c191da2e4d54b31a21a1/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:ae6be797afdef264e8a84864a85b196ca06045586481b3df8a967322fd2fa844", size = 1721052, upload-time = "2026-06-07T21:08:34.167Z" },
{ url = "https://files.pythonhosted.org/packages/63/5a/2833e324a2263e104e31e2e91bc5bbee81bc499afd32203faee048a883f0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:8560b4d712474335d08907db7973f71912d3a9a8f1dee992ec06b5d2fe359496", size = 1766888, upload-time = "2026-06-07T21:08:36.95Z" },
{ url = "https://files.pythonhosted.org/packages/57/fa/dea6511870913162f3b2e8c42a7614eb203a4540b8c2da43e0bfb0548f3c/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7edd08e0a5deb1e8564a2fcd8f4561014a3f05252334671bbf55ddd47db0e5", size = 1581679, upload-time = "2026-06-07T21:08:39.292Z" },
{ url = "https://files.pythonhosted.org/packages/14/bd/3cf0d55e71784b33534e9710a67d382d900598b4787fbce6cc7317f8c42a/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:b6ff7fcee63287ae57b5df3e4f5957ce032122802509246dec1a5bcc55904c95", size = 1782021, upload-time = "2026-06-07T21:08:41.407Z" },
{ url = "https://files.pythonhosted.org/packages/c1/af/14bb5843eccbe234f4dfb78ab73e549d99727247e62ae5d62cbd22eaf5b0/aiohttp-3.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:6ffbb2f4ec1ceaff7e07d43922954da26b223d188bf30658e561b98e23089444", size = 1742574, upload-time = "2026-06-07T21:08:43.795Z" },
{ url = "https://files.pythonhosted.org/packages/f2/1e/fbeb7af9210a67ac0f9c9bec0f8f4568497924e33137a3d5b48e1cf85f3f/aiohttp-3.14.1-cp314-cp314-win32.whl", hash = "sha256:a9875b46d910cff3ea2f5962f9d266b465459fe634e22556ab9bd6fc1192eea0", size = 457773, upload-time = "2026-06-07T21:08:46.168Z" },
{ url = "https://files.pythonhosted.org/packages/f0/2b/13e8d741a9ec5db7d900c060554cf8352ab85e44e2a4469ebb9d377bda17/aiohttp-3.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:af8b4b81a960eeaf1234971ac3cd0ba5901f3cd42eae42a46b4d089a8b492719", size = 485001, upload-time = "2026-06-07T21:08:48.401Z" },
{ url = "https://files.pythonhosted.org/packages/df/30/491acfa2c4d6c3ff59c49a14fc1b50be3241e25bbb0c84c09e2da4d11395/aiohttp-3.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:cf4491381b1b57425c315a56a439251b1bdac07b2275f19a8c44bc57744532ec", size = 453809, upload-time = "2026-06-07T21:08:50.7Z" },
{ url = "https://files.pythonhosted.org/packages/34/e3/19dbe1a1f4cc6230eb9e314de7fe68053b0992f9302b27d12141a0b5db53/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:819c054312f1af92947e6a55883d1b66feefab11531a7fc45e0fb9b63880b5c2", size = 793320, upload-time = "2026-06-07T21:08:52.775Z" },
{ url = "https://files.pythonhosted.org/packages/7f/20/1b7182219ba1b108430d6e4dc53d25ae02dcfcf5a045b33af4e8c5167527/aiohttp-3.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:10ee9c1753a8f706345b22496c79fbddb5be0599e0823f3738b1534058e25340", size = 529077, upload-time = "2026-06-07T21:08:55Z" },
{ url = "https://files.pythonhosted.org/packages/b9/c8/14ce60ec31a2e5f5274bb17d383a6f7a3aabca31ac04eee05585bbadab16/aiohttp-3.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1601cc37baf5750ccacae618ec2daf020769581695550e3b654a911f859c563d", size = 532476, upload-time = "2026-06-07T21:08:57.176Z" },
{ url = "https://files.pythonhosted.org/packages/7e/02/9ac85e081e53da2e061b02fa7758fe0a12d17b8ce2d1f5e6c7cb76730328/aiohttp-3.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4d6e0ac9da31c9c04c84e1c0182ad8d6df35965a85cae29cd71d089621b3ae94", size = 1922347, upload-time = "2026-06-07T21:08:59.563Z" },
{ url = "https://files.pythonhosted.org/packages/c0/3e/d3ba07a0ab38b5389e10bec4362d21e10a4f667cba2d79ba30837b3a5059/aiohttp-3.14.1-cp314-cp314t-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:9e8f2d660c350b3d0e259c7a7e3d9b7fc8b41210cbcc3d4a7076ff0a5e5c2fdc", size = 1786465, upload-time = "2026-06-07T21:09:01.909Z" },
{ url = "https://files.pythonhosted.org/packages/0b/cb/e2ee978a00cfb2df829704a69528b18154eba5939f45bc1efa8f33aee4c5/aiohttp-3.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:4691802dda97be727f79d86818acaad7eb8e9252626a1d6b519fedbb92d5e251", size = 1909423, upload-time = "2026-06-07T21:09:04.357Z" },
{ url = "https://files.pythonhosted.org/packages/73/5d/1430334858b1022b58ae50399a918f0bd6fe8fa7fa183598d657ff61e040/aiohttp-3.14.1-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:c389c482a7e9b9dc3ee2701ac46c4125297a3818875b9c305ddb603c04828fd1", size = 2001906, upload-time = "2026-06-07T21:09:06.722Z" },
{ url = "https://files.pythonhosted.org/packages/66/4e/560c7472d3d198a23aa5c8b19a5115bf6a9b77b7d3e4bb363da320430ad2/aiohttp-3.14.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fc0cacab7ba4e56f0f81c82a98c09bed2f39c940107b03a34b168bdf7597edd3", size = 1877095, upload-time = "2026-06-07T21:09:09.011Z" },
{ url = "https://files.pythonhosted.org/packages/0d/f1/4745806578d447db4a784a8591e2dae3afdfc2bcb96f8f81271b13df6543/aiohttp-3.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:979ed4717f59b8bb12e3963378fa285d93d367e15bcd66c721311826d3c44a6c", size = 1676222, upload-time = "2026-06-07T21:09:11.461Z" },
{ url = "https://files.pythonhosted.org/packages/6a/c9/48255813cca749a229ef0ab476004ec623728ad79a9c0840616f6c076325/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:38e1e7daaea81df51c952e18483f323d878499a1e2bfe564790e0f9701d6f203", size = 1842922, upload-time = "2026-06-07T21:09:14.118Z" },
{ url = "https://files.pythonhosted.org/packages/3d/c0/bbd054e2bee909f529523a5af3891052606af5143c09f5f183ec3b234676/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:4132e72c608fe9fecb8f409113567605915b83e9bdd3ea56538d2f9cd35002f1", size = 1825035, upload-time = "2026-06-07T21:09:16.447Z" },
{ url = "https://files.pythonhosted.org/packages/a8/ae/90395d4376deceb74e09ec26b6adf7d2015a6f8802d6d84446af860fef04/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:eefd9cc9b6d4a2db5f00a26bc3e4f9acf71926a6ec557cd56c9c6f27c290b665", size = 1849512, upload-time = "2026-06-07T21:09:18.742Z" },
{ url = "https://files.pythonhosted.org/packages/93/bd/fb25f3049957553d4ce0ba6ae480aa2f592a6985497fca590837d16c1be0/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b165790117eea512d7f3fb22f1f6dad3d55a7189571993eb015591c1401276d1", size = 1668571, upload-time = "2026-06-07T21:09:21.458Z" },
{ url = "https://files.pythonhosted.org/packages/3f/22/7f73303d64dd567ff3addca90b556690ed1233a47b8f55d242fb90af3681/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:ed09c7eb1c391271c2ed0314a51903e72a3acb653d5ccfc264cdf3ef11f8269d", size = 1881159, upload-time = "2026-06-07T21:09:23.813Z" },
{ url = "https://files.pythonhosted.org/packages/44/be/0474c5a8b5640e1e4aa1923430a91f4151be82e511373fe764189b89aef5/aiohttp-3.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:99abd37084b82f5830c635fddd0b4993b9742a66eb746dacf433c8590e8f9e3c", size = 1841409, upload-time = "2026-06-07T21:09:26.207Z" },
{ url = "https://files.pythonhosted.org/packages/7b/3c/bb4a7cba26956cb3da4553cc2056cf67be5b5ff6e6d8fa4fbdff73bfb7ae/aiohttp-3.14.1-cp314-cp314t-win32.whl", hash = "sha256:47ddf841cdecc810749921d25606dee45857d12d2ad5ddb7b5bd7eab12e4b365", size = 494166, upload-time = "2026-06-07T21:09:28.505Z" },
{ url = "https://files.pythonhosted.org/packages/8a/84/ec80c2c1f66a952555a9f86df6b33af65108a6febfa0471b69013a12f807/aiohttp-3.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:5e78b522b7a6e27e0b25d19b247b75039ac4c94f99823e3c9e53ae1603a9f7e9", size = 530255, upload-time = "2026-06-07T21:09:30.843Z" },
{ url = "https://files.pythonhosted.org/packages/2a/71/6e22be134a4061ada85a92951b842f2657f17d926b727f3f94c56ae963d6/aiohttp-3.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:90d53f1609c29ccc2193945ef732428382a28f78d0456ae4d3daf0d48b74f0f6", size = 469640, upload-time = "2026-06-07T21:09:33.028Z" },
]
[[package]]
@@ -261,56 +311,50 @@ wheels = [
[[package]]
name = "brotli"
version = "1.1.0"
version = "1.2.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/2f/c2/f9e977608bdf958650638c3f1e28f85a1b075f075ebbe77db8555463787b/Brotli-1.1.0.tar.gz", hash = "sha256:81de08ac11bcb85841e440c13611c00b67d3bf82698314928d0b676362546724", size = 7372270, upload-time = "2023-09-07T14:05:41.643Z" }
sdist = { url = "https://files.pythonhosted.org/packages/f7/16/c92ca344d646e71a43b8bb353f0a6490d7f6e06210f8554c8f874e454285/brotli-1.2.0.tar.gz", hash = "sha256:e310f77e41941c13340a95976fe66a8a95b01e783d430eeaf7a2f87e0a57dd0a", size = 7388632, upload-time = "2025-11-05T18:39:42.86Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/96/12/ad41e7fadd5db55459c4c401842b47f7fee51068f86dd2894dd0dcfc2d2a/Brotli-1.1.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:a3daabb76a78f829cafc365531c972016e4aa8d5b4bf60660ad8ecee19df7ccc", size = 873068, upload-time = "2023-09-07T14:03:37.779Z" },
{ url = "https://files.pythonhosted.org/packages/95/4e/5afab7b2b4b61a84e9c75b17814198ce515343a44e2ed4488fac314cd0a9/Brotli-1.1.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:c8146669223164fc87a7e3de9f81e9423c67a79d6b3447994dfb9c95da16e2d6", size = 446244, upload-time = "2023-09-07T14:03:39.223Z" },
{ url = "https://files.pythonhosted.org/packages/9d/e6/f305eb61fb9a8580c525478a4a34c5ae1a9bcb12c3aee619114940bc513d/Brotli-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:30924eb4c57903d5a7526b08ef4a584acc22ab1ffa085faceb521521d2de32dd", size = 2906500, upload-time = "2023-09-07T14:03:40.858Z" },
{ url = "https://files.pythonhosted.org/packages/3e/4f/af6846cfbc1550a3024e5d3775ede1e00474c40882c7bf5b37a43ca35e91/Brotli-1.1.0-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ceb64bbc6eac5a140ca649003756940f8d6a7c444a68af170b3187623b43bebf", size = 2943950, upload-time = "2023-09-07T14:03:42.896Z" },
{ url = "https://files.pythonhosted.org/packages/b3/e7/ca2993c7682d8629b62630ebf0d1f3bb3d579e667ce8e7ca03a0a0576a2d/Brotli-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a469274ad18dc0e4d316eefa616d1d0c2ff9da369af19fa6f3daa4f09671fd61", size = 2918527, upload-time = "2023-09-07T14:03:44.552Z" },
{ url = "https://files.pythonhosted.org/packages/b3/96/da98e7bedc4c51104d29cc61e5f449a502dd3dbc211944546a4cc65500d3/Brotli-1.1.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:524f35912131cc2cabb00edfd8d573b07f2d9f21fa824bd3fb19725a9cf06327", size = 2845489, upload-time = "2023-09-07T14:03:46.594Z" },
{ url = "https://files.pythonhosted.org/packages/e8/ef/ccbc16947d6ce943a7f57e1a40596c75859eeb6d279c6994eddd69615265/Brotli-1.1.0-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:5b3cc074004d968722f51e550b41a27be656ec48f8afaeeb45ebf65b561481dd", size = 2914080, upload-time = "2023-09-07T14:03:48.204Z" },
{ url = "https://files.pythonhosted.org/packages/80/d6/0bd38d758d1afa62a5524172f0b18626bb2392d717ff94806f741fcd5ee9/Brotli-1.1.0-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:19c116e796420b0cee3da1ccec3b764ed2952ccfcc298b55a10e5610ad7885f9", size = 2813051, upload-time = "2023-09-07T14:03:50.348Z" },
{ url = "https://files.pythonhosted.org/packages/14/56/48859dd5d129d7519e001f06dcfbb6e2cf6db92b2702c0c2ce7d97e086c1/Brotli-1.1.0-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:510b5b1bfbe20e1a7b3baf5fed9e9451873559a976c1a78eebaa3b86c57b4265", size = 2938172, upload-time = "2023-09-07T14:03:52.395Z" },
{ url = "https://files.pythonhosted.org/packages/3d/77/a236d5f8cd9e9f4348da5acc75ab032ab1ab2c03cc8f430d24eea2672888/Brotli-1.1.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:a1fd8a29719ccce974d523580987b7f8229aeace506952fa9ce1d53a033873c8", size = 2933023, upload-time = "2023-09-07T14:03:53.96Z" },
{ url = "https://files.pythonhosted.org/packages/f1/87/3b283efc0f5cb35f7f84c0c240b1e1a1003a5e47141a4881bf87c86d0ce2/Brotli-1.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c247dd99d39e0338a604f8c2b3bc7061d5c2e9e2ac7ba9cc1be5a69cb6cd832f", size = 2935871, upload-time = "2024-10-18T12:32:16.688Z" },
{ url = "https://files.pythonhosted.org/packages/f3/eb/2be4cc3e2141dc1a43ad4ca1875a72088229de38c68e842746b342667b2a/Brotli-1.1.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:1b2c248cd517c222d89e74669a4adfa5577e06ab68771a529060cf5a156e9757", size = 2847784, upload-time = "2024-10-18T12:32:18.459Z" },
{ url = "https://files.pythonhosted.org/packages/66/13/b58ddebfd35edde572ccefe6890cf7c493f0c319aad2a5badee134b4d8ec/Brotli-1.1.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:2a24c50840d89ded6c9a8fdc7b6ed3692ed4e86f1c4a4a938e1e92def92933e0", size = 3034905, upload-time = "2024-10-18T12:32:20.192Z" },
{ url = "https://files.pythonhosted.org/packages/84/9c/bc96b6c7db824998a49ed3b38e441a2cae9234da6fa11f6ed17e8cf4f147/Brotli-1.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f31859074d57b4639318523d6ffdca586ace54271a73ad23ad021acd807eb14b", size = 2929467, upload-time = "2024-10-18T12:32:21.774Z" },
{ url = "https://files.pythonhosted.org/packages/e7/71/8f161dee223c7ff7fea9d44893fba953ce97cf2c3c33f78ba260a91bcff5/Brotli-1.1.0-cp311-cp311-win32.whl", hash = "sha256:39da8adedf6942d76dc3e46653e52df937a3c4d6d18fdc94a7c29d263b1f5b50", size = 333169, upload-time = "2023-09-07T14:03:55.404Z" },
{ url = "https://files.pythonhosted.org/packages/02/8a/fece0ee1057643cb2a5bbf59682de13f1725f8482b2c057d4e799d7ade75/Brotli-1.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:aac0411d20e345dc0920bdec5548e438e999ff68d77564d5e9463a7ca9d3e7b1", size = 357253, upload-time = "2023-09-07T14:03:56.643Z" },
{ url = "https://files.pythonhosted.org/packages/5c/d0/5373ae13b93fe00095a58efcbce837fd470ca39f703a235d2a999baadfbc/Brotli-1.1.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:32d95b80260d79926f5fab3c41701dbb818fde1c9da590e77e571eefd14abe28", size = 815693, upload-time = "2024-10-18T12:32:23.824Z" },
{ url = "https://files.pythonhosted.org/packages/8e/48/f6e1cdf86751300c288c1459724bfa6917a80e30dbfc326f92cea5d3683a/Brotli-1.1.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:b760c65308ff1e462f65d69c12e4ae085cff3b332d894637f6273a12a482d09f", size = 422489, upload-time = "2024-10-18T12:32:25.641Z" },
{ url = "https://files.pythonhosted.org/packages/06/88/564958cedce636d0f1bed313381dfc4b4e3d3f6015a63dae6146e1b8c65c/Brotli-1.1.0-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:316cc9b17edf613ac76b1f1f305d2a748f1b976b033b049a6ecdfd5612c70409", size = 873081, upload-time = "2023-09-07T14:03:57.967Z" },
{ url = "https://files.pythonhosted.org/packages/58/79/b7026a8bb65da9a6bb7d14329fd2bd48d2b7f86d7329d5cc8ddc6a90526f/Brotli-1.1.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:caf9ee9a5775f3111642d33b86237b05808dafcd6268faa492250e9b78046eb2", size = 446244, upload-time = "2023-09-07T14:03:59.319Z" },
{ url = "https://files.pythonhosted.org/packages/e5/18/c18c32ecea41b6c0004e15606e274006366fe19436b6adccc1ae7b2e50c2/Brotli-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:70051525001750221daa10907c77830bc889cb6d865cc0b813d9db7fefc21451", size = 2906505, upload-time = "2023-09-07T14:04:01.327Z" },
{ url = "https://files.pythonhosted.org/packages/08/c8/69ec0496b1ada7569b62d85893d928e865df29b90736558d6c98c2031208/Brotli-1.1.0-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7f4bf76817c14aa98cc6697ac02f3972cb8c3da93e9ef16b9c66573a68014f91", size = 2944152, upload-time = "2023-09-07T14:04:03.033Z" },
{ url = "https://files.pythonhosted.org/packages/ab/fb/0517cea182219d6768113a38167ef6d4eb157a033178cc938033a552ed6d/Brotli-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d0c5516f0aed654134a2fc936325cc2e642f8a0e096d075209672eb321cff408", size = 2919252, upload-time = "2023-09-07T14:04:04.675Z" },
{ url = "https://files.pythonhosted.org/packages/c7/53/73a3431662e33ae61a5c80b1b9d2d18f58dfa910ae8dd696e57d39f1a2f5/Brotli-1.1.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6c3020404e0b5eefd7c9485ccf8393cfb75ec38ce75586e046573c9dc29967a0", size = 2845955, upload-time = "2023-09-07T14:04:06.585Z" },
{ url = "https://files.pythonhosted.org/packages/55/ac/bd280708d9c5ebdbf9de01459e625a3e3803cce0784f47d633562cf40e83/Brotli-1.1.0-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:4ed11165dd45ce798d99a136808a794a748d5dc38511303239d4e2363c0695dc", size = 2914304, upload-time = "2023-09-07T14:04:08.668Z" },
{ url = "https://files.pythonhosted.org/packages/76/58/5c391b41ecfc4527d2cc3350719b02e87cb424ef8ba2023fb662f9bf743c/Brotli-1.1.0-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:4093c631e96fdd49e0377a9c167bfd75b6d0bad2ace734c6eb20b348bc3ea180", size = 2814452, upload-time = "2023-09-07T14:04:10.736Z" },
{ url = "https://files.pythonhosted.org/packages/c7/4e/91b8256dfe99c407f174924b65a01f5305e303f486cc7a2e8a5d43c8bec3/Brotli-1.1.0-cp312-cp312-musllinux_1_1_ppc64le.whl", hash = "sha256:7e4c4629ddad63006efa0ef968c8e4751c5868ff0b1c5c40f76524e894c50248", size = 2938751, upload-time = "2023-09-07T14:04:12.875Z" },
{ url = "https://files.pythonhosted.org/packages/5a/a6/e2a39a5d3b412938362bbbeba5af904092bf3f95b867b4a3eb856104074e/Brotli-1.1.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:861bf317735688269936f755fa136a99d1ed526883859f86e41a5d43c61d8966", size = 2933757, upload-time = "2023-09-07T14:04:14.551Z" },
{ url = "https://files.pythonhosted.org/packages/13/f0/358354786280a509482e0e77c1a5459e439766597d280f28cb097642fc26/Brotli-1.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:87a3044c3a35055527ac75e419dfa9f4f3667a1e887ee80360589eb8c90aabb9", size = 2936146, upload-time = "2024-10-18T12:32:27.257Z" },
{ url = "https://files.pythonhosted.org/packages/80/f7/daf538c1060d3a88266b80ecc1d1c98b79553b3f117a485653f17070ea2a/Brotli-1.1.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:c5529b34c1c9d937168297f2c1fde7ebe9ebdd5e121297ff9c043bdb2ae3d6fb", size = 2848055, upload-time = "2024-10-18T12:32:29.376Z" },
{ url = "https://files.pythonhosted.org/packages/ad/cf/0eaa0585c4077d3c2d1edf322d8e97aabf317941d3a72d7b3ad8bce004b0/Brotli-1.1.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:ca63e1890ede90b2e4454f9a65135a4d387a4585ff8282bb72964fab893f2111", size = 3035102, upload-time = "2024-10-18T12:32:31.371Z" },
{ url = "https://files.pythonhosted.org/packages/d8/63/1c1585b2aa554fe6dbce30f0c18bdbc877fa9a1bf5ff17677d9cca0ac122/Brotli-1.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:e79e6520141d792237c70bcd7a3b122d00f2613769ae0cb61c52e89fd3443839", size = 2930029, upload-time = "2024-10-18T12:32:33.293Z" },
{ url = "https://files.pythonhosted.org/packages/5f/3b/4e3fd1893eb3bbfef8e5a80d4508bec17a57bb92d586c85c12d28666bb13/Brotli-1.1.0-cp312-cp312-win32.whl", hash = "sha256:5f4d5ea15c9382135076d2fb28dde923352fe02951e66935a9efaac8f10e81b0", size = 333276, upload-time = "2023-09-07T14:04:16.49Z" },
{ url = "https://files.pythonhosted.org/packages/3d/d5/942051b45a9e883b5b6e98c041698b1eb2012d25e5948c58d6bf85b1bb43/Brotli-1.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:906bc3a79de8c4ae5b86d3d75a8b77e44404b0f4261714306e3ad248d8ab0951", size = 357255, upload-time = "2023-09-07T14:04:17.83Z" },
{ url = "https://files.pythonhosted.org/packages/0a/9f/fb37bb8ffc52a8da37b1c03c459a8cd55df7a57bdccd8831d500e994a0ca/Brotli-1.1.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:8bf32b98b75c13ec7cf774164172683d6e7891088f6316e54425fde1efc276d5", size = 815681, upload-time = "2024-10-18T12:32:34.942Z" },
{ url = "https://files.pythonhosted.org/packages/06/b3/dbd332a988586fefb0aa49c779f59f47cae76855c2d00f450364bb574cac/Brotli-1.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:7bc37c4d6b87fb1017ea28c9508b36bbcb0c3d18b4260fcdf08b200c74a6aee8", size = 422475, upload-time = "2024-10-18T12:32:36.485Z" },
{ url = "https://files.pythonhosted.org/packages/bb/80/6aaddc2f63dbcf2d93c2d204e49c11a9ec93a8c7c63261e2b4bd35198283/Brotli-1.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:3c0ef38c7a7014ffac184db9e04debe495d317cc9c6fb10071f7fefd93100a4f", size = 2906173, upload-time = "2024-10-18T12:32:37.978Z" },
{ url = "https://files.pythonhosted.org/packages/ea/1d/e6ca79c96ff5b641df6097d299347507d39a9604bde8915e76bf026d6c77/Brotli-1.1.0-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:91d7cc2a76b5567591d12c01f019dd7afce6ba8cba6571187e21e2fc418ae648", size = 2943803, upload-time = "2024-10-18T12:32:39.606Z" },
{ url = "https://files.pythonhosted.org/packages/ac/a3/d98d2472e0130b7dd3acdbb7f390d478123dbf62b7d32bda5c830a96116d/Brotli-1.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a93dde851926f4f2678e704fadeb39e16c35d8baebd5252c9fd94ce8ce68c4a0", size = 2918946, upload-time = "2024-10-18T12:32:41.679Z" },
{ url = "https://files.pythonhosted.org/packages/c4/a5/c69e6d272aee3e1423ed005d8915a7eaa0384c7de503da987f2d224d0721/Brotli-1.1.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:f0db75f47be8b8abc8d9e31bc7aad0547ca26f24a54e6fd10231d623f183d089", size = 2845707, upload-time = "2024-10-18T12:32:43.478Z" },
{ url = "https://files.pythonhosted.org/packages/58/9f/4149d38b52725afa39067350696c09526de0125ebfbaab5acc5af28b42ea/Brotli-1.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:6967ced6730aed543b8673008b5a391c3b1076d834ca438bbd70635c73775368", size = 2936231, upload-time = "2024-10-18T12:32:45.224Z" },
{ url = "https://files.pythonhosted.org/packages/5a/5a/145de884285611838a16bebfdb060c231c52b8f84dfbe52b852a15780386/Brotli-1.1.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:7eedaa5d036d9336c95915035fb57422054014ebdeb6f3b42eac809928e40d0c", size = 2848157, upload-time = "2024-10-18T12:32:46.894Z" },
{ url = "https://files.pythonhosted.org/packages/50/ae/408b6bfb8525dadebd3b3dd5b19d631da4f7d46420321db44cd99dcf2f2c/Brotli-1.1.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:d487f5432bf35b60ed625d7e1b448e2dc855422e87469e3f450aa5552b0eb284", size = 3035122, upload-time = "2024-10-18T12:32:48.844Z" },
{ url = "https://files.pythonhosted.org/packages/af/85/a94e5cfaa0ca449d8f91c3d6f78313ebf919a0dbd55a100c711c6e9655bc/Brotli-1.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:832436e59afb93e1836081a20f324cb185836c617659b07b129141a8426973c7", size = 2930206, upload-time = "2024-10-18T12:32:51.198Z" },
{ url = "https://files.pythonhosted.org/packages/c2/f0/a61d9262cd01351df22e57ad7c34f66794709acab13f34be2675f45bf89d/Brotli-1.1.0-cp313-cp313-win32.whl", hash = "sha256:43395e90523f9c23a3d5bdf004733246fba087f2948f87ab28015f12359ca6a0", size = 333804, upload-time = "2024-10-18T12:32:52.661Z" },
{ url = "https://files.pythonhosted.org/packages/7e/c1/ec214e9c94000d1c1974ec67ced1c970c148aa6b8d8373066123fc3dbf06/Brotli-1.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:9011560a466d2eb3f5a6e4929cf4a09be405c64154e12df0dd72713f6500e32b", size = 358517, upload-time = "2024-10-18T12:32:54.066Z" },
{ url = "https://files.pythonhosted.org/packages/7a/ef/f285668811a9e1ddb47a18cb0b437d5fc2760d537a2fe8a57875ad6f8448/brotli-1.2.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:15b33fe93cedc4caaff8a0bd1eb7e3dab1c61bb22a0bf5bdfdfd97cd7da79744", size = 863110, upload-time = "2025-11-05T18:38:12.978Z" },
{ url = "https://files.pythonhosted.org/packages/50/62/a3b77593587010c789a9d6eaa527c79e0848b7b860402cc64bc0bc28a86c/brotli-1.2.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:898be2be399c221d2671d29eed26b6b2713a02c2119168ed914e7d00ceadb56f", size = 445438, upload-time = "2025-11-05T18:38:14.208Z" },
{ url = "https://files.pythonhosted.org/packages/cd/e1/7fadd47f40ce5549dc44493877db40292277db373da5053aff181656e16e/brotli-1.2.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:350c8348f0e76fff0a0fd6c26755d2653863279d086d3aa2c290a6a7251135dd", size = 1534420, upload-time = "2025-11-05T18:38:15.111Z" },
{ url = "https://files.pythonhosted.org/packages/12/8b/1ed2f64054a5a008a4ccd2f271dbba7a5fb1a3067a99f5ceadedd4c1d5a7/brotli-1.2.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:2e1ad3fda65ae0d93fec742a128d72e145c9c7a99ee2fcd667785d99eb25a7fe", size = 1632619, upload-time = "2025-11-05T18:38:16.094Z" },
{ url = "https://files.pythonhosted.org/packages/89/5a/7071a621eb2d052d64efd5da2ef55ecdac7c3b0c6e4f9d519e9c66d987ef/brotli-1.2.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:40d918bce2b427a0c4ba189df7a006ac0c7277c180aee4617d99e9ccaaf59e6a", size = 1426014, upload-time = "2025-11-05T18:38:17.177Z" },
{ url = "https://files.pythonhosted.org/packages/26/6d/0971a8ea435af5156acaaccec1a505f981c9c80227633851f2810abd252a/brotli-1.2.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:2a7f1d03727130fc875448b65b127a9ec5d06d19d0148e7554384229706f9d1b", size = 1489661, upload-time = "2025-11-05T18:38:18.41Z" },
{ url = "https://files.pythonhosted.org/packages/f3/75/c1baca8b4ec6c96a03ef8230fab2a785e35297632f402ebb1e78a1e39116/brotli-1.2.0-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:9c79f57faa25d97900bfb119480806d783fba83cd09ee0b33c17623935b05fa3", size = 1599150, upload-time = "2025-11-05T18:38:19.792Z" },
{ url = "https://files.pythonhosted.org/packages/0d/1a/23fcfee1c324fd48a63d7ebf4bac3a4115bdb1b00e600f80f727d850b1ae/brotli-1.2.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:844a8ceb8483fefafc412f85c14f2aae2fb69567bf2a0de53cdb88b73e7c43ae", size = 1493505, upload-time = "2025-11-05T18:38:20.913Z" },
{ url = "https://files.pythonhosted.org/packages/36/e5/12904bbd36afeef53d45a84881a4810ae8810ad7e328a971ebbfd760a0b3/brotli-1.2.0-cp311-cp311-win32.whl", hash = "sha256:aa47441fa3026543513139cb8926a92a8e305ee9c71a6209ef7a97d91640ea03", size = 334451, upload-time = "2025-11-05T18:38:21.94Z" },
{ url = "https://files.pythonhosted.org/packages/02/8b/ecb5761b989629a4758c394b9301607a5880de61ee2ee5fe104b87149ebc/brotli-1.2.0-cp311-cp311-win_amd64.whl", hash = "sha256:022426c9e99fd65d9475dce5c195526f04bb8be8907607e27e747893f6ee3e24", size = 369035, upload-time = "2025-11-05T18:38:22.941Z" },
{ url = "https://files.pythonhosted.org/packages/11/ee/b0a11ab2315c69bb9b45a2aaed022499c9c24a205c3a49c3513b541a7967/brotli-1.2.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:35d382625778834a7f3061b15423919aa03e4f5da34ac8e02c074e4b75ab4f84", size = 861543, upload-time = "2025-11-05T18:38:24.183Z" },
{ url = "https://files.pythonhosted.org/packages/e1/2f/29c1459513cd35828e25531ebfcbf3e92a5e49f560b1777a9af7203eb46e/brotli-1.2.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7a61c06b334bd99bc5ae84f1eeb36bfe01400264b3c352f968c6e30a10f9d08b", size = 444288, upload-time = "2025-11-05T18:38:25.139Z" },
{ url = "https://files.pythonhosted.org/packages/3d/6f/feba03130d5fceadfa3a1bb102cb14650798c848b1df2a808356f939bb16/brotli-1.2.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:acec55bb7c90f1dfc476126f9711a8e81c9af7fb617409a9ee2953115343f08d", size = 1528071, upload-time = "2025-11-05T18:38:26.081Z" },
{ url = "https://files.pythonhosted.org/packages/2b/38/f3abb554eee089bd15471057ba85f47e53a44a462cfce265d9bf7088eb09/brotli-1.2.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:260d3692396e1895c5034f204f0db022c056f9e2ac841593a4cf9426e2a3faca", size = 1626913, upload-time = "2025-11-05T18:38:27.284Z" },
{ url = "https://files.pythonhosted.org/packages/03/a7/03aa61fbc3c5cbf99b44d158665f9b0dd3d8059be16c460208d9e385c837/brotli-1.2.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:072e7624b1fc4d601036ab3f4f27942ef772887e876beff0301d261210bca97f", size = 1419762, upload-time = "2025-11-05T18:38:28.295Z" },
{ url = "https://files.pythonhosted.org/packages/21/1b/0374a89ee27d152a5069c356c96b93afd1b94eae83f1e004b57eb6ce2f10/brotli-1.2.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:adedc4a67e15327dfdd04884873c6d5a01d3e3b6f61406f99b1ed4865a2f6d28", size = 1484494, upload-time = "2025-11-05T18:38:29.29Z" },
{ url = "https://files.pythonhosted.org/packages/cf/57/69d4fe84a67aef4f524dcd075c6eee868d7850e85bf01d778a857d8dbe0a/brotli-1.2.0-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:7a47ce5c2288702e09dc22a44d0ee6152f2c7eda97b3c8482d826a1f3cfc7da7", size = 1593302, upload-time = "2025-11-05T18:38:30.639Z" },
{ url = "https://files.pythonhosted.org/packages/d5/3b/39e13ce78a8e9a621c5df3aeb5fd181fcc8caba8c48a194cd629771f6828/brotli-1.2.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:af43b8711a8264bb4e7d6d9a6d004c3a2019c04c01127a868709ec29962b6036", size = 1487913, upload-time = "2025-11-05T18:38:31.618Z" },
{ url = "https://files.pythonhosted.org/packages/62/28/4d00cb9bd76a6357a66fcd54b4b6d70288385584063f4b07884c1e7286ac/brotli-1.2.0-cp312-cp312-win32.whl", hash = "sha256:e99befa0b48f3cd293dafeacdd0d191804d105d279e0b387a32054c1180f3161", size = 334362, upload-time = "2025-11-05T18:38:32.939Z" },
{ url = "https://files.pythonhosted.org/packages/1c/4e/bc1dcac9498859d5e353c9b153627a3752868a9d5f05ce8dedd81a2354ab/brotli-1.2.0-cp312-cp312-win_amd64.whl", hash = "sha256:b35c13ce241abdd44cb8ca70683f20c0c079728a36a996297adb5334adfc1c44", size = 369115, upload-time = "2025-11-05T18:38:33.765Z" },
{ url = "https://files.pythonhosted.org/packages/6c/d4/4ad5432ac98c73096159d9ce7ffeb82d151c2ac84adcc6168e476bb54674/brotli-1.2.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:9e5825ba2c9998375530504578fd4d5d1059d09621a02065d1b6bfc41a8e05ab", size = 861523, upload-time = "2025-11-05T18:38:34.67Z" },
{ url = "https://files.pythonhosted.org/packages/91/9f/9cc5bd03ee68a85dc4bc89114f7067c056a3c14b3d95f171918c088bf88d/brotli-1.2.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0cf8c3b8ba93d496b2fae778039e2f5ecc7cff99df84df337ca31d8f2252896c", size = 444289, upload-time = "2025-11-05T18:38:35.6Z" },
{ url = "https://files.pythonhosted.org/packages/2e/b6/fe84227c56a865d16a6614e2c4722864b380cb14b13f3e6bef441e73a85a/brotli-1.2.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c8565e3cdc1808b1a34714b553b262c5de5fbda202285782173ec137fd13709f", size = 1528076, upload-time = "2025-11-05T18:38:36.639Z" },
{ url = "https://files.pythonhosted.org/packages/55/de/de4ae0aaca06c790371cf6e7ee93a024f6b4bb0568727da8c3de112e726c/brotli-1.2.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:26e8d3ecb0ee458a9804f47f21b74845cc823fd1bb19f02272be70774f56e2a6", size = 1626880, upload-time = "2025-11-05T18:38:37.623Z" },
{ url = "https://files.pythonhosted.org/packages/5f/16/a1b22cbea436642e071adcaf8d4b350a2ad02f5e0ad0da879a1be16188a0/brotli-1.2.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:67a91c5187e1eec76a61625c77a6c8c785650f5b576ca732bd33ef58b0dff49c", size = 1419737, upload-time = "2025-11-05T18:38:38.729Z" },
{ url = "https://files.pythonhosted.org/packages/46/63/c968a97cbb3bdbf7f974ef5a6ab467a2879b82afbc5ffb65b8acbb744f95/brotli-1.2.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4ecdb3b6dc36e6d6e14d3a1bdc6c1057c8cbf80db04031d566eb6080ce283a48", size = 1484440, upload-time = "2025-11-05T18:38:39.916Z" },
{ url = "https://files.pythonhosted.org/packages/06/9d/102c67ea5c9fc171f423e8399e585dabea29b5bc79b05572891e70013cdd/brotli-1.2.0-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:3e1b35d56856f3ed326b140d3c6d9db91740f22e14b06e840fe4bb1923439a18", size = 1593313, upload-time = "2025-11-05T18:38:41.24Z" },
{ url = "https://files.pythonhosted.org/packages/9e/4a/9526d14fa6b87bc827ba1755a8440e214ff90de03095cacd78a64abe2b7d/brotli-1.2.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:54a50a9dad16b32136b2241ddea9e4df159b41247b2ce6aac0b3276a66a8f1e5", size = 1487945, upload-time = "2025-11-05T18:38:42.277Z" },
{ url = "https://files.pythonhosted.org/packages/5b/e8/3fe1ffed70cbef83c5236166acaed7bb9c766509b157854c80e2f766b38c/brotli-1.2.0-cp313-cp313-win32.whl", hash = "sha256:1b1d6a4efedd53671c793be6dd760fcf2107da3a52331ad9ea429edf0902f27a", size = 334368, upload-time = "2025-11-05T18:38:43.345Z" },
{ url = "https://files.pythonhosted.org/packages/ff/91/e739587be970a113b37b821eae8097aac5a48e5f0eca438c22e4c7dd8648/brotli-1.2.0-cp313-cp313-win_amd64.whl", hash = "sha256:b63daa43d82f0cdabf98dee215b375b4058cce72871fd07934f179885aad16e8", size = 369116, upload-time = "2025-11-05T18:38:44.609Z" },
{ url = "https://files.pythonhosted.org/packages/17/e1/298c2ddf786bb7347a1cd71d63a347a79e5712a7c0cba9e3c3458ebd976f/brotli-1.2.0-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:6c12dad5cd04530323e723787ff762bac749a7b256a5bece32b2243dd5c27b21", size = 863080, upload-time = "2025-11-05T18:38:45.503Z" },
{ url = "https://files.pythonhosted.org/packages/84/0c/aac98e286ba66868b2b3b50338ffbd85a35c7122e9531a73a37a29763d38/brotli-1.2.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:3219bd9e69868e57183316ee19c84e03e8f8b5a1d1f2667e1aa8c2f91cb061ac", size = 445453, upload-time = "2025-11-05T18:38:46.433Z" },
{ url = "https://files.pythonhosted.org/packages/ec/f1/0ca1f3f99ae300372635ab3fe2f7a79fa335fee3d874fa7f9e68575e0e62/brotli-1.2.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:963a08f3bebd8b75ac57661045402da15991468a621f014be54e50f53a58d19e", size = 1528168, upload-time = "2025-11-05T18:38:47.371Z" },
{ url = "https://files.pythonhosted.org/packages/d6/a6/2ebfc8f766d46df8d3e65b880a2e220732395e6d7dc312c1e1244b0f074a/brotli-1.2.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9322b9f8656782414b37e6af884146869d46ab85158201d82bab9abbcb971dc7", size = 1627098, upload-time = "2025-11-05T18:38:48.385Z" },
{ url = "https://files.pythonhosted.org/packages/f3/2f/0976d5b097ff8a22163b10617f76b2557f15f0f39d6a0fe1f02b1a53e92b/brotli-1.2.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", hash = "sha256:cf9cba6f5b78a2071ec6fb1e7bd39acf35071d90a81231d67e92d637776a6a63", size = 1419861, upload-time = "2025-11-05T18:38:49.372Z" },
{ url = "https://files.pythonhosted.org/packages/9c/97/d76df7176a2ce7616ff94c1fb72d307c9a30d2189fe877f3dd99af00ea5a/brotli-1.2.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7547369c4392b47d30a3467fe8c3330b4f2e0f7730e45e3103d7d636678a808b", size = 1484594, upload-time = "2025-11-05T18:38:50.655Z" },
{ url = "https://files.pythonhosted.org/packages/d3/93/14cf0b1216f43df5609f5b272050b0abd219e0b54ea80b47cef9867b45e7/brotli-1.2.0-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:fc1530af5c3c275b8524f2e24841cbe2599d74462455e9bae5109e9ff42e9361", size = 1593455, upload-time = "2025-11-05T18:38:51.624Z" },
{ url = "https://files.pythonhosted.org/packages/b3/73/3183c9e41ca755713bdf2cc1d0810df742c09484e2e1ddd693bee53877c1/brotli-1.2.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:d2d085ded05278d1c7f65560aae97b3160aeb2ea2c0b3e26204856beccb60888", size = 1488164, upload-time = "2025-11-05T18:38:53.079Z" },
{ url = "https://files.pythonhosted.org/packages/64/6a/0c78d8f3a582859236482fd9fa86a65a60328a00983006bcf6d83b7b2253/brotli-1.2.0-cp314-cp314-win32.whl", hash = "sha256:832c115a020e463c2f67664560449a7bea26b0c1fdd690352addad6d0a08714d", size = 339280, upload-time = "2025-11-05T18:38:54.02Z" },
{ url = "https://files.pythonhosted.org/packages/f5/10/56978295c14794b2c12007b07f3e41ba26acda9257457d7085b0bb3bb90c/brotli-1.2.0-cp314-cp314-win_amd64.whl", hash = "sha256:e7c0af964e0b4e3412a0ebf341ea26ec767fa0b4cf81abb5e897c9338b5ad6a3", size = 375639, upload-time = "2025-11-05T18:38:55.67Z" },
]
[[package]]
@@ -364,32 +408,39 @@ wheels = [
[[package]]
name = "cbor2"
version = "5.6.5"
version = "5.9.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e4/aa/ba55b47d51d27911981a18743b4d3cebfabccbb0598c09801b734cec4184/cbor2-5.6.5.tar.gz", hash = "sha256:b682820677ee1dbba45f7da11898d2720f92e06be36acec290867d5ebf3d7e09", size = 100886, upload-time = "2024-10-09T12:26:24.106Z" }
sdist = { url = "https://files.pythonhosted.org/packages/bd/cb/09939728be094d155b5d4ac262e39877875f5f7e36eea66beb359f647bd0/cbor2-5.9.0.tar.gz", hash = "sha256:85c7a46279ac8f226e1059275221e6b3d0e370d2bb6bd0500f9780781615bcea", size = 111231, upload-time = "2026-03-22T15:56:50.638Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/bd/b7/ef045245180510305648fd604244d3bb1ecf1b20de68f42ab5bc20198024/cbor2-5.6.5-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:863e0983989d56d5071270790e7ed8ddbda88c9e5288efdb759aba2efee670bc", size = 66452, upload-time = "2024-10-09T12:25:36.676Z" },
{ url = "https://files.pythonhosted.org/packages/41/20/5a9d93f86b1e8fd9d9db33aff39c0e3a8459e0803ec24bd837d8b56d4a1d/cbor2-5.6.5-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:5cff06464b8f4ca6eb9abcba67bda8f8334a058abc01005c8e616728c387ad32", size = 67421, upload-time = "2024-10-09T12:25:38.114Z" },
{ url = "https://files.pythonhosted.org/packages/0f/1e/2010f6d02dd117df88df64baf3eeca6aa6614cc81bdd6bfabf615889cf1f/cbor2-5.6.5-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f4c7dbcdc59ea7f5a745d3e30ee5e6b6ff5ce7ac244aa3de6786391b10027bb3", size = 260756, upload-time = "2024-10-09T12:25:39.657Z" },
{ url = "https://files.pythonhosted.org/packages/e1/84/e177d9bef4749d14f31c513b25e341ac84e403e2ffa2bde562eac9e6184b/cbor2-5.6.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:34cf5ab0dc310c3d0196caa6ae062dc09f6c242e2544bea01691fe60c0230596", size = 249210, upload-time = "2024-10-09T12:25:41.316Z" },
{ url = "https://files.pythonhosted.org/packages/38/75/ebfdbb281104b46419fe7cb65979de9927b75acebcb6afa0af291f728cd2/cbor2-5.6.5-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:6797b824b26a30794f2b169c0575301ca9b74ae99064e71d16e6ba0c9057de51", size = 249138, upload-time = "2024-10-09T12:25:42.432Z" },
{ url = "https://files.pythonhosted.org/packages/b2/1e/12d887fb1a8227a16181eeec5d43057e251204626d73e1c20a77046ac1b1/cbor2-5.6.5-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:73b9647eed1493097db6aad61e03d8f1252080ee041a1755de18000dd2c05f37", size = 247156, upload-time = "2024-10-09T12:25:43.588Z" },
{ url = "https://files.pythonhosted.org/packages/6f/76/478c12193de9517ce691bb8a3f7c00eafdd6a1bc3f7f23282ecdd99d02ec/cbor2-5.6.5-cp311-cp311-win_amd64.whl", hash = "sha256:6e14a1bf6269d25e02ef1d4008e0ce8880aa271d7c6b4c329dba48645764f60e", size = 66319, upload-time = "2024-10-09T12:25:44.621Z" },
{ url = "https://files.pythonhosted.org/packages/57/af/84ced14c541451696825b7b8ccbb7668f688372ad8ee74aaca4311e79672/cbor2-5.6.5-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:e25c2aebc9db99af7190e2261168cdde8ed3d639ca06868e4f477cf3a228a8e9", size = 67553, upload-time = "2024-10-09T12:25:45.767Z" },
{ url = "https://files.pythonhosted.org/packages/f2/d6/f63a840c68fed4de67d5441947af2dc695152cc488bb0e57312832fb923a/cbor2-5.6.5-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fde21ac1cf29336a31615a2c469a9cb03cf0add3ae480672d4d38cda467d07fc", size = 67569, upload-time = "2024-10-09T12:25:46.665Z" },
{ url = "https://files.pythonhosted.org/packages/77/ac/5fb79db6e882ec29680f4a974d35c098020a1b4709cad077667a8c3f4676/cbor2-5.6.5-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a8947c102cac79d049eadbd5e2ffb8189952890df7cbc3ee262bbc2f95b011a9", size = 276610, upload-time = "2024-10-09T12:25:48.14Z" },
{ url = "https://files.pythonhosted.org/packages/cf/cb/70751377d94112001d46c311b5c40b45f34863dfa78a6bc71b71f40c8c7f/cbor2-5.6.5-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:38886c41bebcd7dca57739439455bce759f1e4c551b511f618b8e9c1295b431b", size = 270004, upload-time = "2024-10-09T12:25:49.769Z" },
{ url = "https://files.pythonhosted.org/packages/f1/90/08800367e920aef31b93bd7b0cd6fadcb3a3f2243f4ed77a0d1c76f22b99/cbor2-5.6.5-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:ae2b49226224e92851c333b91d83292ec62eba53a19c68a79890ce35f1230d70", size = 264913, upload-time = "2024-10-09T12:25:50.92Z" },
{ url = "https://files.pythonhosted.org/packages/a8/9c/76b11a5ea7548bccb0dfef3e8fb3ede48bfeb39348f0c217519e0c40d33a/cbor2-5.6.5-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:f2764804ffb6553283fc4afb10a280715905a4cea4d6dc7c90d3e89c4a93bc8d", size = 266751, upload-time = "2024-10-09T12:25:52.777Z" },
{ url = "https://files.pythonhosted.org/packages/10/18/3866693a87c90cb12f7942e791d0f03a40ba44887dde7b7fc85319647efe/cbor2-5.6.5-cp312-cp312-win_amd64.whl", hash = "sha256:a3ac50485cf67dfaab170a3e7b527630e93cb0a6af8cdaa403054215dff93adf", size = 66739, upload-time = "2024-10-09T12:25:54.606Z" },
{ url = "https://files.pythonhosted.org/packages/2b/69/77e93caae71d1baee927c9762e702c464715d88073133052c74ecc9d37d4/cbor2-5.6.5-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:f0d0a9c5aabd48ecb17acf56004a7542a0b8d8212be52f3102b8218284bd881e", size = 67647, upload-time = "2024-10-09T12:25:55.637Z" },
{ url = "https://files.pythonhosted.org/packages/84/83/cb941d4fd10e4696b2c0f6fb2e3056d9a296e5765b2000a69e29a507f819/cbor2-5.6.5-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:61ceb77e6aa25c11c814d4fe8ec9e3bac0094a1f5bd8a2a8c95694596ea01e08", size = 67657, upload-time = "2024-10-09T12:25:56.528Z" },
{ url = "https://files.pythonhosted.org/packages/5c/3f/e16a1e29994483c751b714cdf61d2956290b0b30e94690fa714a9f155c5c/cbor2-5.6.5-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:97a7e409b864fecf68b2ace8978eb5df1738799a333ec3ea2b9597bfcdd6d7d2", size = 275863, upload-time = "2024-10-09T12:25:57.462Z" },
{ url = "https://files.pythonhosted.org/packages/64/04/f64bda3eea649fe6644c59f13d0e1f4666d975ce305cadf13835233b2a26/cbor2-5.6.5-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7f6d69f38f7d788b04c09ef2b06747536624b452b3c8b371ab78ad43b0296fab", size = 269131, upload-time = "2024-10-09T12:25:59.635Z" },
{ url = "https://files.pythonhosted.org/packages/f4/8d/0d5ad3467f70578b032b3f52eb0f01f0327d5ae6b1f9e7d4d4e01a73aa95/cbor2-5.6.5-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f91e6d74fa6917df31f8757fdd0e154203b0dd0609ec53eb957016a2b474896a", size = 264728, upload-time = "2024-10-09T12:26:01.407Z" },
{ url = "https://files.pythonhosted.org/packages/77/cb/9b4f7890325eaa374c21fcccfee61a099ccb9ea0bc0f606acf7495f9568c/cbor2-5.6.5-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:5ce13a27ef8fddf643fc17a753fe34aa72b251d03c23da6a560c005dc171085b", size = 266314, upload-time = "2024-10-09T12:26:02.451Z" },
{ url = "https://files.pythonhosted.org/packages/a8/cd/793dc041395609f5dd1edfdf0aecde504dc0fd35ed67eb3b2db79fb8ef4d/cbor2-5.6.5-cp313-cp313-win_amd64.whl", hash = "sha256:54c72a3207bb2d4480c2c39dad12d7971ce0853a99e3f9b8d559ce6eac84f66f", size = 66792, upload-time = "2024-10-09T12:26:03.615Z" },
{ url = "https://files.pythonhosted.org/packages/9b/ef/1c4698cac96d792005ef0611832f38eaee477c275ab4b02cbfc4daba7ad3/cbor2-5.6.5-py3-none-any.whl", hash = "sha256:3038523b8fc7de312bb9cdcbbbd599987e64307c4db357cd2030c472a6c7d468", size = 23752, upload-time = "2024-10-09T12:26:23.167Z" },
{ url = "https://files.pythonhosted.org/packages/43/aa/317c7118b8dda4c9563125c1a12c70c5b41e36677964a49c72b1aac061ec/cbor2-5.9.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:0485d3372fc832c5e16d4eb45fa1a20fc53e806e6c29a1d2b0d3e176cedd52b9", size = 70578, upload-time = "2026-03-22T15:56:03.835Z" },
{ url = "https://files.pythonhosted.org/packages/31/43/fe29b1f897770011a5e7497f4523c2712282ee4a6cbf775ea6383fb7afb9/cbor2-5.9.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a9d6e4e0f988b0e766509a8071975a8ee99f930e14a524620bf38083106158d2", size = 268738, upload-time = "2026-03-22T15:56:05.222Z" },
{ url = "https://files.pythonhosted.org/packages/0a/1a/e494568f3d8aafbcdfe361df44c3bcf5cdab5183e25ea08e3d3f9fcf4075/cbor2-5.9.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5326336f633cc89dfe543c78829c16c3a6449c2c03277d1ddba99086c3323363", size = 262571, upload-time = "2026-03-22T15:56:06.411Z" },
{ url = "https://files.pythonhosted.org/packages/42/2e/92acd6f87382fd44a34d9d7e85cc45372e6ba664040b72d1d9df648b25d0/cbor2-5.9.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:5e702b02d42a5ace45425b595ffe70fe35aebaf9a3cdfdc2c758b6189c744422", size = 262356, upload-time = "2026-03-22T15:56:08.236Z" },
{ url = "https://files.pythonhosted.org/packages/3f/68/52c039a28688baeeb78b0be7483855e6c66ea05884a937444deede0c87b8/cbor2-5.9.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:2372d357d403e7912f104ff085950ffc82a5854d6d717f1ca1ce16a40a0ef5a7", size = 257604, upload-time = "2026-03-22T15:56:09.835Z" },
{ url = "https://files.pythonhosted.org/packages/5b/e4/10d96a7f73ed9227090ce6e3df5d73329eb6a267dab7d5b989e6fbf6c504/cbor2-5.9.0-cp311-cp311-win_amd64.whl", hash = "sha256:1d02b65f070fd726bdc310d927228975bb655d155bf059b6eb7cacefb3dca86f", size = 69388, upload-time = "2026-03-22T15:56:11.28Z" },
{ url = "https://files.pythonhosted.org/packages/d4/c6/eea5829aa5a649db540f47ea35f4bf2313383d28246f0cbc50432cfad6b3/cbor2-5.9.0-cp311-cp311-win_arm64.whl", hash = "sha256:837754ece9052b3f607047e1741e5f852a538aa2b0ee3db11c82a8fa11804aa4", size = 65315, upload-time = "2026-03-22T15:56:12.326Z" },
{ url = "https://files.pythonhosted.org/packages/ee/39/72d8a5a4b06565561ec28f4fcb41aff7bb77f51705c01f00b8254a2aca4f/cbor2-5.9.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1f223dffb1bcdd2764665f04c1152943d9daa4bc124a576cd8dee1cad4264313", size = 71223, upload-time = "2026-03-22T15:56:13.68Z" },
{ url = "https://files.pythonhosted.org/packages/09/fd/7ddf3d3153b54c69c3be77172b8d9aa3a9d74f62a7fbde614d53eaeed9a4/cbor2-5.9.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ae6c706ac1d85a0b3cb3395308fd0c4d55e3202b4760773675957e93cdff45fc", size = 287865, upload-time = "2026-03-22T15:56:14.813Z" },
{ url = "https://files.pythonhosted.org/packages/db/9d/7ede2cc42f9bb4260492e7d29d2aab781eacbbcfb09d983de1e695077199/cbor2-5.9.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4cd43d8fc374b31643b2830910f28177a606a7bc84975a62675dd3f2e320fc7b", size = 288246, upload-time = "2026-03-22T15:56:16.113Z" },
{ url = "https://files.pythonhosted.org/packages/ce/9d/588ebc7c5bc5843f609b05fe07be8575c7dec987735b0bbc908ac9c1264a/cbor2-5.9.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:4aa07b392cc3d76fb31c08a46a226b58c320d1c172ff3073e864409ced7bc50f", size = 280214, upload-time = "2026-03-22T15:56:17.519Z" },
{ url = "https://files.pythonhosted.org/packages/f7/a1/6fc8f4b15c6a27e7fbb7966c30c2b4b18c274a3221fa2f5e6235502d34bc/cbor2-5.9.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:971d425b3a23b75953d8853d5f9911bdeefa09d759ee3b5e6b07b5ff3cbd9073", size = 282162, upload-time = "2026-03-22T15:56:18.975Z" },
{ url = "https://files.pythonhosted.org/packages/cf/20/9a22cfe08be16ddfeef2542cf4eeed1b29f3f57ddbba0b42f7e0bb8331fd/cbor2-5.9.0-cp312-cp312-win_amd64.whl", hash = "sha256:34a6cb15e6ab6a8eae94ad2041731cd3ef786af43a8df99f847969af5b902ee7", size = 70049, upload-time = "2026-03-22T15:56:20.502Z" },
{ url = "https://files.pythonhosted.org/packages/c6/9e/695f92d09006614034e25a9f5b10620f3b219f79c1bec3c37b7c6f27a7a9/cbor2-5.9.0-cp312-cp312-win_arm64.whl", hash = "sha256:7d1ddc4541e7367ac58c2470cc0df847f7137167fe4f5729e2d3cc0b993d7da4", size = 65382, upload-time = "2026-03-22T15:56:21.526Z" },
{ url = "https://files.pythonhosted.org/packages/81/c5/4901e21a8afe9448fd947b11e8f383903207cd6dd0800e5f5a386838de5b/cbor2-5.9.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:fbb06f34aa645b4deca66643bba3d400d20c15312d1fe88d429be60c1ab50f27", size = 71284, upload-time = "2026-03-22T15:56:22.836Z" },
{ url = "https://files.pythonhosted.org/packages/1b/10/df643a381aebc3f05486de4813662bc58accb640fc3275cb276a75e89694/cbor2-5.9.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ac684fe195c39821fca70d18afbf748f728aefbfbf88456018d299e559b8cae0", size = 287682, upload-time = "2026-03-22T15:56:24.024Z" },
{ url = "https://files.pythonhosted.org/packages/c6/0c/8aa6b766059ae4a0ca1ec3ff96fe3823a69a7be880dba2e249f7fbe2700b/cbor2-5.9.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2a54fbb32cb828c214f7f333a707e4aec61182e7efdc06ea5d9596d3ecee624a", size = 288009, upload-time = "2026-03-22T15:56:25.305Z" },
{ url = "https://files.pythonhosted.org/packages/74/07/6236bc25c183a9cf7e8062e5dddf9eae9b0b14ebf14a58a69fe5a1e872c6/cbor2-5.9.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:4753a6d1bc71054d9179557bc65740860f185095ccb401d46637fff028a5b3ec", size = 280437, upload-time = "2026-03-22T15:56:26.479Z" },
{ url = "https://files.pythonhosted.org/packages/4e/0a/84328d23c3c68874ac6497edb9b1900579a1028efa54734df3f1762bbc15/cbor2-5.9.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:380e534482b843e43442b87d8777a7bf9bed20cb7526f89b780c3400f617304b", size = 282247, upload-time = "2026-03-22T15:56:28.644Z" },
{ url = "https://files.pythonhosted.org/packages/9b/f6/89b4627e09d028c8e5fcaf7cb55f225c33ce6e037ec1844e65d02bcfa945/cbor2-5.9.0-cp313-cp313-win_amd64.whl", hash = "sha256:dcf0f695873e5c94bd072d6af8698e72b8fb7f7a18f37e0bced1041b7111a6cf", size = 70089, upload-time = "2026-03-22T15:56:29.801Z" },
{ url = "https://files.pythonhosted.org/packages/e2/7c/efadcd5f0102db692490e4e206988a2f98d39a09912090db497a2b800885/cbor2-5.9.0-cp313-cp313-win_arm64.whl", hash = "sha256:f7c9751a9611601ab326d8f5837f01379195bbf06175fb4effeb552140e7c9e8", size = 65466, upload-time = "2026-03-22T15:56:30.823Z" },
{ url = "https://files.pythonhosted.org/packages/08/7d/9ccc36d10ef96e6038e48046ebe1ce35a1e7814da0e1e204d09e6ef09b8d/cbor2-5.9.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:23606d31ba1368bd1b6602e3020ee88fe9523ca80e8630faf6b2fc904fd84560", size = 71500, upload-time = "2026-03-22T15:56:31.876Z" },
{ url = "https://files.pythonhosted.org/packages/70/e1/a6cca2cc72e13f00030c6a649f57ae703eb2c620806ab70c40db8eab33fa/cbor2-5.9.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0322296b9d52f55880e300ba8ba09ecf644303b99b51138bbb1c0fb644fa7c3e", size = 286953, upload-time = "2026-03-22T15:56:33.292Z" },
{ url = "https://files.pythonhosted.org/packages/08/3c/24cd5ef488a957d90e016f200a3aad820e4c2f85edd61c9fe4523007a1ee/cbor2-5.9.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:422817286c1d0ce947fb2f7eca9212b39bddd7231e8b452e2d2cc52f15332dba", size = 285454, upload-time = "2026-03-22T15:56:34.703Z" },
{ url = "https://files.pythonhosted.org/packages/a4/35/dca96818494c0ba47cdd73e8d809b27fa91f8fa0ce32a068a09237687454/cbor2-5.9.0-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:9a4907e0c3035bb8836116854ed8e56d8aef23909d601fa59706320897ec2551", size = 279441, upload-time = "2026-03-22T15:56:35.888Z" },
{ url = "https://files.pythonhosted.org/packages/a4/44/d3362378b16e53cf7e535a3f5aed8476e2109068154e24e31981ef5bde9e/cbor2-5.9.0-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:fb7afe77f8d269e42d7c4b515c6fd14f1ccc0625379fb6829b269f493d16eddd", size = 279673, upload-time = "2026-03-22T15:56:37.08Z" },
{ url = "https://files.pythonhosted.org/packages/43/d1/3533a697e5842fff7c2f64912eb251f8dcab3a8b5d88e228d6eebc3b5021/cbor2-5.9.0-cp314-cp314-win_amd64.whl", hash = "sha256:86baf870d4c0bfc6f79de3801f3860a84ab76d9c8b0abb7f081f2c14c38d79d3", size = 71940, upload-time = "2026-03-22T15:56:38.366Z" },
{ url = "https://files.pythonhosted.org/packages/ff/e2/c6ba75f3fb25dfa15ab6999cc8709c821987e9ed8e375d7f58539261bcb9/cbor2-5.9.0-cp314-cp314-win_arm64.whl", hash = "sha256:7221483fad0c63afa4244624d552abf89d7dfdbc5f5edfc56fc1ff2b4b818975", size = 67639, upload-time = "2026-03-22T15:56:39.39Z" },
{ url = "https://files.pythonhosted.org/packages/42/ff/b83492b096fbef26e9cb62c1a4bf2d3cef579ea7b33138c6c37c4ae66f67/cbor2-5.9.0-py3-none-any.whl", hash = "sha256:27695cbd70c90b8de5c4a284642c2836449b14e2c2e07e3ffe0744cb7669a01b", size = 24627, upload-time = "2026-03-22T15:56:48.847Z" },
]
[[package]]
@@ -1475,14 +1526,14 @@ wheels = [
[[package]]
name = "marshmallow"
version = "3.26.1"
version = "3.26.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "packaging" },
]
sdist = { url = "https://files.pythonhosted.org/packages/ab/5e/5e53d26b42ab75491cda89b871dab9e97c840bf12c63ec58a1919710cd06/marshmallow-3.26.1.tar.gz", hash = "sha256:e6d8affb6cb61d39d26402096dc0aee12d5a26d490a121f118d2e81dc0719dc6", size = 221825, upload-time = "2025-02-03T15:32:25.093Z" }
sdist = { url = "https://files.pythonhosted.org/packages/55/79/de6c16cc902f4fc372236926b0ce2ab7845268dcc30fb2fbb7f71b418631/marshmallow-3.26.2.tar.gz", hash = "sha256:bbe2adb5a03e6e3571b573f42527c6fe926e17467833660bebd11593ab8dfd57", size = 222095, upload-time = "2025-12-22T06:53:53.309Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/34/75/51952c7b2d3873b44a0028b1bd26a25078c18f92f256608e8d1dc61b39fd/marshmallow-3.26.1-py3-none-any.whl", hash = "sha256:3350409f20a70a7e4e11a27661187b77cdcaeb20abca41c1454fe33636bea09c", size = 50878, upload-time = "2025-02-03T15:32:22.295Z" },
{ url = "https://files.pythonhosted.org/packages/be/2f/5108cb3ee4ba6501748c4908b908e55f42a5b66245b4cfe0c99326e1ef6e/marshmallow-3.26.2-py3-none-any.whl", hash = "sha256:013fa8a3c4c276c24d26d84ce934dc964e2aa794345a0f8c7e5a7191482c8a73", size = 50964, upload-time = "2025-12-22T06:53:51.801Z" },
]
[[package]]
@@ -2062,16 +2113,16 @@ wheels = [
[[package]]
name = "pydantic-settings"
version = "2.14.0"
version = "2.14.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pydantic" },
{ name = "python-dotenv" },
{ name = "typing-inspection" },
]
sdist = { url = "https://files.pythonhosted.org/packages/42/98/c8345dccdc31de4228c039a98f6467a941e39558da41c1744fbe29fa5666/pydantic_settings-2.14.0.tar.gz", hash = "sha256:24285fd4b0e0c06507dd9fdfd331ee23794305352aaec8fc4eb92d4047aeb67d", size = 235709, upload-time = "2026-04-20T13:37:40.293Z" }
sdist = { url = "https://files.pythonhosted.org/packages/5c/b5/8f48e906c3e0205276e8bd8cb7512217a87b2685304d64be27cad5b3019f/pydantic_settings-2.14.2.tar.gz", hash = "sha256:c19dd64b19097f1de80184f0cc7b0272a13ae6e170cbf240a3e27e381ed14a5f", size = 237700, upload-time = "2026-06-19T13:44:56.324Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/01/dd/bebff3040138f00ae8a102d426b27349b9a49acc310fcae7f92112d867e3/pydantic_settings-2.14.0-py3-none-any.whl", hash = "sha256:fc8d5d692eb7092e43c8647c1c35a3ecd00e040fcf02ed86f4cb5458ca62182e", size = 60940, upload-time = "2026-04-20T13:37:38.586Z" },
{ url = "https://files.pythonhosted.org/packages/77/c1/6e422f34e569cf8e18df68d1939c81c099d2b61e4f7d9621c8a77560799c/pydantic_settings-2.14.2-py3-none-any.whl", hash = "sha256:a20c97b37910b6550d5ea50fbcc2d4187defe58cd57070b73863d069419c9440", size = 61715, upload-time = "2026-06-19T13:44:55.02Z" },
]
[[package]]
@@ -2085,11 +2136,11 @@ wheels = [
[[package]]
name = "pyjwt"
version = "2.10.1"
version = "2.13.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/e7/46/bd74733ff231675599650d3e47f361794b22ef3e3770998dda30d3b63726/pyjwt-2.10.1.tar.gz", hash = "sha256:3cc5772eb20009233caf06e9d8a0577824723b44e6648ee0a2aedb6cf9381953", size = 87785, upload-time = "2024-11-28T03:43:29.933Z" }
sdist = { url = "https://files.pythonhosted.org/packages/3b/81/58d0ac84e1ef3a3843791d6954d94c0b33d526c75eeb1efbce9d0a4c4077/pyjwt-2.13.0.tar.gz", hash = "sha256:41571c89ca91598c79e8ef18a2d07367d4810fbbd6f637794879baf1b7703423", size = 107515, upload-time = "2026-05-21T19:54:36.618Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/61/ad/689f02752eeec26aed679477e80e632ef1b682313be70793d798c1d5fc8f/PyJWT-2.10.1-py3-none-any.whl", hash = "sha256:dcdd193e30abefd5debf142f9adfcdd2b58004e644f25406ffaebd50bd98dacb", size = 22997, upload-time = "2024-11-28T03:43:27.893Z" },
{ url = "https://files.pythonhosted.org/packages/a3/5e/ecf12fdb62546d64385c158514e9b2b671f7832108ef2ecd2020ce0af2d1/pyjwt-2.13.0-py3-none-any.whl", hash = "sha256:66adcc2aff09b3f1bbd95fc1e1577df8ac8723c978552fd43304c8a290ac5728", size = 31274, upload-time = "2026-05-21T19:54:35.362Z" },
]
[[package]]
@@ -2149,11 +2200,11 @@ wheels = [
[[package]]
name = "python-dotenv"
version = "1.1.1"
version = "1.2.2"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f6/b0/4bc07ccd3572a2f9df7e6782f52b0c6c90dcbb803ac4a167702d7d0dfe1e/python_dotenv-1.1.1.tar.gz", hash = "sha256:a8a6399716257f45be6a007360200409fce5cda2661e3dec71d23dc15f6189ab", size = 41978, upload-time = "2025-06-24T04:21:07.341Z" }
sdist = { url = "https://files.pythonhosted.org/packages/82/ed/0301aeeac3e5353ef3d94b6ec08bbcabd04a72018415dcb29e588514bba8/python_dotenv-1.2.2.tar.gz", hash = "sha256:2c371a91fbd7ba082c2c1dc1f8bf89ca22564a087c2c287cd9b662adde799cf3", size = 50135, upload-time = "2026-03-01T16:00:26.196Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/5f/ed/539768cf28c661b5b068d66d96a2f155c4971a5d55684a514c1a0e0dec2f/python_dotenv-1.1.1-py3-none-any.whl", hash = "sha256:31f23644fe2602f88ff55e1f5c79ba497e01224ee7737937930c448e4d0e24dc", size = 20556, upload-time = "2025-06-24T04:21:06.073Z" },
{ url = "https://files.pythonhosted.org/packages/0b/d7/1959b9648791274998a9c3526f6d0ec8fd2233e4d4acce81bbae76b44b2a/python_dotenv-1.2.2-py3-none-any.whl", hash = "sha256:1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a", size = 22101, upload-time = "2026-03-01T16:00:25.09Z" },
]
[[package]]
@@ -2167,11 +2218,11 @@ wheels = [
[[package]]
name = "python-multipart"
version = "0.0.20"
version = "0.0.32"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/f3/87/f44d7c9f274c7ee665a29b885ec97089ec5dc034c7f3fafa03da9e39a09e/python_multipart-0.0.20.tar.gz", hash = "sha256:8dd0cab45b8e23064ae09147625994d090fa46f5b0d1e13af944c331a7fa9d13", size = 37158, upload-time = "2024-12-16T19:45:46.972Z" }
sdist = { url = "https://files.pythonhosted.org/packages/5b/42/55c32bb9b12693c092ad250a0e82edb5b31ddeda6eb772de5f308b3804ad/python_multipart-0.0.32.tar.gz", hash = "sha256:be54b7f3fa167bb83e4fcd936b887b708f4e57fe75911c02aebf53efaf8d938e", size = 46881, upload-time = "2026-06-04T16:18:58.647Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/45/58/38b5afbc1a800eeea951b9285d3912613f2603bdf897a4ab0f4bd7f405fc/python_multipart-0.0.20-py3-none-any.whl", hash = "sha256:8a62d3a8335e06589fe01f2a3e178cdcc632f3fbe0d492ad9ee0ec35aab1f104", size = 24546, upload-time = "2024-12-16T19:45:44.423Z" },
{ url = "https://files.pythonhosted.org/packages/e1/04/e8135ebd1ad02c56ec633277529b2602ff99ff634be76cdba5744cf554fd/python_multipart-0.0.32-py3-none-any.whl", hash = "sha256:ff6d3f776f16878c894e52e107296ffc890e913c611b1a4ec6c44e2821fe2e23", size = 30042, upload-time = "2026-06-04T16:18:57.319Z" },
]
[[package]]
@@ -2384,7 +2435,7 @@ wheels = [
[[package]]
name = "routstr"
version = "0.4.3"
version = "0.4.4"
source = { editable = "." }
dependencies = [
{ name = "aiosqlite" },
@@ -2908,11 +2959,11 @@ wheels = [
[[package]]
name = "urllib3"
version = "2.6.3"
version = "2.7.0"
source = { registry = "https://pypi.org/simple" }
sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" }
sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" },
{ url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" },
]
[[package]]