Compare commits

...

61 Commits

Author SHA1 Message Date
9qeklajc
e19f679609 Merge pull request #622 from Routstr/fix/retry-cashu-storage-writes
fix: retry critical Cashu storage writes
2026-07-22 00:39:21 +02:00
9qeklajc
a3a4d69ed3 fix: make storage retries idempotent 2026-07-18 14:33:40 +02:00
9qeklajc
c9533c872a fix: retry critical Cashu storage writes 2026-07-18 14:28:18 +02:00
9qeklajc
90da3803c6 fix: preserve caller recovery on storage errors 2026-07-18 14:22:16 +02:00
9qeklajc
8b3b59e176 fix: propagate Cashu transaction storage errors 2026-07-18 14:16:30 +02:00
9qeklajc
b3bf1f0e90 Merge pull request #613 from Routstr/fix-overflowing-in-mobile
fix overflow
2026-07-17 21:51:38 +02:00
9qeklajc
3035292d2a Merge pull request #575 from Routstr/ehbp-proxy-refactor
EHBP proxy support, Tinfoil direct integration, rate limiting, wallet fixes and more
2026-07-17 20:22:14 +02:00
9qeklajc
ac436d0862 Merge pull request #618 from jeroenubbink/config/docker-compose-restart-unless-stopped
config: Ensure docker compose services are restarted unless explicitly stopped
2026-07-17 20:21:34 +02:00
Jeroen Ubbink
19082231f9 config: Ensure docker compose services are restarted unless explicitly stopped 2026-07-17 14:14:27 +02:00
9qeklajc
281607108c fix(ui): use dynamic viewport heights 2026-07-16 13:25:44 +02:00
9qeklajc
8314f3c1b0 Merge pull request #614 from Routstr/pr-575-review-fixes
Fix review blockers: hop-by-hop headers, exact attestation routing, doc updates
2026-07-16 13:18:56 +02:00
9qeklajc
1a9041766b Merge pull request #616 from Routstr/fix/ppq-upstream-inference-cost
fix: bill PPQ.AI BYOK upstream_inference_cost + BYOK fee
2026-07-16 13:16:30 +02:00
redshift
01c01fe8ad fix: bill PPQ.AI BYOK upstream_inference_cost + BYOK fee
PPQ.AI (BYOK) requests were billed at ~5% of their true cost because
_resolve_usd_cost fell through to usage.cost (a small BYOK routing fee)
instead of using cost_details.upstream_inference_cost (the real inference
cost). The proxy operator absorbed the inference cost.

The fix adds a BYOK-specific branch in _resolve_usd_cost: when is_byok is
true and cost_details.upstream_inference_cost is present, bill
upstream_inference_cost + byok_fee — what PPQ actually deducts from the
balance. Non-BYOK providers (e.g. OpenRouter) are unaffected because their
usage.cost already equals upstream_inference_cost.

Regression tests mirror the live glm-5.2-fast request from GitHub issue #615,
asserting the corrected billing (940,274 msats vs the old 45,202 msats — a
20.8× undercharge).

Closes #615
2026-07-16 17:01:55 +08:00
redshift
892aed61cc Fix mypy: move type: ignore onto ASGITransport line 2026-07-16 16:43:52 +08:00
9qeklajc
c6733dbb62 fix overflow 2026-07-14 21:47:26 +02:00
redshift
57fef44ce7 Merge branch 'main' of https://github.com/Routstr/routstr-core into ehbp-proxy-refactor 2026-07-13 11:14:15 +08:00
9qeklajc
c671d277d3 Merge pull request #603 from Routstr/refund-sweep-behavior-tests
test: exercise refund sweep state transitions
2026-07-13 00:09:30 +02:00
9qeklajc
9460e24f21 fix: persist outbound Cashu tokens before sending 2026-07-13 00:07:00 +02:00
9qeklajc
f31929b538 Merge pull request #610 from Routstr/fix-mypy
fix mypy
2026-07-12 23:35:35 +02:00
9qeklajc
129ea7bb76 fix mypy 2026-07-12 23:33:22 +02:00
9qeklajc
b88c6d84fa Merge pull request #608 from Routstr/fix/refund-sweep-test-coverage
test: cover periodic refund sweep behavior
2026-07-12 23:06:51 +02:00
9qeklajc
238beb3e52 Merge pull request #605 from Routstr/fix/admin-withdraw-transaction-audit
fix: record admin withdrawals as outgoing transactions
2026-07-12 15:05:20 +02:00
9qeklajc
dc833d9f5f Merge pull request #602 from Routstr/fix/auto-topup-transaction-recovery
fix: make auto-topup tokens recoverable
2026-07-12 14:57:03 +02:00
9qeklajc
31e7ff8fbd test: cover periodic refund sweep behavior 2026-07-12 14:54:26 +02:00
9qeklajc
a1850dfa55 fix: record admin withdrawals as outgoing transactions 2026-07-12 14:49:51 +02:00
9qeklajc
110ec5da5d test: exercise refund sweep state transitions 2026-07-12 14:46:19 +02:00
9qeklajc
a5cee796c9 fix: make auto-topup tokens recoverable 2026-07-12 14:45:17 +02:00
redshift
b81c5add6a test: satisfy mypy usage parser assertions 2026-07-10 23:40:37 +08:00
redshift
02109616b9 fix(ehbp): compare resolved upstream model identities 2026-07-10 15:16:21 +08:00
redshift
b30346bcb6 chore: untrack stray local files & gitignore
Remove accidentally-committed local-only files:
- .wallet/wallet.sqlite3-shm / .wal (runtime DB sidecar files)
- AGENTS.md, TEST_SUITE_OVERVIEW.md (local agent context artifacts)

Add .wallet/, AGENTS.md, TEST_SUITE_OVERVIEW.md to .gitignore.
2026-07-10 10:17:22 +05:30
redshift
956a1ac3e1 fix(ehbp): harden model comparison against casing & date-versioned aliases
- Case-insensitive comparison of served model vs forwarded_model_id
  (matches get_model_instance's lowercasing semantics)
- Suppress spurious mismatch when get_model_instance resolves back to
  the same model (e.g. date-versioned glm-5-2-20260415 -> glm-5-2)
- Preserve unknown-model warning only when lookup returns None
- Tests: case-insensitive match + date-versioned alias resolution
2026-07-10 09:58:35 +05:30
redshift
4287f038cf feat(ehbp): extract & use model name from Tinfoil usage metrics header
Tinfoil PR #385 added model=<name> to the X-Tinfoil-Usage-Metrics
header/trailer.  This commit uses that field for accurate billing.

Changes in routstr/upstream/ehbp.py:

- parse_tinfoil_usage_metrics(): extract the model= field as a string
  alongside the existing token counts (previously silently discarded
  because int() failed on it).

- _build_cost_info(): accept optional actual_model parameter propagated
  through to callers when a real mismatch is detected.

- _compute_ehbp_actual_cost(): compare the served model against
  model_obj.forwarded_model_id (the expected upstream ID) rather than
  model_obj.id (the client-facing alias).  This prevents spurious
  mismatches when a node runner maps e.g. tinfoil-glm-5-2 -> glm-5-2
  and the header correctly reports glm-5-2.  On a genuine mismatch
  (failover to a different upstream model), look up the actual model's
  pricing via get_model_instance() (forwarded_model_id values are
  registered as routable aliases in the global model map).

- forward_ehbp_request() / forward_ehbp_x_cashu_request(): use the
  actual served model for payment finalization and logging when a
  mismatch is detected.

Tests: 6 new scenarios (alias match, real mismatch with alias, unknown
model fallback, old-format compat, cache token details + model), plus
forwarded_model_id set on all existing mock model objects to keep them
passing.  All 49 Tinfoil/EHBP unit tests pass.
2026-07-10 09:57:10 +05:30
redshift
82fd2c08a7 Merge branch 'main' of https://github.com/Routstr/routstr-core into ehbp-proxy-refactor 2026-07-10 12:12:51 +08:00
redshift
0f7d3d2f86 fix(wallet): return 425 when refund is pending, not 404
The X-Cashu refund endpoint raised 404 "Refund not found" when the
"in" transaction existed with a request_id but the "out" (refund)
transaction had not been written yet. This is a timing race: the
endpoint is polled while the upstream request is still in flight,
before send_refund() has minted and stored the refund token.

The 404 was indistinguishable from a genuinely-missing refund, so
clients had no signal that retrying would succeed — leading to
stranded refunds when clients gave up.

Replace the third 404 branch with 425 Too Early + Retry-After: 2.
The two earlier 404 branches (no "in" row, no request_id) remain 404
since those genuinely mean no refund will ever exist.

Also adds debug logging on all three not-found/pending branches so
the race is no longer invisible to operators (middleware does not log
request headers).

Adds unit tests for the new 425 pending path and the no-request_id
404 path.

Refs: refund-race-condition
2026-07-07 22:00:21 +08:00
redshift
af2abc3a1c Rename TinfoilUpstreamProvider.from_db_row to _build_from_row
Aligns Tinfoil with the base class hook pattern: subclasses override
_build_from_row so the base from_db_row wrapper stamps db_id onto
the instance. Previously Tinfoil overrode from_db_row directly,
bypassing the identity-stamping wrapper.
2026-07-07 21:15:34 +08:00
redshift
bc6d0af6d6 Merge branch 'main' of github.com-red:Routstr/routstr-core into ehbp-proxy-refactor 2026-07-07 13:47:06 +05:30
redshift
65ac1b0dcd remove test for .well-known/ bypass (path no longer routed)
The .well-known/ path was removed from Tinfoil attestation routing
since Tinfoil doesn't use it. The test that asserted GET /.well-known/
bypasses model lookup is no longer valid.
2026-07-02 20:15:00 +05:30
redshift
ab5596ed70 remove .well-known/ from Tinfoil attestation routing
Tinfoil's proxy server guide only requires /attestation (GET) and
/v1/chat/completions + /v1/responses (POST). The .well-known/ path was
never requested by Tinfoil and was incorrectly added to:
- _API_PATH_PREFIXES (prefix gate)
- the unauthenticated GET bypass branch
- the Tinfoil integration docs

Remove it from all three.
2026-07-02 20:09:58 +05:30
redshift
55622cb956 Merge pull request #580 from Routstr/refactor-pr575-review
Reviewing the changes
2026-07-02 14:37:16 +00:00
9qeklajc
7328a5ac45 Address EHBP review comments 2026-07-01 22:41:14 +02:00
9qeklajc
07d39c2a7b simplify 2026-06-30 23:50:25 +02:00
redshift
bd4f4e8207 fix: expose Ehbp-Response-Nonce and Ehbp-Encapsulated-Key in CORS
Browser clients need these EHBP protocol headers visible to JavaScript
so the Tinfoil SDK can detect and decrypt encrypted responses. Without
them, CORS hides the headers, the SDK treats the response as a plaintext
proxy error, and users see 'The provider did not respond to this request.'

Node.js scripts are unaffected (no CORS enforcement).
See ../routstr-chat/TINFOIL_CORS_ISSUE.md for full root-cause analysis.
2026-06-29 16:33:25 +08:00
redshift
46b1f8f72d chore: remove secp256k1 git source pin 2026-06-29 13:51:31 +08:00
redshift
1c0570cc0e Merge branch 'main' of github.com-red:Routstr/routstr-core into ehbp-proxy-refactor 2026-06-29 13:33:56 +08:00
redshift
a8f60643e0 feat: return per-request cost headers on EHBP/Tinfoil responses
EHBP response bodies are opaque encrypted blobs, so cost cannot be injected
into JSON like the normal proxy flow. Instead, surface cost as response headers:

  X-Routstr-Cost-Msats       — total msats charged (bearer + x-cashu)
  X-Routstr-Cost-Usd         — USD equivalent (bearer only)
  X-Routstr-Input-Cost-Msats — msats attributed to input tokens
  X-Routstr-Output-Cost-Msats— msats attributed to output tokens

- Refactor _compute_ehbp_actual_cost from int→dict to carry full cost breakdown
- Add _build_cost_info() and _inject_cost_response_headers() helpers
- Capture adjust_payment_for_tokens return in bearer path (was discarded)
- Update CORS expose_headers, docs, and unit tests
2026-06-23 20:42:01 +08:00
redshift
e4753b5864 fix: stream buffered EHBP body instead of returning fixed-length Response
The Tinfoil SDK expects chunked transfer encoding for streaming
responses. Returning Response with content=body caused the stream to
terminate abruptly instead of ending gracefully. Use StreamingResponse
with a generator that yields the buffered body so the client gets
proper chunked transfer encoding while still benefiting from trailer
capture and exact billing.
2026-06-23 15:00:41 +08:00
redshift
66d7bd87b5 fix: capture HTTP trailers for EHBP streaming usage metrics
Tinfoil returns X-Tinfoil-Usage-Metrics as an HTTP trailer on streaming
responses, but httpx/httpcore silently discard trailers during chunked
transfer decoding. This caused all streaming EHBP requests to fall back
to max-cost billing instead of charging actual token usage.

- Add routstr/upstream/tinfoil_trailer.py: h11-based HTTP client that
  preserves trailers via the EndOfMessage event (httpx discards them)
- Rewrite forward_ehbp_request and forward_ehbp_x_cashu_request to use
  forward_with_trailer instead of httpx
- Add _extract_usage_from_response() to check both response headers
  (non-streaming) and trailers (streaming) for usage metrics
- Buffer EHBP response bodies (acceptable since they are opaque
  encrypted blobs the client decrypts regardless)
- Both bearer and X-Cashu paths now bill based on actual token usage
  for both streaming and non-streaming requests
2026-06-23 14:44:57 +08:00
redshift
4c48df8aa8 fix: remove unused Field import after mypy fix 2026-06-22 21:08:57 +08:00
redshift
abc1ea5b35 fix: resolve all mypy errors (Field defaults, missing return types, ASGITransport arg-type) 2026-06-22 21:04:51 +08:00
redshift
52dd011cd8 fix: add TYPE_CHECKING import for EHBPForwardingTarget (ruff F821) 2026-06-22 20:55:05 +08:00
redshift
94a3215894 docs: fix forward_get_request docstring for Tinfoil
The docstring claimed X-Tinfoil-Enclave-Url is honored for GET requests,
but the implementation delegates to the base class which builds URLs from
self.base_url. X-Tinfoil-Enclave-Url is an EHBP-only header for encrypted
POST requests and is not used for unencrypted GETs.
2026-06-22 16:37:13 +08:00
redshift
5dd3cbbc22 fix: route Tinfoil attestation directly 2026-06-22 13:40:32 +08:00
redshift
55fc25b4de fix: preserve provider EHBP target headers 2026-06-22 12:08:03 +08:00
redshift
36ed3ec9f0 fix: validate Tinfoil EHBP enclave URL overrides 2026-06-22 12:04:06 +08:00
redshift
2e350e082b feat: add Tinfoil direct blind-upstream integration
Add TinfoilUpstreamProvider that uses inference.tinfoil.sh as a direct
EHBP upstream. Routstr acts as a blind relay: it forwards the opaque
encrypted body to the Tinfoil enclave without ever seeing plaintext,
and bills from the X-Tinfoil-Usage-Metrics response header.

- New routstr/upstream/tinfoil.py: fetches models from public
  GET /v1/models, parses Tinfoil pricing into standard Model/Pricing
  schema, supports_ehbp=True, proxies /attestation to atc.tinfoil.sh
- Updated routstr/upstream/ehbp.py:
  - parse_tinfoil_usage_metrics() parses prompt=N,completion=N header
  - _resolve_ehbp_target_url() honors X-Tinfoil-Enclave-Url from SDK
  - _strip_proxy_headers() removes proxy-only headers before forwarding
  - _compute_ehbp_actual_cost() converts usage to msats via calculate_cost
  - forward_ehbp_request() finalizes with exact token cost when usage
    header is present (non-streaming), falls back to max-cost otherwise
  - forward_ehbp_x_cashu_request() computes refund from actual cost
    when usage is available
- Updated routstr/proxy.py: forward /attestation and /.well-known/ paths
  without model/cost/auth lookups
- Updated routstr/upstream/__init__.py and helpers.py: register and
  auto-seed Tinfoil provider from TINFOIL_API_KEY env var
- Updated .env.example with TINFOIL_API_KEY
- 22 new unit tests in tests/unit/test_tinfoil_integration.py
- Updated docs/tinfoil-direct-integration.md and docs/ehbp-proxy-support.md
  with implementation status and billing behavior table
2026-06-21 13:16:01 +08:00
redshift
24b90af6e6 refactor: move EHBP logic to dedicated module with explicit opt-in
- Add supports_ehbp + get_ehbp_forwarding_target hooks to BaseUpstreamProvider,
  keeping base.py minimal (no large forwarding methods)
- Create routstr/upstream/ehbp.py with forward_ehbp_request and
  forward_ehbp_x_cashu_request helpers, plus max-cost finalization
- PPQAIUpstreamProvider: set supports_ehbp = True, implement target to
  /private/v1/... with X-Private-Model header
- proxy.py: filter to EHBP-capable providers, route to ehbp helpers
- Fix bearer EHBP payment: reserve upfront, finalize max cost on success
- Fix X-Cashu EHBP: refund full token on failure, refund excess on success
- Update docs/ehbp-proxy-support.md to reflect new architecture
2026-06-20 17:18:53 +08:00
redshift
b9bf0ea23b Merge branch 'main' of https://github.com/Routstr/routstr-core 2026-06-20 11:07:05 +08:00
redshift
11fc826bfb Merge branch 'main' of https://github.com/Routstr/routstr-core 2026-06-15 15:43:55 +08:00
redshift
757d4400af Merge branch 'main' of https://github.com/Routstr/routstr-core 2026-06-12 10:18:22 +08:00
redshift
766d59d666 Merge branch 'main' of https://github.com/Routstr/routstr-core 2026-06-04 16:08:28 +08:00
redshift
98ddd37853 2 weeks 2026-06-04 16:07:23 +08:00
46 changed files with 4630 additions and 100 deletions

View File

@@ -2,6 +2,9 @@
UPSTREAM_BASE_URL=https://api.openai.com/v1
UPSTREAM_API_KEY=your-upstream-api-key
# Tinfoil (confidential inference enclaves, EHBP)
# TINFOIL_API_KEY=your-tinfoil-api-key
# ADMIN_PASSWORD=secure-admin-password
# Database

4
.gitignore vendored
View File

@@ -16,6 +16,9 @@ dist/
*.db-shm
*.db-wal
.*wallet.sqlite3
.wallet/
AGENTS.md
TEST_SUITE_OVERVIEW.md
*models.json
.cashu
.relay
@@ -38,3 +41,4 @@ proof_backups
*.todo
ui_out
.worktrees

View File

@@ -13,6 +13,7 @@ services:
- ./ui_out:/output:z
command:
["sh", "-c", "mkdir -p /output && cp -r /app/built/. /output/ && echo 'UI build copied to mounted volume' && ls -la /output/ && echo 'UI built and ready' && tail -f /dev/null"]
restart: unless-stopped
routstr:
build: .
@@ -31,6 +32,7 @@ services:
- 8000:8000
extra_hosts: # Needed to access locally running models
- "host.docker.internal:host-gateway"
restart: unless-stopped
tor:
image: ghcr.io/hundehausen/tor-hidden-service:latest
@@ -41,6 +43,7 @@ services:
- HS_ROUTER=routstr:8000:80
depends_on:
- routstr
restart: unless-stopped
volumes:
tor-data:

150
docs/ehbp-proxy-support.md Normal file
View File

@@ -0,0 +1,150 @@
# EHBP Proxy Support for Tinfoil Models
## Problem
The SDK's `SecureClient.fetch` encrypts request bodies with HPKE (EHBP protocol)
and sends them to the Routstr provider. The Routstr proxy had no EHBP handling:
1. It tried to `json.loads()` the binary HPKE-sealed body → failed with a 400
2. The upstream PPQ.AI public endpoint (`/v1/chat/completions`) doesn't speak
EHBP, so the response had no `Ehbp-Response-Nonce` header
3. `SecureClient` threw `Missing Ehbp-Response-Nonce header` because it expects
every response from an EHBP-configured `baseURL` to carry that header
## Root cause
PPQ.AI exposes EHBP-aware inference at `/private/`, separate from the public
`/v1/` endpoint. The Routstr proxy was forwarding to `/v1/` (the public
endpoint) instead of `/private/` (the enclave endpoint). The public endpoint
can't decrypt the body, returns a normal HTTP response, and the SDK can't
decrypt it because there's no nonce header.
The PPQ private-mode proxy (`ppq-private-mode-proxy/lib/proxy.ts`) shows the
correct pattern: `SecureClient` talks to `api.ppq.ai/private/v1/chat/completions`,
which decrypts inside the attested enclave and returns an EHBP-encrypted
response with the `Ehbp-Response-Nonce` header.
## What was changed
### `routstr/proxy.py`
Detects EHBP requests by checking for the `Ehbp-Encapsulated-Key` header (set
by the EHBP transport on every encrypted request). For EHBP requests:
- Skips JSON body parsing (the body is binary ciphertext, not JSON)
- Reads the model ID from the `X-Routstr-Model` header (set by the SDK) instead
of from `body.model`
- Routes through new `forward_ehbp_request` (bearer auth) and
`forward_ehbp_x_cashu_request` (x-cashu auth) methods
- Skips reactive 400 param correction (can't parse encrypted response body)
- Still charges the user via `pay_for_request` (uses `max_cost_for_model` from
the model registry, not the body)
### `routstr/upstream/base.py`
Keeps EHBP as an explicit opt-in provider capability instead of making every
upstream provider appear EHBP-capable:
- `supports_ehbp = False` by default
- `get_ehbp_forwarding_target(path, model_obj)` raises `NotImplementedError`
unless a provider opts in and returns a provider-specific EHBP target
The actual EHBP forwarding logic does **not** live in `base.py`.
### `routstr/upstream/ehbp.py`
Contains the shared opaque EHBP transport and billing helpers:
- `EHBPForwardingTarget` — provider-specific target URL plus extra headers
- `forward_ehbp_request()` — forwards the encrypted body, captures Tinfoil
usage from a response header or streaming HTTP trailer, and finalizes bearer
billing at actual cost (falling back to max cost when usage is unavailable)
- `forward_ehbp_x_cashu_request()` — redeems the Cashu token, refunds the full
token on upstream failure, and refunds the difference between the redeemed
amount and actual cost (or max cost when usage is unavailable)
### Provider support
EHBP is currently enabled only for `TinfoilUpstreamProvider`. It forwards to
Tinfoil's attested enclave and requests `X-Tinfoil-Usage-Metrics` for billing.
PPQ.AI retains its private-target implementation, but `supports_ehbp = False`
until it has a provider-specific trusted usage/model-binding strategy.
## Why it's done this way
The proxy is a **blind relay** for EHBP requests. It cannot decrypt the body
(only the attested enclave can), so it must:
1. Get the model ID from a header, not the body
2. Forward the raw bytes without parsing or transformation
3. Stream the response back without SSE/cost parsing
4. Pass through EHBP protocol headers (`Ehbp-Encapsulated-Key` on request,
`Ehbp-Response-Nonce` on response)
Cost tracking happens at the proxy level. Routstr reserves or redeems up to
`max_cost_for_model`, then Tinfoil's out-of-band usage header/trailer allows it
to finalize at actual token cost. If trusted usage is missing or invalid, the
proxy safely falls back to max-cost billing.
## End-to-end flow
```
SDK Routstr Proxy PPQ.AI /private/
│ │ │
│── X-Routstr-Model: tinfoil-kimi-k2-6 ─│ │
│── Ehbp-Encapsulated-Key: <hex> ───────│ │
│── Authorization: Bearer <cashu> ──────│ │
│── body = HPKE-encrypted(kimi-k2-6) ───│ │
│ │ │
│ detects Ehbp-Encapsulated-Key │
│ reads model from X-Routstr-Model │
│ does billing/routing │
│ │ │
│ adds X-Private-Model: private/kimi-k2-6
│ forwards raw body to /private/v1/... │
│ │──────────────────────────────▶│
│ │ enclave decrypts
│ │ runs inference
│ │◀── Ehbp-Response-Nonce ──────│
│ │◀── encrypted response ────────│
│ │ │
│ streams response back untouched │
│◀── encrypted response ────────────────│ │
│ │ │
SecureClient reads nonce, decrypts │ │
SDK SSE processing sees plaintext │ │
```
## Model ID mapping
Three parties see three different model IDs:
| Party | Header/Body | Value | Source |
|---|---|---|---|
| Routstr proxy | `X-Routstr-Model` header | `tinfoil-kimi-k2-6` | SDK sends full caller-facing id |
| Tinfoil usage metrics | `model` field | `kimi-k2-6` | Enclave reports the model actually served |
| Tinfoil enclave | `body.model` (encrypted) | `kimi-k2-6` | SDK strips `tinfoil-` prefix before encryption |
## Implementation status
A dedicated `TinfoilUpstreamProvider` (`routstr/upstream/tinfoil.py`) now
implements the direct blind-upstream pattern described above. The shared EHBP
helpers in `routstr/upstream/ehbp.py` were extended to:
- Request usage metrics via `X-Tinfoil-Request-Usage-Metrics: true`.
- Parse `X-Tinfoil-Usage-Metrics` from the response header (non-streaming) or
HTTP trailer (streaming).
- Override the forwarding URL with a validated `X-Tinfoil-Enclave-Url` when the
SDK sends it.
- Finalize bearer billing with the dedicated EHBP actual-cost finalizer.
- Compute X-Cashu refunds from actual cost instead of max cost.
See `docs/tinfoil-direct-integration.md` for the full implementation notes.
## Verification status
Unit coverage includes usage parsing, target validation, HTTP trailer capture,
response-size limits, and bearer payment finalization. End-to-end requests have
verified both non-streaming usage headers and streaming usage trailers against
Tinfoil. SDK behavior on proxy-generated non-2xx responses without an
`Ehbp-Response-Nonce` still merits explicit end-to-end coverage.

View File

@@ -0,0 +1,493 @@
# Tinfoil / PPQ Private-Mode Integration Notes
This document summarizes the current options for integrating Tinfoil/PPQ private models with Routstr, based on the EHBP work in this branch and local testing against `ppq-private-mode-proxy`.
## Background
PPQ private models run behind a Tinfoil/EHBP flow:
- Request bodies are HPKE-encrypted by a Tinfoil client.
- The PPQ `/private/` endpoint routes ciphertext to the attested enclave.
- The enclave decrypts, runs inference, and returns an encrypted response.
- The caller's Tinfoil client decrypts the response locally.
The PPQ private-mode proxy (`~/projects/ppq-private-mode-proxy`) uses this pattern with the JavaScript `tinfoil` SDK:
```ts
import { SecureClient } from "tinfoil";
const apiBase = "https://api.ppq.ai";
const client = new SecureClient({
baseURL: `${apiBase}/private/`,
attestationBundleURL: `${apiBase}/private`,
transport: "ehbp",
});
await client.ready();
const response = await client.fetch(`${apiBase}/private/v1/chat/completions`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${process.env.PPQ_API_KEY}`,
"X-Private-Model": "private/gpt-oss-120b",
"x-query-source": "api",
},
body: JSON.stringify({
model: "gpt-oss-120b", // enclave-internal model id
messages: [{ role: "user", content: "Hello" }],
}),
});
const json = await response.json();
console.log(json.usage);
```
`X-Private-Model` carries the PPQ-facing private model id, while the encrypted JSON body uses the enclave-internal model id without the `private/` prefix.
## PPQ private model pricing
PPQ exposes private model pricing through:
```text
GET https://api.ppq.ai/v1/models?type=all
```
Filter models whose IDs start with `private/`.
Example private pricing observed:
| Model | Input USD / 1M tokens | Output USD / 1M tokens |
|---|---:|---:|
| `private/gpt-oss-120b` | `0.79125` | `1.31875` |
| `private/llama3-3-70b` | `1.84625` | `2.90125` |
| `private/qwen3-vl-30b` | `1.31875` | `4.22` |
| `private/glm-5-2` | `1.5825` | `5.53875` |
| `private/gemma4-31b` | `0.47475` | `1.055` |
| `private/kimi-k2-6` | `1.5825` | `5.53875` |
The `ppq-private-mode-proxy` commit `ba984214793d3bca0f7d046b6955d42abc1c6843` changed OpenClaw display metadata to align with a 5% API margin. The live PPQ model endpoint returns the more precise rates above, which already include that margin.
Actual PPQ private billing is:
```text
price_usd =
input_tokens * input_per_1M_tokens / 1_000_000
+ output_tokens * output_per_1M_tokens / 1_000_000
```
A test request to `private/gpt-oss-120b` produced:
```json
{
"input_count": 74,
"output_count": 3,
"price_in_usd": 0.00006250875
}
```
which exactly matches:
```text
74 * 0.79125 / 1_000_000 + 3 * 1.31875 / 1_000_000
= 0.00006250875 USD
```
## Integration architectures
There are three materially different ways Routstr could integrate Tinfoil/PPQ private inference.
## Option A: User integrates Tinfoil directly
```text
User app / SDK
-> Tinfoil SecureClient / TinfoilAI
-> EHBP-encrypted request
-> Tinfoil/PPQ private enclave
```
Properties:
- Best privacy for the user.
- Routstr is not in the request path.
- User's client encrypts requests and decrypts responses.
- Usage is visible to the user's app after decryption.
- Billing is handled directly by Tinfoil/PPQ.
Example with Tinfoil's OpenAI-compatible client:
```ts
import { TinfoilAI } from "tinfoil";
const client = new TinfoilAI({
apiKey: process.env.TINFOIL_API_KEY,
transport: "ehbp",
});
const res = await client.chat.completions.create({
model: "llama3-3-70b",
messages: [{ role: "user", content: "Hello" }],
});
console.log(res.choices[0].message.content);
console.log(res.usage);
```
This is not a Routstr marketplace flow unless Routstr only acts as discovery/UI around direct Tinfoil/PPQ usage.
## Option B: Routstr integrates Tinfoil as an upstream client
```text
User -> Routstr plaintext request
-> Routstr Tinfoil SecureClient encrypts to PPQ/Tinfoil
-> PPQ private enclave
-> Routstr receives decrypted response
-> Routstr bills from decrypted usage
-> Routstr returns plaintext response to user
```
Properties:
- Easier exact billing.
- Routstr can read the decrypted OpenAI response and `usage` object.
- Routstr can charge exact PPQ token pricing.
- Privacy is different: the user sends plaintext to Routstr, and Routstr sees prompts/responses.
- End-to-end encryption is only Routstr-to-enclave, not user-to-enclave.
This should be considered a separate product/provider mode, not the same as an end-to-end private relay.
Practical implementation options:
1. Run a Node sidecar that uses the `tinfoil` npm package and expose it as a local HTTP upstream to Routstr.
2. Port EHBP client behavior to Python.
3. Reuse or adapt `ppq-private-mode-proxy` as a local upstream.
A Node sidecar is probably the quickest implementation path because `ppq-private-mode-proxy` already demonstrates the full flow.
## Option C: Routstr uses Tinfoil as a direct blind upstream
This is the current branch's design intent and is likely the best fit if Tinfoil/PPQ exposes usage metadata headers:
```text
User Tinfoil SecureClient
-> encrypted request body
-> Routstr proxy
-> Tinfoil/PPQ private API
with X-Tinfoil-Request-Usage-Metrics: true
-> PPQ private enclave
<- encrypted response
plus X-Tinfoil-Usage-Metrics / cost headers
<- Routstr proxy
-> user decrypts response
```
In this mode Routstr is still a normal upstream proxy from the user's point of view, but the upstream is Tinfoil/PPQ private inference and the body remains opaque to Routstr.
Properties:
- Strongest privacy with Routstr in the path.
- Routstr never sees plaintext prompt or plaintext response.
- Routstr can authenticate and route based on plaintext headers.
- Routstr should request Tinfoil usage metadata by adding `X-Tinfoil-Request-Usage-Metrics: true` to the upstream request.
- Tinfoil documents `X-Tinfoil-Usage-Metrics` as an upstream response header for non-streaming requests.
- For streaming requests, Tinfoil documents `X-Tinfoil-Usage-Metrics` as an HTTP trailer available only after the response body completes.
- If Tinfoil/PPQ returns `X-Tinfoil-Usage-Metrics` or a cost header on the encrypted response, Routstr can bill exactly without decrypting the body.
- If usage is only present inside the encrypted response body, Routstr still cannot read it and exact billing is not possible without a separate metadata path.
This is the only architecture that preserves end-to-end encryption from the user to the PPQ/Tinfoil enclave while still letting Routstr mediate payment. The key requirement is that usage/cost metadata must be returned outside the encrypted body, ideally as a response header available before body streaming begins.
## Current Routstr problem
The current EHBP implementation charges successful EHBP requests at `max_cost_for_model` because Routstr cannot decrypt the response body:
```text
successful EHBP request -> charge full reserved max cost
```
That is incorrect for PPQ private models. Max cost should be only a reservation/solvency ceiling. Final charge should use actual PPQ private token pricing.
Desired behavior:
```text
reserve max cost
forward encrypted request
obtain actual usage/cost metadata
finalize actual cost
refund/release the difference
```
## Usage/cost metadata requirement
For blind-relay exact billing, PPQ should return one of the following outside the encrypted response body:
```http
X-PPQ-Cost-USD: 0.00006250875
```
or:
```http
X-Private-Usage-Metrics: input=74,output=3
```
or:
```http
X-Tinfoil-Usage-Metrics: prompt=74,completion=3,total=77
```
Tinfoil proxy documentation references a usage-metrics flow where a proxy can request usage via:
```http
X-Tinfoil-Request-Usage-Metrics: true
```
and read usage from:
```http
X-Tinfoil-Usage-Metrics
```
Docs/example references:
- https://docs.tinfoil.sh/guides/proxy-server
- https://github.com/tinfoilsh/encrypted-request-proxy-example
During local PPQ testing, PPQ responses included this CORS exposure header:
```http
Access-Control-Expose-Headers: Ehbp-Response-Nonce, X-Private-Usage-Metrics, X-Encrypted-Usage-Metrics, X-Tinfoil-Usage-Metrics
```
However, the actual tested non-streaming response did not include any of these usage headers, even when `X-Tinfoil-Request-Usage-Metrics: true` was sent.
The decrypted body did include normal OpenAI usage, but only the decrypting Tinfoil client can see that body.
## Query-history fallback
PPQ's query history endpoint exposes actual usage and cost:
```text
GET https://api.ppq.ai/queries/history?page=1&page_count=...
```
A record includes:
```json
{
"timestamp": "...",
"model": "private/gpt-oss-120b",
"input_count": 74,
"output_count": 3,
"price_in_usd": 0.00006250875,
"query_type": "chat_completion",
"query_source": "api"
}
```
This could be used as a fallback, but it is less robust than response headers/trailers because matching a request to a history row can be race-prone under concurrency. It would need a reliable request identifier or metadata field that PPQ stores in history.
## Recommended Routstr direction
Use Tinfoil/PPQ as a direct blind upstream and have Routstr explicitly request usage metadata:
```text
User encrypts body
Routstr reserves max cost
Routstr forwards encrypted body to Tinfoil/PPQ /private/
Routstr includes X-Tinfoil-Request-Usage-Metrics: true
Tinfoil/PPQ returns X-Tinfoil-Usage-Metrics as a response header for non-streaming,
or as an HTTP trailer after the body completes for streaming
Routstr finalizes exact charge
User decrypts encrypted response
```
This preserves both:
- privacy: Routstr cannot read prompts/responses;
- exact billing: Routstr can charge actual PPQ private model cost, assuming Tinfoil/PPQ returns usage/cost metadata outside the encrypted body.
Implementation steps:
1. Update PPQ model fetching to include private models:
```text
GET https://api.ppq.ai/v1/models?type=all
```
2. Register `private/*` models and any Routstr-facing aliases with correct `forwarded_model_id`.
3. Keep max-cost reservation for bearer keys and X-Cashu solvency checks.
4. Add parsing support for possible usage/cost headers:
```http
X-PPQ-Cost-USD
X-Private-Usage-Metrics
X-Tinfoil-Usage-Metrics
X-Encrypted-Usage-Metrics
```
5. Finalize by actual cost instead of max cost:
```text
actual_msats = ceil((actual_usd / sats_usd_price()) * 1000)
actual_msats = max(actual_msats, settings.min_request_msat)
actual_msats = min(actual_msats, reserved_msats)
```
or, if only token counts are available:
```text
actual_usd =
input_tokens * input_per_1M_tokens / 1_000_000
+ output_tokens * output_per_1M_tokens / 1_000_000
```
6. If usage/cost metadata is missing, choose an explicit policy:
- fail closed and refund/revert;
- query PPQ history as a fallback;
- fallback to max-cost billing only if explicitly configured and clearly disclosed.
Silent max-cost billing should not be the default for PPQ private requests.
## X-Cashu consideration
For bearer-auth requests, finalization can happen after the response stream completes if usage is delivered as a trailer.
For `X-Cashu`, Routstr needs to return the refund token in the response headers. If usage/cost is only available after consuming the encrypted response stream, Routstr may need to buffer EHBP responses before sending them to the client so it can compute the refund amount first.
Possible approaches:
1. Prefer a non-trailer response header with actual cost, available before streaming body starts.
2. Buffer EHBP X-Cashu responses and then return `X-Cashu` refund.
3. Introduce a later/refund-claim mechanism, which would be a larger protocol change.
## Summary
- PPQ private models are billed per actual input/output tokens.
- Private model rates are available from `GET /v1/models?type=all`.
- Current Routstr EHBP billing at max cost is wrong for PPQ private models.
- Direct Tinfoil integration inside Routstr would enable exact usage billing but would make Routstr see plaintext.
- A blind EHBP relay preserves privacy but requires PPQ/Tinfoil to expose usage/cost in plaintext headers/trailers.
- The preferred solution is to keep Routstr blind and have PPQ return billing metadata outside the encrypted body.
## Implementation status
Direct Tinfoil upstream integration is implemented in `routstr/upstream/tinfoil.py`
and `routstr/upstream/ehbp.py`.
### What was built
- `TinfoilUpstreamProvider` (`provider_type = "tinfoil"`):
- Base URL: `https://inference.tinfoil.sh`
- Fetches models from the public `GET /v1/models` endpoint (no auth needed).
- Parses Tinfoil's pricing (`inputTokenPricePer1M`, `outputTokenPricePer1M`,
`requestPrice`) into the standard `Model`/`Pricing` schema.
- `supports_ehbp = True` — acts as a blind EHBP relay.
- `get_ehbp_forwarding_target()` returns a target that includes
`X-Tinfoil-Request-Usage-Metrics: true`.
- `forward_get_request()` proxies `/attestation` to `https://atc.tinfoil.sh/attestation`
so the SDK can fetch attestation bundles through Routstr.
- Registered in `routstr/upstream/__init__.py` and seeded from
`TINFOIL_API_KEY` env var.
- `routstr/upstream/ehbp.py`:
- `parse_tinfoil_usage_metrics()` parses
`prompt=N,completion=N[,total=N][,model=<name>]` into an OpenAI-style
usage dict. The `model` field (added in tinfoilsh/confidential-model-router
PR #385) is extracted as a string.
- `_resolve_ehbp_target_url()` overrides the forwarding URL with
`X-Tinfoil-Enclave-Url` when the SDK sends it.
- `_strip_proxy_headers()` removes `X-Routstr-Model`,
`X-Tinfoil-Enclave-Url`, and `X-Tinfoil-Request-Usage-Metrics` before
forwarding to the enclave.
- `_compute_ehbp_actual_cost()` converts the usage header into msats via
`calculate_cost()`, clamped to `[min_request_msat, max_cost_for_model]`.
When the header's `model=<name>` differs from the requested model, the
actual served model's pricing is used for cost calculation.
- `forward_ehbp_request()` (bearer auth): if `X-Tinfoil-Usage-Metrics` is
present in the response header, finalizes with `adjust_payment_for_tokens()`
for exact billing; otherwise falls back to max-cost. Billing uses the
actual served model when it differs from the requested one.
- `forward_ehbp_x_cashu_request()`: if usage is available, computes the
refund from actual cost instead of max cost, using the actual served
model's pricing when applicable.
- `routstr/proxy.py`: `/attestation` and `/tee/attestation` paths are forwarded
to Tinfoil upstreams without model/cost/auth lookups.
### Billing behavior
| Request shape | Usage source | Billing |
|---|---|---|
| Bearer, non-streaming | `X-Tinfoil-Usage-Metrics` response header | Exact token cost via `adjust_payment_for_tokens` |
| Bearer, streaming | `X-Tinfoil-Usage-Metrics` HTTP trailer | Exact token cost (h11 captures trailers) |
| Bearer, no usage header/trailer | N/A | Max-cost fallback |
| X-Cashu, non-streaming | `X-Tinfoil-Usage-Metrics` response header | Refund = `redeemed - actual_cost` |
| X-Cashu, streaming | `X-Tinfoil-Usage-Metrics` HTTP trailer | Refund = `redeemed - actual_cost` (h11 captures trailers) |
| X-Cashu, no usage header/trailer | N/A | Refund = `redeemed - max_cost` |
### Cost response headers
Since EHBP response bodies are opaque encrypted blobs, per-request cost cannot
be injected into the JSON body (as done in the normal proxy flow). Instead,
Routstr returns cost info as response headers:
| Header | Auth | Description |
|---|---|---|
| `X-Routstr-Cost-Msats` | Bearer, X-Cashu | Total msats charged for this request |
| `X-Routstr-Cost-Usd` | Bearer | USD equivalent of the charge |
| `X-Routstr-Input-Cost-Msats` | Bearer, X-Cashu | msats attributed to input tokens |
| `X-Routstr-Output-Cost-Msats` | Bearer, X-Cashu | msats attributed to output tokens |
The client/Tinfoil SDK can read these headers from the HTTP response without
needing to decrypt the body.
### Setup
```bash
TINFOIL_API_KEY=your-tinfoil-api-key
```
The provider is auto-seeded on first startup.
### Usage metrics header format
Tinfoil returns usage metrics in the `X-Tinfoil-Usage-Metrics` response header
(non-streaming) or HTTP trailer (streaming) when `X-Tinfoil-Request-Usage-Metrics:
true` is sent. As of tinfoilsh/confidential-model-router PR #385, the format is:
```
prompt=<prompt_tokens>,completion=<completion_tokens>,total=<total_tokens>,model=<served_model>
```
The `model` field carries the actual model name served by the enclave.
Routstr uses this to:
- Verify the served model matches the expected upstream model. The comparison
uses ``model_obj.forwarded_model_id`` (the actual upstream ID, e.g.
``glm-5-2``) rather than ``model_obj.id`` (the client-facing alias, e.g.
``tinfoil-glm-5-2``), so aliased models don't trigger a spurious mismatch.
- When they genuinely differ (Tinfoil served a different upstream model than
expected), look up the actual served model's pricing and use it for billing.
The reverse lookup uses ``get_model_instance``, which resolves
``forwarded_model_id`` values registered as routable aliases.
- Log the discrepancy for observability.
If the actual model is not found in Routstr's model registry, billing falls
back to the requested model's pricing.
### What still needs verification
- ~~End-to-end test with a real Tinfoil SDK client against a Routstr node with
`TINFOIL_API_KEY` set.~~ Verified: both non-streaming (header) and streaming
(trailer) responses include `model=<name>`.
- Streaming trailer capture is implemented by buffering the encrypted response
in `forward_with_trailer()` and then using the dedicated EHBP payment
finalizers for bearer and X-Cashu requests. This provides actual-cost billing
today, at the cost of full time-to-last-byte latency for streaming responses.
- Whether Tinfoil's `/v1/responses` endpoint also returns usage metrics
headers or trailers.

View File

@@ -10,6 +10,7 @@ dependencies = [
"aiosqlite>=0.20",
"sqlmodel>=0.0.24",
"httpx[socks]>=0.25.2",
"h11>=0.14",
"greenlet>=3.2.1",
"alembic>=1.13",
"python-json-logger>=2.0.0",

View File

@@ -15,7 +15,9 @@ from .core.db import (
AsyncSession,
CashuTransaction,
get_session,
store_cashu_transaction,
)
from .core.db import (
store_cashu_transaction_with_retry as store_cashu_transaction,
)
from .core.logging import get_logger
from .core.settings import settings

View File

@@ -29,6 +29,9 @@ from .db import (
UpstreamProviderRow,
create_session,
)
from .db import (
store_cashu_transaction_with_retry as store_cashu_transaction,
)
from .log_manager import log_manager
from .logging import get_logger
from .provider_slugs import allocate_unique_provider_slug
@@ -411,12 +414,11 @@ async def withdraw(
# Get wallet and check balance
from .settings import settings as global_settings
wallet = await get_wallet(
withdraw_request.mint_url or global_settings.primary_mint, withdraw_request.unit
)
effective_mint = withdraw_request.mint_url or global_settings.primary_mint
wallet = await get_wallet(effective_mint, withdraw_request.unit)
proofs = get_proofs_per_mint_and_unit(
wallet,
withdraw_request.mint_url or global_settings.primary_mint,
effective_mint,
withdraw_request.unit,
not_reserved=True,
)
@@ -432,8 +434,27 @@ async def withdraw(
raise HTTPException(status_code=400, detail="Insufficient wallet balance")
token = await send_token(
withdraw_request.amount, withdraw_request.unit, withdraw_request.mint_url
withdraw_request.amount, withdraw_request.unit, effective_mint
)
try:
await store_cashu_transaction(
token=token,
amount=withdraw_request.amount,
unit=withdraw_request.unit,
mint_url=effective_mint,
typ="out",
collected=False,
source="admin",
)
except Exception:
logger.critical(
"Admin withdrawal token issued without a persisted audit record",
extra={
"amount": withdraw_request.amount,
"unit": withdraw_request.unit,
"mint_url": effective_mint,
},
)
return {"token": token}

View File

@@ -1,3 +1,5 @@
import asyncio
import hashlib
import os
import pathlib
import sqlite3
@@ -10,7 +12,7 @@ from alembic import command
from alembic.config import Config
from alembic.util.exc import CommandError
from sqlalchemy import UniqueConstraint, delete
from sqlalchemy.exc import OperationalError
from sqlalchemy.exc import IntegrityError, OperationalError
from sqlalchemy.ext.asyncio.engine import create_async_engine
from sqlalchemy.orm import aliased
from sqlmodel import Field, Relationship, SQLModel, col, func, select, update
@@ -287,10 +289,13 @@ async def store_cashu_transaction(
created_at: int | None = None,
source: str = "x-cashu",
api_key_hashed_key: str | None = None,
) -> None:
transaction_id: str | None = None,
log_failure: bool = True,
) -> bool:
try:
async with create_session() as session:
tx = CashuTransaction(
id=transaction_id or uuid.uuid4().hex,
token=token,
amount=amount,
unit=unit,
@@ -304,11 +309,93 @@ async def store_cashu_transaction(
)
session.add(tx)
await session.commit()
except Exception as e:
logger.warning(
f"Failed to store cashu transaction: {e} (type={typ})",
extra={"error": str(e), "type": typ},
)
except Exception:
if log_failure:
logger.critical(
"Failed to store Cashu transaction",
extra={"type": typ, "request_id": request_id, "source": source},
exc_info=True,
)
raise
return True
async def _cashu_transaction_exists(transaction_id: str) -> bool:
async with create_session() as session:
return await session.get(CashuTransaction, transaction_id) is not None
async def store_cashu_transaction_with_retry(
token: str,
amount: int,
unit: str,
mint_url: str | None = None,
typ: str = "out",
request_id: str | None = None,
collected: bool = False,
created_at: int | None = None,
source: str = "x-cashu",
api_key_hashed_key: str | None = None,
max_attempts: int = 3,
) -> bool:
"""Retry a critical Cashu transaction write with bounded backoff."""
transaction_id = hashlib.sha256(f"{typ}\0{token}".encode()).hexdigest()
last_error: Exception | None = None
for attempt in range(1, max_attempts + 1):
try:
return await store_cashu_transaction(
token=token,
amount=amount,
unit=unit,
mint_url=mint_url,
typ=typ,
request_id=request_id,
collected=collected,
created_at=created_at,
source=source,
api_key_hashed_key=api_key_hashed_key,
transaction_id=transaction_id,
log_failure=False,
)
except IntegrityError as error:
try:
if await _cashu_transaction_exists(transaction_id):
return True
except Exception as lookup_error:
last_error = lookup_error
else:
last_error = error
except Exception as error:
last_error = error
if last_error is not None:
if attempt == max_attempts:
break
delay = 0.25 * (2 ** (attempt - 1))
logger.warning(
"Cashu transaction storage failed; retrying",
extra={
"type": typ,
"request_id": request_id,
"attempt": attempt,
"max_attempts": max_attempts,
"retry_delay_seconds": delay,
},
)
await asyncio.sleep(delay)
logger.critical(
"Cashu transaction storage failed after bounded retries",
extra={
"type": typ,
"request_id": request_id,
"attempts": max_attempts,
"error": str(last_error),
},
)
if last_error is None:
raise RuntimeError("Cashu transaction storage failed without an exception")
raise last_error
class UpstreamProviderRow(SQLModel, table=True): # type: ignore

View File

@@ -257,7 +257,20 @@ app.add_middleware(
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
expose_headers=["x-routstr-request-id", "x-cashu"],
expose_headers=[
"x-routstr-request-id",
"x-cashu",
"x-routstr-cost-msats",
"x-routstr-cost-usd",
"x-routstr-input-cost-msats",
"x-routstr-output-cost-msats",
# EHBP (Tinfoil) protocol headers must be exposed so browser clients
# can detect and decrypt encrypted responses. Without these, the
# browser hides them via CORS and the SDK treats the response as a
# plaintext proxy error, returning raw ciphertext.
"Ehbp-Response-Nonce",
"Ehbp-Encapsulated-Key",
],
)
# Add logging middleware

View File

@@ -264,7 +264,17 @@ def _coerce_usd(value: object) -> float:
def _resolve_usd_cost(usage_data: dict, response_data: dict) -> float:
"""Resolve USD cost with clear priority order.
Priority: cost_details.total_cost → total_cost → cost (in both usage and response).
Priority:
1. ``cost_details.total_cost``
2. ``cost_details.upstream_inference_cost`` (BYOK — see below)
3. ``total_cost`` → ``cost`` (in both usage and response)
**BYOK path (PPQ.AI):** when ``is_byok`` is true the ``usage.cost`` field
is only a small (~5 %) routing fee, not the inference cost. The real cost
lives in ``cost_details.upstream_inference_cost`` and the provider's
balance is debited by ``upstream_inference_cost + byok_fee``. Billing just
the fee under-charges by ~20×.
"""
cost_details = usage_data.get("cost_details")
if isinstance(cost_details, dict):
@@ -272,6 +282,18 @@ def _resolve_usd_cost(usage_data: dict, response_data: dict) -> float:
if cost > 0:
return cost
# PPQ.AI BYOK: upstream_inference_cost is the real inference cost;
# usage.cost is only a ~5 % BYOK routing fee. Bill the sum — what PPQ
# actually deducts from the balance. For non-BYOK providers (e.g.
# OpenRouter) usage.cost already equals upstream_inference_cost, so we
# fall through to the normal ``cost`` lookup below.
upstream_cost = _coerce_usd(
cost_details.get("upstream_inference_cost")
)
if upstream_cost > 0 and usage_data.get("is_byok"):
byok_fee = _coerce_usd(usage_data.get("cost"))
return upstream_cost + byok_fee
for source in [usage_data, response_data]:
if not isinstance(source, dict):
continue

View File

@@ -29,6 +29,7 @@ from .payment.helpers import (
)
from .payment.models import Model
from .upstream import BaseUpstreamProvider
from .upstream.ehbp import forward_ehbp_request, forward_ehbp_x_cashu_request
from .upstream.helpers import init_upstreams
from .upstream.request_correction import correct_request, extract_error_message
@@ -104,6 +105,34 @@ def get_unique_models() -> list[Model]:
return list(_unique_models.values())
def _is_tinfoil_attestation_path(path: str) -> bool:
"""Return True for exact Tinfoil attestation routes, with optional slash."""
return path in {
"attestation",
"attestation/",
"tee/attestation",
"tee/attestation/",
}
def _select_unauthenticated_get_upstreams(
path: str, upstreams: list[BaseUpstreamProvider]
) -> list[BaseUpstreamProvider]:
"""Select upstream candidates for unauthenticated GET bypass paths.
Tinfoil attestation endpoints are provider-specific. Trying every enabled
upstream can return an unrelated provider's 404 before Tinfoil is reached,
so route those paths only to Tinfoil providers.
"""
if _is_tinfoil_attestation_path(path):
return [
upstream
for upstream in upstreams
if getattr(upstream, "provider_type", None) == "tinfoil"
]
return upstreams
async def refresh_model_maps() -> None:
"""Refresh global model and provider maps using the cost-based algorithm."""
from sqlalchemy.orm import selectinload
@@ -167,6 +196,7 @@ _API_PATH_PREFIXES = (
"moderations",
"providers",
"tee/",
"attestation",
)
@@ -185,20 +215,54 @@ async def proxy(
is_responses_api = path.startswith("v1/responses") or path.startswith("responses")
request_body = await request.body()
request_body_dict = parse_request_body_json(request_body, path)
# /tee/* GET requests (e.g. attestation) don't map to models — just
# forward to all enabled upstreams without model/cost/auth lookups.
if request.method == "GET" and path.startswith("tee/"):
all_upstreams = _upstreams
# EHBP (Encrypted HTTP Body Protocol) requests carry an Ehbp-Encapsulated-Key
# header and a binary HPKE-sealed body. The proxy cannot parse the body to
# extract the model id, so the SDK sends it in X-Routstr-Model. Forward the
# raw encrypted body to the upstream's /private/ endpoint and stream the
# encrypted response back untouched — the SDK's SecureClient decrypts it.
is_ehbp = "ehbp-encapsulated-key" in headers
if is_ehbp:
request_body_dict = {}
model_id = headers.get("x-routstr-model", "")
if not model_id:
return create_error_response(
"invalid_request",
"EHBP request missing X-Routstr-Model header",
400,
request=request,
)
else:
request_body_dict = parse_request_body_json(request_body, path)
if is_responses_api:
model_id = extract_model_from_responses_request(request_body_dict)
else:
model_id = request_body_dict.get("model", "unknown")
# Exact Tinfoil attestation GET routes don't map to models — forward
# without model/cost/auth lookups. Do not prefix-match here: paths such as
# /attestationjunk must continue through normal authentication.
if request.method == "GET" and _is_tinfoil_attestation_path(path):
selected_upstreams = _select_unauthenticated_get_upstreams(path, _upstreams)
if not selected_upstreams:
return create_error_response(
"upstream_error",
"No upstream available for unauthenticated GET path",
502,
request=request,
)
last_error_response = None
for i, upstream in enumerate(all_upstreams):
for i, upstream in enumerate(selected_upstreams):
try:
headers = upstream.prepare_headers(dict(request.headers))
response = await upstream.forward_get_request(request, path, headers)
if response.status_code in [502, 429] and i < len(all_upstreams) - 1:
if (
response.status_code in [502, 429]
and i < len(selected_upstreams) - 1
):
logger.warning(
"Upstream %s returned %s for tee GET %s, trying next",
"Upstream %s returned %s for unauthenticated GET %s, trying next",
upstream.provider_type,
response.status_code,
path,
@@ -207,23 +271,18 @@ async def proxy(
return response
except UpstreamError as e:
logger.warning(
"Upstream %s failed for tee GET %s: %s",
"Upstream %s failed for unauthenticated GET %s: %s",
upstream.provider_type,
path,
e,
)
if i == len(all_upstreams) - 1:
if i == len(selected_upstreams) - 1:
last_error_response = create_upstream_error_response(e, request)
continue
return last_error_response or create_error_response(
"upstream_error", "All upstreams failed", 502, request=request
)
if is_responses_api:
model_id = extract_model_from_responses_request(request_body_dict)
else:
model_id = request_body_dict.get("model", "unknown")
model_obj = get_model_instance(model_id)
if not model_obj:
@@ -240,6 +299,16 @@ async def proxy(
request=request,
)
if is_ehbp:
upstreams = [upstream for upstream in upstreams if upstream.supports_ehbp]
if not upstreams:
return create_error_response(
"unsupported_request",
f"No EHBP-capable provider found for model '{model_id}'",
400,
request=request,
)
# todo figure out cost calculation since fallback provider is usually not the same price
# Use first provider for initial checks/cost calculation
# primary_upstream = upstreams[0]
@@ -259,7 +328,23 @@ async def proxy(
last_error = None
for i, upstream in enumerate(upstreams):
try:
if is_responses_api:
if is_ehbp:
if not upstream.supports_ehbp:
logger.warning(
"Upstream %s does not support EHBP for model=%s",
upstream.provider_type,
model_id,
)
continue
return await forward_ehbp_x_cashu_request(
request=request,
x_cashu_token=x_cashu,
path=path,
max_cost_for_model=max_cost_for_model,
model_obj=model_obj,
upstream=upstream,
)
elif is_responses_api:
return await upstream.handle_x_cashu_responses(
request, x_cashu, path, max_cost_for_model, model_obj
)
@@ -348,7 +433,7 @@ async def proxy(
"upstream_error", "All upstreams failed", 502, request=request
)
if request_body_dict:
if is_ehbp or request_body_dict:
await pay_for_request(key, max_cost_for_model, session)
# Tracks request params already removed in response to upstream rejections,
@@ -362,7 +447,29 @@ async def proxy(
try:
while True:
try:
if is_responses_api:
if is_ehbp:
if not upstream.supports_ehbp:
logger.warning(
"Upstream %s does not support EHBP for model=%s",
upstream.provider_type,
model_id,
)
raise UpstreamError(
f"Provider {upstream.provider_type} does not support EHBP",
status_code=400,
)
response = await forward_ehbp_request(
request=request,
path=path,
headers=headers,
request_body=request_body,
upstream=upstream,
key=key,
max_cost_for_model=max_cost_for_model,
session=session,
model_obj=model_obj,
)
elif is_responses_api:
response = await upstream.forward_responses_request(
request,
path,
@@ -407,7 +514,7 @@ async def proxy(
# When the upstream 400s naming such a param, strip it from the
# body and retry the SAME upstream. ``already_stripped`` bounds
# this to one retry per distinct param so it always terminates.
if response.status_code == 400:
if response.status_code == 400 and not is_ehbp:
correction = correct_request(
request_body,
extract_error_message(response),

View File

@@ -11,6 +11,7 @@ from .openrouter import OpenRouterUpstreamProvider
from .perplexity import PerplexityUpstreamProvider
from .ppqai import PPQAIUpstreamProvider
from .routstr import RoutstrUpstreamProvider
from .tinfoil import TinfoilUpstreamProvider
from .xai import XAIUpstreamProvider
upstream_provider_classes: list[type[BaseUpstreamProvider]] = [
@@ -26,6 +27,7 @@ upstream_provider_classes: list[type[BaseUpstreamProvider]] = [
PerplexityUpstreamProvider,
PPQAIUpstreamProvider,
RoutstrUpstreamProvider,
TinfoilUpstreamProvider,
XAIUpstreamProvider,
]
"""List of all upstream classes"""

View File

@@ -4,7 +4,14 @@ import json
from sqlmodel import select
from ..core import get_logger
from ..core.db import UpstreamProviderRow, create_session
from ..core.db import (
CashuTransaction,
UpstreamProviderRow,
create_session,
)
from ..core.db import (
store_cashu_transaction_with_retry as store_cashu_transaction,
)
from ..wallet import send_token
from .routstr import RoutstrUpstreamProvider
@@ -123,7 +130,6 @@ async def _check_and_topup(row: UpstreamProviderRow) -> None:
},
)
print(amount, mint_url)
try:
token = await send_token(amount, "sat", mint_url)
except Exception as e:
@@ -138,6 +144,23 @@ async def _check_and_topup(row: UpstreamProviderRow) -> None:
)
return
try:
await store_cashu_transaction(
token=token,
amount=amount,
unit="sat",
mint_url=mint_url,
typ="out",
collected=False,
source="auto_topup",
)
except Exception:
logger.critical(
"Aborting auto top-up because its cashu token could not be persisted",
extra={"provider_id": row.id, "mint_url": mint_url},
)
return
result = await provider.topup(token)
if "error" in result:
@@ -149,6 +172,26 @@ async def _check_and_topup(row: UpstreamProviderRow) -> None:
},
)
else:
async with create_session() as session:
transaction = (
await session.exec(
select(CashuTransaction).where(
CashuTransaction.token == token,
CashuTransaction.type == "out",
CashuTransaction.source == "auto_topup",
)
)
).first()
if transaction is None:
logger.critical(
"Completed auto top-up transaction is missing from the database",
extra={"provider_id": row.id, "mint_url": mint_url},
)
else:
transaction.collected = True
session.add(transaction)
await session.commit()
logger.info(
"Auto top-up completed successfully",
extra={

View File

@@ -4,6 +4,7 @@ import asyncio
import json
import math
import traceback
import typing
import uuid
from collections.abc import AsyncGenerator, AsyncIterator, Iterator
from typing import Any, Mapping, Self, cast
@@ -20,7 +21,9 @@ from ..core.db import (
AsyncSession,
UpstreamProviderRow,
create_session,
store_cashu_transaction,
)
from ..core.db import (
store_cashu_transaction_with_retry as store_cashu_transaction,
)
from ..core.exceptions import UpstreamError
from ..core.redaction import redact_org_ids
@@ -55,6 +58,9 @@ from .count_tokens import count_tokens_locally
from .litellm_routing import detect_litellm_prefix
from .rate_limit import UPSTREAM_RATE_LIMIT, classify_rate_limit
if typing.TYPE_CHECKING:
from .ehbp import ConfidentialInferenceProfile, EHBPForwardingTarget
logger = get_logger(__name__)
@@ -2845,6 +2851,26 @@ class BaseUpstreamProvider:
# Don't revert here — proxy.py owns payment revert to avoid double-revert
raise UpstreamError("An unexpected server error occurred", status_code=500)
supports_ehbp: bool = False
def get_confidential_inference_profile(self) -> "ConfidentialInferenceProfile | None":
"""Return provider policy for encrypted/confidential inference forwarding."""
return None
def get_ehbp_forwarding_target(
self, path: str, model_obj: Model
) -> "EHBPForwardingTarget":
"""Return the EHBP forwarding target for this provider.
Providers must explicitly opt in by setting ``supports_ehbp = True``
and overriding this method. Most upstreams do not accept EHBP-encrypted
request bodies, so the base provider intentionally does not provide a
default endpoint.
"""
raise NotImplementedError(
f"Provider {self.provider_type} does not support EHBP forwarding"
)
async def forward_responses_request(
self,
request: Request,

1202
routstr/upstream/ehbp.py Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -272,6 +272,7 @@ async def _seed_providers_from_settings(
("PERPLEXITY_API_KEY", "perplexity", None, None),
("FIREWORKS_API_KEY", "fireworks", None, None),
("XAI_API_KEY", "xai", None, None),
("TINFOIL_API_KEY", "tinfoil", None, None),
]
for env_key, provider_type, _, _ in env_mappings:

View File

@@ -8,6 +8,7 @@ from pydantic.v1 import BaseModel, Field
from ..core.logging import get_logger
from ..payment.models import Architecture, Model, Pricing, async_fetch_openrouter_models
from .base import BaseUpstreamProvider, TopupData
from .ehbp import EHBPForwardingTarget
if TYPE_CHECKING:
from ..core.db import UpstreamProviderRow
@@ -39,6 +40,10 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
default_base_url = "https://api.ppq.ai"
platform_url = "https://ppq.ai/api-docs"
IGNORED_MODEL_IDS: list[str] = ["auto"]
# PPQ.AI has a private encrypted endpoint, but this proxy currently has no
# provider-attested usage extractor/model binding for it. Keep EHBP disabled
# until a ConfidentialInferenceProfile can bill it without max-cost fallback.
supports_ehbp = False
def __init__(self, api_key: str, provider_fee: float = 1.0):
super().__init__(
@@ -70,6 +75,20 @@ class PPQAIUpstreamProvider(BaseUpstreamProvider):
def transform_model_name(self, model_id: str) -> str:
return model_id
def get_ehbp_forwarding_target(
self, path: str, model_obj: Model
) -> EHBPForwardingTarget:
"""Return the PPQ.AI private enclave target for EHBP requests.
PPQ.AI exposes EHBP-aware inference under /private/v1/... separate
from the public /v1/... endpoint. The encrypted body remains opaque to
Routstr, so PPQ.AI also needs X-Private-Model for routing/billing.
"""
return EHBPForwardingTarget(
url=f"{self.base_url.rstrip('/')}/private/{path.lstrip('/')}",
headers={"X-Private-Model": model_obj.forwarded_model_id or model_obj.id},
)
@classmethod
async def create_account_static(cls) -> dict[str, object]:
"""Create a new PPQ.AI account without requiring an instance.

236
routstr/upstream/tinfoil.py Normal file
View File

@@ -0,0 +1,236 @@
from __future__ import annotations
from typing import TYPE_CHECKING
import httpx
from fastapi import Request
from fastapi.responses import Response, StreamingResponse
from pydantic.v1 import BaseModel
from ..core.exceptions import UpstreamError
from ..core.logging import get_logger
from ..payment.models import Architecture, Model, Pricing
from .base import BaseUpstreamProvider
from .ehbp import (
_ENCLAVE_URL_HEADER,
_PROXY_ONLY_HEADERS,
_RESPONSE_USAGE_HEADER,
ConfidentialInferenceProfile,
EHBPForwardingTarget,
)
if TYPE_CHECKING:
from ..core.db import UpstreamProviderRow
logger = get_logger(__name__)
class TinfoilModelPricing(BaseModel):
inputTokenPricePer1M: float = 0.0
outputTokenPricePer1M: float = 0.0
requestPrice: float = 0.0
class TinfoilModel(BaseModel):
id: str
context_window: int = 0
created: int = 0
multimodal: bool = False
reasoning: bool = False
tool_calling: bool = False
type: str = "chat"
pricing: TinfoilModelPricing = TinfoilModelPricing()
endpoints: list[str] = []
class TinfoilUpstreamProvider(BaseUpstreamProvider):
"""Direct upstream provider for the Tinfoil inference API.
Tinfoil hosts open-source models inside attested secure enclaves and exposes
an OpenAI-compatible API at ``https://inference.tinfoil.sh``. Request and
response bodies are encrypted end-to-end with EHBP (HPKE), so Routstr acts
as a blind relay: it forwards the opaque encrypted body, never sees
plaintext, and bills from the ``X-Tinfoil-Usage-Metrics`` header that
Tinfoil returns outside the encrypted body when
``X-Tinfoil-Request-Usage-Metrics: true`` is set.
"""
provider_type = "tinfoil"
default_base_url = "https://inference.tinfoil.sh"
platform_url = "https://docs.tinfoil.sh"
supports_ehbp = True
confidential_inference_profile = ConfidentialInferenceProfile(
usage_response_header=_RESPONSE_USAGE_HEADER,
client_target_url_header=_ENCLAVE_URL_HEADER,
allow_client_target_override=True,
proxy_only_headers=_PROXY_ONLY_HEADERS,
)
def __init__(self, api_key: str, provider_fee: float = 1.0):
super().__init__(
base_url=self.default_base_url,
api_key=api_key,
provider_fee=provider_fee,
)
@classmethod
def _build_from_row(
cls, provider_row: "UpstreamProviderRow"
) -> "TinfoilUpstreamProvider":
return cls(
api_key=provider_row.api_key,
provider_fee=provider_row.provider_fee,
)
@classmethod
def get_provider_metadata(cls) -> dict[str, object]:
return {
"id": cls.provider_type,
"name": "Tinfoil",
"default_base_url": cls.default_base_url,
"fixed_base_url": True,
"platform_url": cls.platform_url,
"can_create_account": False,
"can_topup": False,
"can_show_balance": False,
}
def transform_model_name(self, model_id: str) -> str:
return model_id.removeprefix("tinfoil/")
def get_confidential_inference_profile(self) -> ConfidentialInferenceProfile:
return self.confidential_inference_profile
async def forward_get_request(
self,
request: Request,
path: str,
headers: dict,
) -> Response | StreamingResponse:
"""Handle Tinfoil-specific GET endpoints.
* ``/attestation`` (or ``/tee/attestation``): proxy to the Tinfoil ATC
(attestation bundle proxy) at ``https://atc.tinfoil.sh/attestation``.
* Other GETs: forward to the provider base URL
(``https://inference.tinfoil.sh``). ``X-Tinfoil-Enclave-Url`` is an
EHBP-only header used for encrypted POST requests and is not honored
for unencrypted GET requests.
"""
clean_path = path.removeprefix("tee/").rstrip("/")
if clean_path == "attestation":
return await self._proxy_attestation(headers)
return await super().forward_get_request(request, path, headers)
async def _proxy_attestation(self, headers: dict) -> Response:
url = "https://atc.tinfoil.sh/attestation"
async with httpx.AsyncClient(
transport=httpx.AsyncHTTPTransport(retries=1),
timeout=30.0,
) as client:
try:
resp = await client.get(
url,
headers={
"Accept": headers.get("accept", "application/json"),
},
)
response_headers = dict(resp.headers)
response_headers.pop("content-encoding", None)
response_headers.pop("content-length", None)
return Response(
content=resp.content,
status_code=resp.status_code,
headers=response_headers,
)
except Exception as exc:
raise UpstreamError(
f"Error fetching Tinfoil attestation: {type(exc).__name__}",
status_code=502,
) from exc
def get_ehbp_forwarding_target(
self, path: str, model_obj: Model
) -> EHBPForwardingTarget:
"""Return the Tinfoil enclave target for EHBP requests.
Requests usage metrics from the enclave so Routstr can bill exactly
without decrypting the response body. The actual forwarding URL is
overridden at dispatch time by ``X-Tinfoil-Enclave-Url`` when the SDK
sends it (see ``routstr/upstream/ehbp.py``).
"""
return EHBPForwardingTarget(
url=f"{self.base_url.rstrip('/')}/{path.lstrip('/')}",
headers={"X-Tinfoil-Request-Usage-Metrics": "true"},
profile=self.confidential_inference_profile,
)
async def fetch_models(self) -> list[Model]:
"""Fetch models from the public Tinfoil models endpoint.
``GET /v1/models`` is unauthenticated and returns all available models
with their pricing in USD per 1M tokens.
"""
url = f"{self.base_url}/v1/models"
try:
async with httpx.AsyncClient(timeout=30.0) as client:
response = await client.get(url)
response.raise_for_status()
data = response.json()
models_data = data.get("data", [])
models: list[Model] = []
for model_data in models_data:
try:
tf = TinfoilModel.parse_obj(model_data)
input_price = tf.pricing.inputTokenPricePer1M
output_price = tf.pricing.outputTokenPricePer1M
request_price = tf.pricing.requestPrice
modality = "text->text"
input_modalities = ["text"]
output_modalities = ["text"]
if tf.multimodal:
modality = "text->text+image"
input_modalities = ["text", "image"]
models.append(
Model(
id=tf.id,
name=tf.id,
created=tf.created,
description=f"Tinfoil {tf.type} model",
context_length=tf.context_window,
architecture=Architecture(
modality=modality,
input_modalities=input_modalities,
output_modalities=output_modalities,
tokenizer="Unknown",
instruct_type=None,
),
pricing=Pricing(
prompt=input_price / 1_000_000,
completion=output_price / 1_000_000,
request=request_price,
image=0.0,
web_search=0.0,
internal_reasoning=0.0,
),
)
)
except Exception as e:
logger.warning(
"Failed to parse Tinfoil model",
extra={
"model_id": model_data.get("id", "unknown"),
"error": str(e),
"error_type": type(e).__name__,
},
)
return models
except Exception as e:
logger.error(
"Error fetching models from Tinfoil",
extra={"error": str(e), "error_type": type(e).__name__},
)
return []

View File

@@ -0,0 +1,189 @@
"""h11-based HTTP client for EHBP requests that captures HTTP trailers.
httpx/httpcore silently discard HTTP trailers during chunked transfer
decoding. Tinfoil returns ``X-Tinfoil-Usage-Metrics`` as a trailer on
streaming responses, so we need a lower-level HTTP client that preserves
trailers from the h11 ``EndOfMessage`` event.
Because EHBP response bodies are opaque encrypted blobs, buffering the full
response is acceptable — the client decrypts the complete body regardless of
whether it arrived streamed or buffered.
"""
from __future__ import annotations
import asyncio
import ssl
from dataclasses import dataclass, field
from urllib.parse import urlsplit
import h11
from ..core import get_logger
logger = get_logger(__name__)
_READ_BUFSIZE = 65536
_DEFAULT_TIMEOUT_SECONDS = 30.0
_DEFAULT_CLOSE_TIMEOUT_SECONDS = 1.0
_DEFAULT_MAX_RESPONSE_BYTES = 25 * 1024 * 1024
_HOP_BY_HOP_HEADERS = {
"connection",
"keep-alive",
"proxy-authenticate",
"proxy-authorization",
"te",
"trailer",
"transfer-encoding",
"upgrade",
}
@dataclass
class TrailerResponse:
"""Buffered HTTP response with optional trailer headers."""
status_code: int
headers: list[tuple[str, str]]
body: bytes
trailers: list[tuple[str, str]] = field(default_factory=list)
def _get_header(headers: list[tuple[str, str]], name: str) -> str | None:
name_lower = name.lower()
for k, v in headers:
if k.lower() == name_lower:
return v
return None
def _strip_hop_by_hop_headers(headers: dict[str, str]) -> dict[str, str]:
"""Remove connection-specific headers before serializing a new request."""
connection_tokens: set[str] = set()
for key, value in headers.items():
if key.lower() == "connection":
connection_tokens.update(
token.strip().lower() for token in value.split(",") if token.strip()
)
excluded = _HOP_BY_HOP_HEADERS | connection_tokens
return {key: value for key, value in headers.items() if key.lower() not in excluded}
async def forward_with_trailer(
*,
method: str,
url: str,
headers: dict[str, str],
body: bytes,
timeout_seconds: float = _DEFAULT_TIMEOUT_SECONDS,
max_response_bytes: int = _DEFAULT_MAX_RESPONSE_BYTES,
close_timeout_seconds: float = _DEFAULT_CLOSE_TIMEOUT_SECONDS,
) -> TrailerResponse:
"""Send an HTTP/1.1 request via h11 and capture HTTP trailers.
Returns a :class:`TrailerResponse` with the full buffered body and any
trailer headers from the ``EndOfMessage`` event.
"""
parsed = urlsplit(url)
host = parsed.hostname
if not host:
raise ValueError(f"Invalid URL (no hostname): {url}")
port = parsed.port or 443
path = parsed.path or "/"
if parsed.query:
path = f"{path}?{parsed.query}"
# FastAPI has already decoded the incoming request body. Do not carry the
# original connection's framing or other hop-by-hop metadata into the new
# upstream connection.
headers = _strip_hop_by_hop_headers(headers)
ssl_ctx = ssl.create_default_context()
reader, writer = await asyncio.wait_for(
asyncio.open_connection(host, port, ssl=ssl_ctx),
timeout=timeout_seconds,
)
try:
# Build HTTP/1.1 request
header_lines = [f"{method} {path} HTTP/1.1"]
has_host = any(k.lower() == "host" for k in headers)
if not has_host:
header_lines.append(f"Host: {host}")
header_lines.append("Connection: close")
for key, value in headers.items():
if key.lower() == "host":
continue
header_lines.append(f"{key}: {value}")
if body and not any(k.lower() == "content-length" for k in headers):
header_lines.append(f"Content-Length: {len(body)}")
request_data = "\r\n".join(header_lines).encode() + b"\r\n\r\n"
if body:
request_data += body
writer.write(request_data)
await asyncio.wait_for(writer.drain(), timeout=timeout_seconds)
# Parse response with h11
conn = h11.Connection(h11.CLIENT)
status_code = 0
resp_headers: list[tuple[str, str]] = []
body_chunks: list[bytes] = []
body_size = 0
trailers: list[tuple[str, str]] = []
while True:
event = conn.next_event()
if event is h11.NEED_DATA:
data = await asyncio.wait_for(
reader.read(_READ_BUFSIZE),
timeout=timeout_seconds,
)
conn.receive_data(data if data else b"")
continue
if isinstance(event, h11.Response):
status_code = event.status_code
resp_headers = [(k.decode(), v.decode()) for k, v in event.headers]
elif isinstance(event, h11.Data):
body_size += len(event.data)
if body_size > max_response_bytes:
raise ValueError(
f"EHBP response exceeded {max_response_bytes} bytes"
)
body_chunks.append(event.data)
elif isinstance(event, h11.EndOfMessage):
for k, v in event.headers:
trailers.append((k.decode(), v.decode()))
break
elif isinstance(event, h11.PAUSED):
# Shouldn't happen for simple request/response, but break safely
logger.warning("h11 PAUSED event during EHBP response parsing")
break
elif isinstance(event, h11.ConnectionClosed):
break
return TrailerResponse(
status_code=status_code,
headers=resp_headers,
body=b"".join(body_chunks),
trailers=trailers,
)
finally:
writer.close()
if close_timeout_seconds > 0:
try:
await asyncio.wait_for(
writer.wait_closed(), timeout=close_timeout_seconds
)
except Exception:
pass

View File

@@ -14,7 +14,7 @@ from pydantic_core import PydanticUndefined
from sqlmodel import col, select, update
from .core import db, get_logger
from .core.db import store_cashu_transaction
from .core.db import store_cashu_transaction_with_retry as store_cashu_transaction
from .core.settings import settings
from .payment.lnurl import raw_send_to_lnurl
@@ -745,11 +745,11 @@ async def credit_balance(
)
except Exception:
pass
logger.debug(
"Cashu token successfully redeemed and stored",
extra={"amount": amount, "unit": unit, "mint_url": mint_url},
)
else:
logger.debug(
"Cashu token successfully redeemed and stored",
extra={"amount": amount, "unit": unit, "mint_url": mint_url},
)
return amount
except Exception as e:
logger.error(
@@ -988,54 +988,63 @@ async def periodic_payout() -> None:
)
async def _refund_sweep_once(cutoff: int) -> None:
async with db.create_session() as session:
stmt = select(db.CashuTransaction).where(
db.CashuTransaction.type == "out",
db.CashuTransaction.collected == False, # noqa: E712
db.CashuTransaction.swept == False, # noqa: E712
db.CashuTransaction.created_at < cutoff,
)
results = await session.exec(stmt)
refunds = results.all()
for refund in refunds:
try:
await recieve_token(refund.token)
refund.swept = True
session.add(refund)
logger.info(
"Swept uncollected refund",
extra={
"id": refund.id,
"amount": refund.amount,
"unit": refund.unit,
},
)
except Exception as e:
error_msg = str(e).lower()
if "already spent" in error_msg:
refund.collected = True
session.add(refund)
logger.info(
"Refund already spent (client collected), marking swept",
extra={
"id": refund.id,
},
)
else:
logger.warning(
"Failed to sweep refund",
extra={
"id": refund.id,
"error": str(e),
},
)
await session.commit()
async def refund_sweep_once() -> None:
"""Sweep eligible uncollected refund tokens once."""
cutoff = int(time.time()) - settings.refund_sweep_ttl_seconds
await _refund_sweep_once(cutoff)
async def periodic_refund_sweep() -> None:
while True:
await asyncio.sleep(60 * 60) # every hour
try:
cutoff = int(time.time()) - settings.refund_sweep_ttl_seconds
async with db.create_session() as session:
stmt = select(db.CashuTransaction).where(
db.CashuTransaction.type == "out",
db.CashuTransaction.collected == False, # noqa: E712
db.CashuTransaction.swept == False, # noqa: E712
db.CashuTransaction.created_at < cutoff,
)
results = await session.exec(stmt)
refunds = results.all()
for refund in refunds:
try:
await recieve_token(refund.token)
refund.swept = True
session.add(refund)
logger.info(
"Swept uncollected refund",
extra={
"id": refund.id,
"amount": refund.amount,
"unit": refund.unit,
},
)
except Exception as e:
error_msg = str(e).lower()
if "already spent" in error_msg:
refund.collected = True
session.add(refund)
logger.info(
"Refund already spent (client collected), marking swept",
extra={
"id": refund.id,
},
)
else:
logger.warning(
"Failed to sweep refund",
extra={
"id": refund.id,
"error": str(e),
},
)
await session.commit()
await refund_sweep_once()
except Exception as e:
logger.error(
"Error in periodic refund sweep",

View File

@@ -0,0 +1,82 @@
from types import SimpleNamespace
from unittest.mock import AsyncMock, Mock
import pytest
from routstr.core import admin
@pytest.mark.asyncio
@pytest.mark.parametrize("requested_mint", [None, "https://secondary.example"])
async def test_withdraw_uses_effective_mint_and_records_outgoing_transaction(
monkeypatch: pytest.MonkeyPatch, requested_mint: str | None
) -> None:
primary_mint = "https://primary.example"
effective_mint = requested_mint or primary_mint
wallet = object()
proofs = [SimpleNamespace(amount=40), SimpleNamespace(amount=60)]
token = "cashuBoutgoing"
get_wallet = AsyncMock(return_value=wallet)
get_proofs = Mock(return_value=proofs)
filter_proofs = AsyncMock(return_value=proofs)
send_token = AsyncMock(return_value=token)
store_transaction = AsyncMock(return_value=True)
monkeypatch.setattr(admin, "get_wallet", get_wallet)
monkeypatch.setattr(admin, "get_proofs_per_mint_and_unit", get_proofs)
monkeypatch.setattr(admin, "slow_filter_spend_proofs", filter_proofs)
monkeypatch.setattr(admin, "send_token", send_token)
monkeypatch.setattr(admin, "store_cashu_transaction", store_transaction)
monkeypatch.setattr(admin.settings, "primary_mint", primary_mint)
result = await admin.withdraw(
Mock(),
admin.WithdrawRequest(amount=75, mint_url=requested_mint, unit="sat"),
)
assert result == {"token": token}
get_wallet.assert_awaited_once_with(effective_mint, "sat")
get_proofs.assert_called_once_with(wallet, effective_mint, "sat", not_reserved=True)
filter_proofs.assert_awaited_once_with(proofs, wallet)
send_token.assert_awaited_once_with(75, "sat", effective_mint)
store_transaction.assert_awaited_once_with(
token=token,
amount=75,
unit="sat",
mint_url=effective_mint,
typ="out",
collected=False,
source="admin",
)
@pytest.mark.asyncio
async def test_withdraw_returns_issued_token_when_audit_storage_fails(
monkeypatch: pytest.MonkeyPatch,
) -> None:
mint = "https://primary.example"
proofs = [SimpleNamespace(amount=100)]
token = "cashuBrecoverable"
monkeypatch.setattr(admin, "get_wallet", AsyncMock(return_value=object()))
monkeypatch.setattr(
admin, "get_proofs_per_mint_and_unit", Mock(return_value=proofs)
)
monkeypatch.setattr(
admin, "slow_filter_spend_proofs", AsyncMock(return_value=proofs)
)
monkeypatch.setattr(admin, "send_token", AsyncMock(return_value=token))
monkeypatch.setattr(
admin,
"store_cashu_transaction",
AsyncMock(side_effect=RuntimeError("database unavailable")),
)
critical = Mock()
monkeypatch.setattr(admin.logger, "critical", critical)
monkeypatch.setattr(admin.settings, "primary_mint", mint)
result = await admin.withdraw(Mock(), admin.WithdrawRequest(amount=75))
assert result == {"token": token}
critical.assert_called_once()

View File

@@ -0,0 +1,143 @@
import json
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from routstr.core.db import CashuTransaction
from routstr.upstream.auto_topup import _check_and_topup
def _row() -> MagicMock:
row = MagicMock()
row.id = "provider-1"
row.base_url = "https://provider.test"
row.api_key = "secret"
row.provider_settings = json.dumps(
{
"auto_topup": True,
"topup_threshold": 100,
"topup_amount_limit": 50,
"topup_mint_url": "https://mint.test",
}
)
return row
class _Session:
def __init__(self, transaction: CashuTransaction) -> None:
self.transaction = transaction
self.commit = AsyncMock()
async def __aenter__(self) -> "_Session":
return self
async def __aexit__(self, *args: object) -> None:
return None
async def exec(self, query: object) -> MagicMock:
result = MagicMock()
result.first.return_value = self.transaction
return result
def add(self, transaction: CashuTransaction) -> None:
self.transaction = transaction
@pytest.mark.asyncio
async def test_auto_topup_persists_before_sending_and_marks_success_collected() -> None:
provider = MagicMock()
provider.get_balance = AsyncMock(return_value=0)
provider.topup = AsyncMock(return_value={"balance": 50})
transaction = CashuTransaction(
token="cashu-token", amount=50, unit="sat", source="auto_topup"
)
session = _Session(transaction)
with (
patch(
"routstr.upstream.auto_topup.RoutstrUpstreamProvider.from_db_row",
return_value=provider,
),
patch(
"routstr.upstream.auto_topup.send_token",
AsyncMock(return_value="cashu-token"),
),
patch(
"routstr.upstream.auto_topup.store_cashu_transaction",
AsyncMock(return_value=True),
) as store,
patch("routstr.upstream.auto_topup.create_session", return_value=session),
):
await _check_and_topup(_row())
store.assert_awaited_once_with(
token="cashu-token",
amount=50,
unit="sat",
mint_url="https://mint.test",
typ="out",
collected=False,
source="auto_topup",
)
provider.topup.assert_awaited_once_with("cashu-token")
assert transaction.collected is True
session.commit.assert_awaited_once()
@pytest.mark.asyncio
@pytest.mark.parametrize("outcome", [{"error": "rejected"}, RuntimeError("network")])
async def test_auto_topup_failure_leaves_persisted_token_uncollected(
outcome: object,
) -> None:
provider = MagicMock()
provider.get_balance = AsyncMock(return_value=0)
provider.topup = AsyncMock(
side_effect=outcome if isinstance(outcome, Exception) else None,
return_value=outcome,
)
with (
patch(
"routstr.upstream.auto_topup.RoutstrUpstreamProvider.from_db_row",
return_value=provider,
),
patch(
"routstr.upstream.auto_topup.send_token",
AsyncMock(return_value="cashu-token"),
),
patch(
"routstr.upstream.auto_topup.store_cashu_transaction",
AsyncMock(return_value=True),
),
patch("routstr.upstream.auto_topup.create_session") as create_session,
):
if isinstance(outcome, Exception):
with pytest.raises(RuntimeError):
await _check_and_topup(_row())
else:
await _check_and_topup(_row())
create_session.assert_not_called()
@pytest.mark.asyncio
async def test_auto_topup_does_not_send_untracked_token() -> None:
provider = MagicMock()
provider.get_balance = AsyncMock(return_value=0)
provider.topup = AsyncMock()
with (
patch(
"routstr.upstream.auto_topup.RoutstrUpstreamProvider.from_db_row",
return_value=provider,
),
patch(
"routstr.upstream.auto_topup.send_token",
AsyncMock(return_value="cashu-token"),
),
patch(
"routstr.upstream.auto_topup.store_cashu_transaction",
AsyncMock(side_effect=RuntimeError("database unavailable")),
),
):
await _check_and_topup(_row())
provider.topup.assert_not_awaited()

View File

@@ -0,0 +1,56 @@
from unittest.mock import AsyncMock, patch
import pytest
from routstr.core.db import store_cashu_transaction
@pytest.mark.asyncio
@pytest.mark.parametrize(
"error",
[
OSError("disk full"),
RuntimeError("connection lost"),
ConnectionRefusedError("database unavailable"),
],
)
async def test_store_cashu_transaction_propagates_commit_errors(
error: Exception,
) -> None:
session = AsyncMock()
session.commit.side_effect = error
session.__aenter__.return_value = session
session.__aexit__.return_value = None
with (
patch("routstr.core.db.create_session", return_value=session),
patch("routstr.core.db.logger.critical") as critical,
):
with pytest.raises(type(error), match=str(error)):
await store_cashu_transaction(
token="cashuAtest",
amount=1_000,
unit="sat",
mint_url="https://mint.example",
typ="out",
request_id="request-1",
)
critical.assert_called_once()
@pytest.mark.asyncio
async def test_store_cashu_transaction_returns_true_after_commit() -> None:
session = AsyncMock()
session.__aenter__.return_value = session
session.__aexit__.return_value = None
with patch("routstr.core.db.create_session", return_value=session):
stored = await store_cashu_transaction(
token="cashuAtest",
amount=1_000,
unit="sat",
)
assert stored is True
session.commit.assert_awaited_once()

View File

@@ -0,0 +1,91 @@
from typing import Any
from unittest.mock import AsyncMock, patch
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core import db
@pytest.mark.asyncio
async def test_cashu_transaction_storage_retries_then_succeeds() -> None:
store = AsyncMock(side_effect=[OSError("database locked"), True])
sleep = AsyncMock()
with (
patch("routstr.core.db.store_cashu_transaction", store),
patch("routstr.core.db.asyncio.sleep", sleep),
):
stored = await db.store_cashu_transaction_with_retry(
token="cashuAretry",
amount=100,
unit="sat",
)
assert stored is True
assert store.await_count == 2
sleep.assert_awaited_once_with(0.25)
@pytest.mark.asyncio
async def test_cashu_transaction_retry_is_idempotent_after_ambiguous_commit() -> None:
engine = create_async_engine("sqlite+aiosqlite://")
async with engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all)
original_store = db.store_cashu_transaction
attempts = 0
async def ambiguous_store(**kwargs: Any) -> bool:
nonlocal attempts
attempts += 1
stored = await original_store(**kwargs)
if attempts == 1:
raise OSError("connection dropped after commit")
return stored
with (
patch.object(db, "engine", engine),
patch("routstr.core.db.store_cashu_transaction", ambiguous_store),
patch("routstr.core.db.asyncio.sleep", AsyncMock()),
):
stored = await db.store_cashu_transaction_with_retry(
token="cashuAambiguous",
amount=100,
unit="sat",
)
async with AsyncSession(engine) as session:
result = await session.exec(select(db.CashuTransaction))
transactions = result.all()
assert stored is True
assert attempts == 2
assert len(transactions) == 1
await engine.dispose()
@pytest.mark.asyncio
async def test_cashu_transaction_storage_raises_after_bounded_retries() -> None:
error = OSError("database unavailable")
store = AsyncMock(side_effect=error)
sleep = AsyncMock()
with (
patch("routstr.core.db.store_cashu_transaction", store),
patch("routstr.core.db.asyncio.sleep", sleep),
patch("routstr.core.db.logger.critical") as critical,
):
with pytest.raises(OSError, match="database unavailable"):
await db.store_cashu_transaction_with_retry(
token="cashuAfail",
amount=100,
unit="sat",
max_attempts=3,
)
assert store.await_count == 3
assert [call.args[0] for call in sleep.await_args_list] == [0.25, 0.5]
critical.assert_called_once()

View File

@@ -532,6 +532,72 @@ async def test_openrouter_upstream_inference_cost_components_are_used() -> None:
assert result.input_msats + result.output_msats == result.total_msats == 4471
# ============================================================================
# PPQ.AI BYOK: upstream_inference_cost + BYOK fee billing
#
# PPQ.AI (bring-your-own-key) returns a small ~5 % BYOK routing fee in
# ``usage.cost`` and the real inference cost in
# ``cost_details.upstream_inference_cost``. The old code billed only the fee,
# under-charging by ~20×. The fix bills ``upstream_inference_cost + byok_fee``
# — what PPQ actually deducts from the balance.
# Payload numbers are from a live ``glm-5.2-fast`` request (GitHub issue #615).
# ============================================================================
@pytest.mark.asyncio
async def test_ppq_byok_bills_upstream_inference_cost_plus_fee() -> None:
"""PPQ.AI BYOK must bill upstream_inference_cost + byok_fee, not just the
fee. Mirrors the live request from GitHub issue #615."""
response = {
"model": "glm-5.2-fast",
"usage": {
"prompt_tokens": 164371, # includes 159301 cached
"completion_tokens": 99,
"cost": 0.002260057305, # ~5% BYOK routing fee
"is_byok": True,
"prompt_tokens_details": {"cached_tokens": 159301},
"cost_details": {
"upstream_inference_cost": 0.04475361,
"upstream_inference_prompt_cost": 0.04410021,
"upstream_inference_completions_cost": 0.0006534,
},
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# The fix bills upstream_inference_cost + byok_fee (~0.047 USD → ~940k
# msats), not the fee alone (~0.0023 USD → ~45k msats). ~20× correction.
assert result.total_msats == 940274
assert result.input_msats + result.output_msats == result.total_msats
assert result.input_msats == 926546
assert result.output_msats == 13728
assert result.total_usd == pytest.approx(0.047013667305)
# Token normalisation (OpenAI dialect: cached included in prompt_tokens)
assert result.input_tokens == 5070 # 164371 - 159301
assert result.cache_read_input_tokens == 159301
assert result.output_tokens == 99
@pytest.mark.asyncio
async def test_ppq_byok_fee_only_would_undercharge() -> None:
"""Sanity check: billing only usage.cost (the BYOK fee) under-charges by
~20×. This documents the regression the fix prevents."""
response = {
"model": "glm-5.2-fast",
"usage": {
"prompt_tokens": 164371,
"completion_tokens": 99,
"cost": 0.002260057305, # BYOK fee only — no upstream_inference_cost
"is_byok": True,
"prompt_tokens_details": {"cached_tokens": 159301},
},
}
result = await calculate_cost(response, max_cost=100000)
assert isinstance(result, CostData)
# Without upstream_inference_cost, only the fee is billed — the old bug.
assert result.total_msats == 45202
# ============================================================================
# Test 13: Missing Usage Block
# ============================================================================

View File

@@ -0,0 +1,185 @@
from __future__ import annotations
from typing import AsyncGenerator
import pytest
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlalchemy.pool import StaticPool
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.db import ApiKey
from routstr.upstream.ehbp import (
finalize_ehbp_actual_cost_payment,
finalize_ehbp_max_cost_payment,
)
def _make_engine() -> AsyncEngine:
return create_async_engine(
"sqlite+aiosqlite://",
poolclass=StaticPool,
connect_args={"check_same_thread": False},
)
@pytest.fixture
async def session(monkeypatch: pytest.MonkeyPatch) -> AsyncGenerator[AsyncSession, None]:
monkeypatch.setattr("routstr.upstream.ehbp.ROUTSTR_FEE_PERCENT", 0)
engine = _make_engine()
async with engine.begin() as conn:
await conn.run_sync(SQLModel.metadata.create_all)
db_session = AsyncSession(engine, expire_on_commit=False)
try:
yield db_session
finally:
await db_session.close()
await engine.dispose()
async def _api_key(session: AsyncSession, hashed_key: str) -> ApiKey | None:
return (
await session.exec(select(ApiKey).where(ApiKey.hashed_key == hashed_key))
).one_or_none()
@pytest.mark.asyncio
async def test_finalize_actual_cost_payment_updates_balance_and_releases_reserve(
session: AsyncSession,
) -> None:
key = ApiKey(
hashed_key="ehbp-actual",
balance=10_000,
reserved_balance=3_000,
reserved_at=123,
)
session.add(key)
await session.commit()
await finalize_ehbp_actual_cost_payment(
key,
session,
reserved_cost_for_model=3_000,
model_id="tinfoil/model",
cost_info={
"total_msats": 1_200,
"input_tokens": 10,
"output_tokens": 20,
"input_msats": 500,
"output_msats": 700,
},
)
updated = await _api_key(session, "ehbp-actual")
assert updated is not None
assert updated.balance == 8_800
assert updated.reserved_balance == 0
assert updated.reserved_at is None
assert updated.total_spent == 1_200
@pytest.mark.asyncio
async def test_finalize_max_cost_payment_updates_parent_and_child_spend(
session: AsyncSession,
) -> None:
parent = ApiKey(
hashed_key="ehbp-parent",
balance=10_000,
reserved_balance=3_000,
reserved_at=123,
)
child = ApiKey(
hashed_key="ehbp-child",
balance=0,
reserved_balance=3_000,
reserved_at=123,
parent_key_hash="ehbp-parent",
)
session.add(parent)
session.add(child)
await session.commit()
await finalize_ehbp_max_cost_payment(
child,
session,
max_cost_for_model=3_000,
model_id="tinfoil/model",
)
updated_parent = await _api_key(session, "ehbp-parent")
updated_child = await _api_key(session, "ehbp-child")
assert updated_parent is not None
assert updated_child is not None
assert updated_parent.balance == 7_000
assert updated_parent.reserved_balance == 0
assert updated_parent.reserved_at is None
assert updated_parent.total_spent == 3_000
assert updated_child.balance == 0
assert updated_child.reserved_balance == 0
assert updated_child.reserved_at is None
assert updated_child.total_spent == 3_000
@pytest.mark.asyncio
async def test_finalize_actual_cost_payment_rolls_back_when_parent_update_matches_no_rows(
session: AsyncSession,
) -> None:
key = ApiKey(
hashed_key="ehbp-missing-parent",
balance=10_000,
reserved_balance=3_000,
reserved_at=123,
)
session.add(key)
await session.commit()
await session.delete(key)
await session.commit()
await finalize_ehbp_actual_cost_payment(
key,
session,
reserved_cost_for_model=3_000,
model_id="tinfoil/model",
cost_info={"total_msats": 1_200},
)
assert await _api_key(session, "ehbp-missing-parent") is None
@pytest.mark.asyncio
async def test_finalize_max_cost_payment_rolls_back_parent_when_child_update_matches_no_rows(
session: AsyncSession,
) -> None:
parent = ApiKey(
hashed_key="ehbp-rollback-parent",
balance=10_000,
reserved_balance=3_000,
reserved_at=123,
)
child = ApiKey(
hashed_key="ehbp-missing-child",
balance=0,
reserved_balance=3_000,
reserved_at=123,
parent_key_hash="ehbp-rollback-parent",
)
session.add(parent)
session.add(child)
await session.commit()
await session.delete(child)
await session.commit()
await finalize_ehbp_max_cost_payment(
child,
session,
max_cost_for_model=3_000,
model_id="tinfoil/model",
)
updated_parent = await _api_key(session, "ehbp-rollback-parent")
assert updated_parent is not None
assert updated_parent.balance == 10_000
assert updated_parent.reserved_balance == 3_000
assert updated_parent.reserved_at == 123
assert updated_parent.total_spent == 0
assert await _api_key(session, "ehbp-missing-child") is None

View File

@@ -0,0 +1,126 @@
from contextlib import asynccontextmanager
from typing import AsyncGenerator
from unittest.mock import AsyncMock
import pytest
from sqlalchemy.ext.asyncio import AsyncEngine, create_async_engine
from sqlalchemy.pool import StaticPool
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr import wallet
from routstr.core.db import CashuTransaction
def _make_engine() -> AsyncEngine:
return create_async_engine(
"sqlite+aiosqlite://",
poolclass=StaticPool,
connect_args={"check_same_thread": False},
)
@pytest.fixture
async def refund_db(
monkeypatch: pytest.MonkeyPatch,
) -> AsyncGenerator[AsyncEngine, None]:
engine = _make_engine()
async with engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all)
@asynccontextmanager
async def create_session() -> AsyncGenerator[AsyncSession, None]:
async with AsyncSession(engine, expire_on_commit=False) as session:
yield session
monkeypatch.setattr(wallet.db, "create_session", create_session)
try:
yield engine
finally:
await engine.dispose()
async def _store(
engine: AsyncEngine,
*,
token: str,
created_at: int,
typ: str = "out",
collected: bool = False,
swept: bool = False,
) -> None:
async with AsyncSession(engine, expire_on_commit=False) as session:
session.add(
CashuTransaction(
token=token,
amount=10,
unit="sat",
type=typ,
created_at=created_at,
collected=collected,
swept=swept,
)
)
await session.commit()
async def _transactions(engine: AsyncEngine) -> dict[str, CashuTransaction]:
async with AsyncSession(engine, expire_on_commit=False) as session:
results = await session.exec(select(CashuTransaction))
return {transaction.token: transaction for transaction in results.all()}
@pytest.mark.asyncio
async def test_refund_sweep_only_processes_eligible_outbound_transactions(
refund_db: AsyncEngine, monkeypatch: pytest.MonkeyPatch
) -> None:
cutoff = 1_000
await _store(refund_db, token="eligible", created_at=cutoff - 1)
await _store(refund_db, token="too-new", created_at=cutoff)
await _store(
refund_db, token="collected", created_at=cutoff - 1, collected=True
)
await _store(refund_db, token="swept", created_at=cutoff - 1, swept=True)
await _store(refund_db, token="inbound", created_at=cutoff - 1, typ="in")
receive_token = AsyncMock()
monkeypatch.setattr(wallet, "recieve_token", receive_token)
await wallet._refund_sweep_once(cutoff)
receive_token.assert_awaited_once_with("eligible")
transactions = await _transactions(refund_db)
assert transactions["eligible"].swept is True
assert transactions["too-new"].swept is False
assert transactions["collected"].collected is True
assert transactions["collected"].swept is False
assert transactions["swept"].swept is True
assert transactions["inbound"].swept is False
@pytest.mark.asyncio
async def test_refund_sweep_persists_success_and_isolates_token_failures(
refund_db: AsyncEngine, monkeypatch: pytest.MonkeyPatch
) -> None:
cutoff = 1_000
for token in ("success", "already-spent", "temporary-failure"):
await _store(refund_db, token=token, created_at=cutoff - 1)
async def receive_token(token: str) -> None:
if token == "already-spent":
raise RuntimeError("Token already spent")
if token == "temporary-failure":
raise RuntimeError("mint unavailable")
receive = AsyncMock(side_effect=receive_token)
monkeypatch.setattr(wallet, "recieve_token", receive)
await wallet._refund_sweep_once(cutoff)
assert receive.await_count == 3
transactions = await _transactions(refund_db)
assert transactions["success"].swept is True
assert transactions["success"].collected is False
assert transactions["already-spent"].collected is True
assert transactions["already-spent"].swept is False
assert transactions["temporary-failure"].collected is False
assert transactions["temporary-failure"].swept is False

View File

@@ -0,0 +1,141 @@
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi import FastAPI
from fastapi.responses import Response
from httpx import ASGITransport, AsyncClient
from routstr import proxy as proxy_module
@pytest.fixture
def proxy_app() -> FastAPI:
app = FastAPI()
app.include_router(proxy_module.proxy_router)
return app
@pytest.mark.asyncio
async def test_attestation_get_routes_directly_to_tinfoil_provider(
monkeypatch: pytest.MonkeyPatch, proxy_app: FastAPI
) -> None:
non_tinfoil = MagicMock()
non_tinfoil.provider_type = "openai"
non_tinfoil.prepare_headers = MagicMock(return_value={})
non_tinfoil.forward_get_request = AsyncMock(
return_value=Response(status_code=404, content=b"wrong upstream")
)
tinfoil = MagicMock()
tinfoil.provider_type = "tinfoil"
tinfoil.prepare_headers = MagicMock(return_value={"accept": "application/json"})
tinfoil.forward_get_request = AsyncMock(
return_value=Response(status_code=200, content=b'{"attestation":true}')
)
monkeypatch.setattr(proxy_module, "_upstreams", [non_tinfoil, tinfoil])
async with AsyncClient(
transport=ASGITransport(app=proxy_app), # type: ignore[arg-type]
base_url="http://test",
) as client:
response = await client.get("/attestation")
assert response.status_code == 200
assert response.content == b'{"attestation":true}'
non_tinfoil.forward_get_request.assert_not_called()
tinfoil.forward_get_request.assert_awaited_once()
@pytest.mark.asyncio
async def test_tee_attestation_get_routes_directly_to_tinfoil_provider(
monkeypatch: pytest.MonkeyPatch, proxy_app: FastAPI
) -> None:
non_tinfoil = MagicMock()
non_tinfoil.provider_type = "openrouter"
non_tinfoil.prepare_headers = MagicMock(return_value={})
non_tinfoil.forward_get_request = AsyncMock(
return_value=Response(status_code=404, content=b"wrong upstream")
)
tinfoil = MagicMock()
tinfoil.provider_type = "tinfoil"
tinfoil.prepare_headers = MagicMock(return_value={"accept": "application/json"})
tinfoil.forward_get_request = AsyncMock(
return_value=Response(status_code=200, content=b'{"tee":true}')
)
monkeypatch.setattr(proxy_module, "_upstreams", [non_tinfoil, tinfoil])
async with AsyncClient(
transport=ASGITransport(app=proxy_app), # type: ignore[arg-type]
base_url="http://test",
) as client:
response = await client.get("/tee/attestation")
assert response.status_code == 200
assert response.content == b'{"tee":true}'
non_tinfoil.forward_get_request.assert_not_called()
tinfoil.forward_get_request.assert_awaited_once()
@pytest.mark.parametrize("path", ["attestation/", "tee/attestation/"])
@pytest.mark.asyncio
async def test_attestation_trailing_slash_routes_directly_to_tinfoil(
monkeypatch: pytest.MonkeyPatch, proxy_app: FastAPI, path: str
) -> None:
tinfoil = MagicMock()
tinfoil.provider_type = "tinfoil"
tinfoil.prepare_headers = MagicMock(return_value={})
tinfoil.forward_get_request = AsyncMock(return_value=Response(status_code=200))
monkeypatch.setattr(proxy_module, "_upstreams", [tinfoil])
async with AsyncClient(
transport=ASGITransport(app=proxy_app), # type: ignore[arg-type]
base_url="http://test",
) as client:
response = await client.get(f"/{path}")
assert response.status_code == 200
tinfoil.forward_get_request.assert_awaited_once()
@pytest.mark.parametrize("path", ["attestation/foo", "attestationjunk"])
@pytest.mark.asyncio
async def test_non_attestation_prefix_does_not_bypass_authentication(
monkeypatch: pytest.MonkeyPatch, proxy_app: FastAPI, path: str
) -> None:
tinfoil = MagicMock()
tinfoil.provider_type = "tinfoil"
tinfoil.forward_get_request = AsyncMock()
monkeypatch.setattr(proxy_module, "_upstreams", [tinfoil])
async with AsyncClient(
transport=ASGITransport(app=proxy_app), # type: ignore[arg-type]
base_url="http://test",
) as client:
response = await client.get(f"/{path}")
assert response.status_code == 400
assert response.json()["error"]["type"] == "invalid_model"
tinfoil.forward_get_request.assert_not_awaited()
def test_attestation_upstream_selection_is_tinfoil_only() -> None:
non_tinfoil = MagicMock(provider_type="openai")
tinfoil = MagicMock(provider_type="tinfoil")
assert proxy_module._select_unauthenticated_get_upstreams(
"attestation", [non_tinfoil, tinfoil]
) == [tinfoil]
assert proxy_module._select_unauthenticated_get_upstreams(
"tee/attestation", [non_tinfoil, tinfoil]
) == [tinfoil]
assert proxy_module._select_unauthenticated_get_upstreams(
"attestation/", [non_tinfoil, tinfoil]
) == [tinfoil]
assert proxy_module._select_unauthenticated_get_upstreams(
"attestationjunk", [non_tinfoil, tinfoil]
) == [non_tinfoil, tinfoil]

View File

@@ -0,0 +1,118 @@
from collections.abc import AsyncIterator
from pathlib import Path
from unittest.mock import AsyncMock, patch
import pytest
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlmodel import SQLModel, select
from sqlmodel.ext.asyncio.session import AsyncSession
from routstr.core.db import CashuTransaction
from routstr.wallet import refund_sweep_once
@pytest.fixture
async def session_factory(
tmp_path: Path,
) -> AsyncIterator[async_sessionmaker[AsyncSession]]:
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'refunds.db'}")
async with engine.begin() as connection:
await connection.run_sync(SQLModel.metadata.create_all)
factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
yield factory
await engine.dispose()
async def _insert(
factory: async_sessionmaker[AsyncSession], *rows: CashuTransaction
) -> None:
async with factory() as session:
session.add_all(rows)
await session.commit()
async def _load(
factory: async_sessionmaker[AsyncSession],
) -> dict[str, CashuTransaction]:
async with factory() as session:
result = await session.exec(select(CashuTransaction))
return {row.token: row for row in result.all()}
@pytest.mark.asyncio
async def test_refund_sweep_only_processes_expired_eligible_outgoing_tokens(
session_factory: async_sessionmaker[AsyncSession],
) -> None:
rows = [
CashuTransaction(
token="eligible", amount=1, unit="sat", type="out", created_at=800
),
CashuTransaction(
token="fresh", amount=1, unit="sat", type="out", created_at=950
),
CashuTransaction(
token="boundary", amount=1, unit="sat", type="out", created_at=900
),
CashuTransaction(
token="collected",
amount=1,
unit="sat",
type="out",
created_at=800,
collected=True,
),
CashuTransaction(
token="swept", amount=1, unit="sat", type="out", created_at=800, swept=True
),
CashuTransaction(
token="incoming", amount=1, unit="sat", type="in", created_at=800
),
]
await _insert(session_factory, *rows)
receive = AsyncMock()
with (
patch("routstr.wallet.db.create_session", side_effect=session_factory),
patch("routstr.wallet.settings.refund_sweep_ttl_seconds", 100),
patch("routstr.wallet.time.time", return_value=1000),
patch("routstr.wallet.recieve_token", receive),
):
await refund_sweep_once()
receive.assert_awaited_once_with("eligible")
loaded = await _load(session_factory)
assert loaded["eligible"].swept is True
assert loaded["fresh"].swept is False
assert loaded["boundary"].swept is False
assert loaded["collected"].swept is False
assert loaded["swept"].swept is True
assert loaded["incoming"].swept is False
@pytest.mark.asyncio
@pytest.mark.parametrize(
("error", "collected"),
[
(RuntimeError("token already spent"), True),
(RuntimeError("mint unavailable"), False),
],
)
async def test_refund_sweep_records_terminal_but_not_transient_failures(
session_factory: async_sessionmaker[AsyncSession], error: Exception, collected: bool
) -> None:
await _insert(
session_factory,
CashuTransaction(
token="refund", amount=1, unit="sat", type="out", created_at=800
),
)
with (
patch("routstr.wallet.db.create_session", side_effect=session_factory),
patch("routstr.wallet.settings.refund_sweep_ttl_seconds", 100),
patch("routstr.wallet.time.time", return_value=1000),
patch("routstr.wallet.recieve_token", AsyncMock(side_effect=error)),
):
await refund_sweep_once()
refund = (await _load(session_factory))["refund"]
assert refund.collected is collected
assert refund.swept is False

View File

@@ -0,0 +1,749 @@
"""Unit tests for Tinfoil direct integration.
Covers the EHBP usage-metrics header parser, the proxy header stripping, the
enclave URL override, and the TinfoilUpstreamProvider model fetching/forwarding
target logic.
"""
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from routstr.upstream.ehbp import (
_PROXY_ONLY_HEADERS,
_compute_ehbp_actual_cost,
_prepare_ehbp_upstream_headers,
_resolve_ehbp_target_url,
_strip_proxy_headers,
parse_tinfoil_usage_metrics,
)
from routstr.upstream.tinfoil import (
TinfoilModel,
TinfoilUpstreamProvider,
)
# ---------------------------------------------------------------------------
# parse_tinfoil_usage_metrics
# ---------------------------------------------------------------------------
class TestParseTinfoilUsageMetrics:
def test_full_header(self) -> None:
result = parse_tinfoil_usage_metrics("prompt=67,completion=42,total=109")
assert result == {
"prompt_tokens": 67,
"completion_tokens": 42,
"total_tokens": 109,
}
def test_without_total(self) -> None:
result = parse_tinfoil_usage_metrics("prompt=10,completion=5")
assert result == {"prompt_tokens": 10, "completion_tokens": 5}
def test_none(self) -> None:
assert parse_tinfoil_usage_metrics(None) is None
def test_empty(self) -> None:
assert parse_tinfoil_usage_metrics("") is None
def test_malformed(self) -> None:
assert parse_tinfoil_usage_metrics("garbage") is None
def test_missing_completion(self) -> None:
assert parse_tinfoil_usage_metrics("prompt=10") is None
def test_extra_whitespace(self) -> None:
result = parse_tinfoil_usage_metrics(
"prompt = 100 , completion = 200 , total = 300"
)
assert result == {
"prompt_tokens": 100,
"completion_tokens": 200,
"total_tokens": 300,
}
def test_with_model_field(self) -> None:
result = parse_tinfoil_usage_metrics(
"prompt=42,completion=10,total=52,model=llama3-3-70b"
)
assert result == {
"prompt_tokens": 42,
"completion_tokens": 10,
"total_tokens": 52,
"model": "llama3-3-70b",
}
def test_with_model_no_total(self) -> None:
result = parse_tinfoil_usage_metrics(
"prompt=67,completion=42,model=gpt-oss-120b"
)
assert result == {
"prompt_tokens": 67,
"completion_tokens": 42,
"model": "gpt-oss-120b",
}
def test_model_with_dashes_and_numbers(self) -> None:
result = parse_tinfoil_usage_metrics(
"prompt=1,completion=1,total=2,model=kimi-k2-6"
)
assert result is not None
assert result["model"] == "kimi-k2-6"
def test_model_with_extra_fields(self) -> None:
result = parse_tinfoil_usage_metrics(
"prompt=69,completion=20,total=89,"
"cached_prompt_tokens=64,uncached_prompt_tokens=5,"
"model=kimi-k2-6"
)
assert result is not None
assert result["prompt_tokens"] == 69
assert result["completion_tokens"] == 20
assert result["total_tokens"] == 89
assert result["model"] == "kimi-k2-6"
def test_old_format_still_works(self) -> None:
"""Headers without the model field (pre-PR #385) still parse."""
result = parse_tinfoil_usage_metrics(
"prompt=67,completion=42,total=109"
)
assert result == {
"prompt_tokens": 67,
"completion_tokens": 42,
"total_tokens": 109,
}
assert "model" not in result
# ---------------------------------------------------------------------------
# _strip_proxy_headers
# ---------------------------------------------------------------------------
class TestStripProxyHeaders:
def test_strips_all_proxy_only(self) -> None:
headers = {
"x-routstr-model": "tinfoil-llama3-3-70b",
"X-Tinfoil-Enclave-Url": "https://inference.tinfoil.sh",
"X-Tinfoil-Request-Usage-Metrics": "true",
"Authorization": "Bearer secret",
"Ehbp-Encapsulated-Key": "abc123",
}
clean = _strip_proxy_headers(headers)
assert "x-routstr-model" not in clean
assert "X-Tinfoil-Enclave-Url" not in clean
assert "X-Tinfoil-Request-Usage-Metrics" not in clean
assert clean["Authorization"] == "Bearer secret"
assert clean["Ehbp-Encapsulated-Key"] == "abc123"
def test_all_proxy_only_headers_covered(self) -> None:
assert _PROXY_ONLY_HEADERS == {
"x-routstr-model",
"x-tinfoil-enclave-url",
"x-tinfoil-request-usage-metrics",
}
class TestPrepareEHBPUpstreamHeaders:
def test_strips_client_proxy_headers_before_merging_target_headers(self) -> None:
headers = {
"x-routstr-model": "tinfoil-llama3-3-70b",
"X-Tinfoil-Enclave-Url": "https://enclave.tinfoil.sh",
"X-Tinfoil-Request-Usage-Metrics": "false",
"Authorization": "Bearer upstream-key",
"Ehbp-Encapsulated-Key": "abc123",
}
target_headers = {"X-Tinfoil-Request-Usage-Metrics": "true"}
clean = _prepare_ehbp_upstream_headers(headers, target_headers)
assert "x-routstr-model" not in clean
assert "X-Tinfoil-Enclave-Url" not in clean
assert clean["Authorization"] == "Bearer upstream-key"
assert clean["Ehbp-Encapsulated-Key"] == "abc123"
assert clean["X-Tinfoil-Request-Usage-Metrics"] == "true"
# ---------------------------------------------------------------------------
# _resolve_ehbp_target_url
# ---------------------------------------------------------------------------
class TestResolveEhbpTargetUrl:
def test_override_with_enclave_url_for_tinfoil(self) -> None:
result = _resolve_ehbp_target_url(
"https://default.example.com/v1/chat/completions",
"v1/chat/completions",
{"X-Tinfoil-Enclave-Url": "https://enclave.tinfoil.sh"},
"tinfoil",
)
assert result == "https://enclave.tinfoil.sh/v1/chat/completions"
def test_override_lowercase_header_for_tinfoil(self) -> None:
result = _resolve_ehbp_target_url(
"https://default.example.com/v1/chat/completions",
"v1/chat/completions",
{"x-tinfoil-enclave-url": "https://enclave.tinfoil.sh"},
"tinfoil",
)
assert result == "https://enclave.tinfoil.sh/v1/chat/completions"
def test_no_override(self) -> None:
default = "https://inference.tinfoil.sh/v1/chat/completions"
result = _resolve_ehbp_target_url(
default,
"v1/chat/completions",
{},
"tinfoil",
)
assert result == default
def test_non_tinfoil_provider_ignores_enclave_url(self) -> None:
default = "https://api.ppq.ai/private/v1/chat/completions"
result = _resolve_ehbp_target_url(
default,
"v1/chat/completions",
{"X-Tinfoil-Enclave-Url": "https://enclave.tinfoil.sh"},
"ppqai",
)
assert result == default
@pytest.mark.parametrize(
"bad_url",
[
"http://enclave.tinfoil.sh",
"https://attacker.example",
"https://tinfoil.sh.attacker.example",
"https://127.0.0.1",
"https://enclave.tinfoil.sh:8443",
"https://user:pass@enclave.tinfoil.sh",
],
)
def test_tinfoil_rejects_unsafe_enclave_url(self, bad_url: str) -> None:
from routstr.core.exceptions import UpstreamError
with pytest.raises(UpstreamError):
_resolve_ehbp_target_url(
"https://default.example.com/v1/chat/completions",
"v1/chat/completions",
{"X-Tinfoil-Enclave-Url": bad_url},
"tinfoil",
)
# ---------------------------------------------------------------------------
# _compute_ehbp_actual_cost
# ---------------------------------------------------------------------------
class TestComputeEhbpActualCost:
@pytest.mark.asyncio
async def test_no_usage_falls_back_to_max_cost(self) -> None:
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
result = await _compute_ehbp_actual_cost(None, model_obj, 100_000)
assert result["total_msats"] == 100_000
assert result["input_tokens"] == 0
assert result["output_tokens"] == 0
@pytest.mark.asyncio
async def test_usage_parsed_and_clamped(self) -> None:
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
# The actual cost from calculate_cost will be small; we just verify
# it's clamped to min_request_msat at minimum.
with patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=10,
output_msats=20,
total_msats=30,
total_usd=0.0001,
input_tokens=67,
output_tokens=42,
)
result = await _compute_ehbp_actual_cost(
"prompt=67,completion=42,total=109",
model_obj,
100_000,
)
assert result["total_msats"] == 30
assert result["total_msats"] <= 100_000
assert result["input_tokens"] == 67
assert result["output_tokens"] == 42
assert result["total_tokens"] == 109
assert result["input_msats"] == 10
assert result["output_msats"] == 20
@pytest.mark.asyncio
async def test_max_cost_data_falls_back(self) -> None:
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
with patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import MaxCostData
mock_calc.return_value = MaxCostData(
base_msats=0,
input_msats=0,
output_msats=0,
total_msats=0,
total_usd=0.0,
input_tokens=0,
output_tokens=0,
)
result = await _compute_ehbp_actual_cost(
"prompt=0,completion=0",
model_obj,
50_000,
)
assert result["total_msats"] == 50_000
assert result["input_tokens"] == 0
assert result["output_tokens"] == 0
@pytest.mark.asyncio
async def test_model_match_no_actual_model_key(self) -> None:
"""When the served model matches the requested one, no actual_model key."""
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
with patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=llama3-3-70b",
model_obj,
100_000,
)
assert "actual_model" not in result
# calculate_cost called with requested model
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "llama3-3-70b"
@pytest.mark.asyncio
async def test_alias_match_no_actual_model_key(self) -> None:
"""When the served upstream model matches forwarded_model_id through
a client-facing alias, no actual_model key is set."""
model_obj = MagicMock()
model_obj.id = "tinfoil-glm-5-2" # client-facing alias
model_obj.forwarded_model_id = "glm-5-2" # actual upstream ID
with patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
# Tinfoil header returns the actual upstream model ID
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=glm-5-2",
model_obj,
100_000,
)
assert "actual_model" not in result
# calculate_cost called with the client-facing model ID (whose
# pricing includes the correct upstream rates)
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "tinfoil-glm-5-2"
@pytest.mark.asyncio
async def test_real_mismatch_uses_actual_model_for_pricing(self) -> None:
"""When the served model differs from the expected upstream model,
the actual model's pricing is used."""
model_obj = MagicMock()
model_obj.id = "tinfoil-gpt-oss-120b" # client-facing alias
model_obj.forwarded_model_id = "gpt-oss-120b" # expected upstream
actual_model_obj = MagicMock()
actual_model_obj.id = "tinfoil-llama3-3-70b" # client-facing of actual
actual_model_obj.forwarded_model_id = "llama3-3-70b"
with patch(
"routstr.proxy.get_model_instance",
return_value=actual_model_obj,
), patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=20,
output_msats=40,
total_msats=60,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
# Tinfoil served llama3-3-70b instead of gpt-oss-120b
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=llama3-3-70b",
model_obj,
100_000,
)
assert result["actual_model"] == "llama3-3-70b"
assert result["total_msats"] == 60
# calculate_cost called with the actual model's client-facing ID
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "tinfoil-llama3-3-70b"
@pytest.mark.asyncio
async def test_model_mismatch_unknown_model_falls_back(self) -> None:
"""When the served model is not in the registry, use requested model."""
model_obj = MagicMock()
model_obj.id = "gpt-oss-120b"
model_obj.forwarded_model_id = "gpt-oss-120b"
with patch(
"routstr.proxy.get_model_instance",
return_value=None,
), patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=nonexistent",
model_obj,
100_000,
)
assert "actual_model" not in result
# calculate_cost called with the requested model (fallback)
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "gpt-oss-120b"
@pytest.mark.asyncio
async def test_old_format_no_model_uses_requested(self) -> None:
"""Old format without model field uses requested model for pricing."""
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
with patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=67,
output_tokens=42,
)
result = await _compute_ehbp_actual_cost(
"prompt=67,completion=42,total=109",
model_obj,
100_000,
)
assert "actual_model" not in result
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "llama3-3-70b"
@pytest.mark.asyncio
async def test_case_insensitive_model_match(self) -> None:
"""Casing differences between the header and forwarded_model_id
should not trigger a spurious mismatch."""
model_obj = MagicMock()
model_obj.id = "tinfoil-glm-5-2"
model_obj.forwarded_model_id = "glm-5-2" # lowercase
with patch(
"routstr.proxy.get_model_instance"
) as mock_get_model, patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
# Header returns uppercase — same model, different casing
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=GLM-5-2",
model_obj,
100_000,
)
assert "actual_model" not in result
# No mismatch: requested model pricing used
call_args = mock_calc.call_args
assert call_args[0][0]["model"] == "tinfoil-glm-5-2"
mock_get_model.assert_not_called()
@pytest.mark.asyncio
async def test_date_versioned_alias_resolves_to_requested(self) -> None:
"""When the served model is a date-versioned alias that resolves back
to the requested model, no mismatch is propagated."""
model_obj = MagicMock()
model_obj.id = "tinfoil-glm-5-2"
model_obj.forwarded_model_id = "glm-5-2"
resolved_model_obj = MagicMock()
resolved_model_obj.id = "other-provider-glm-5-2"
resolved_model_obj.forwarded_model_id = "glm-5-2"
with patch(
"routstr.proxy.get_model_instance",
return_value=resolved_model_obj,
) as mock_get_model, patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
# Tinfoil returns a date-versioned ID with different casing.
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=GLM-5-2-20260415",
model_obj,
100_000,
)
# Registry resolution, rather than unconditional suffix removal,
# establishes that this alias represents the expected model.
assert "actual_model" not in result
assert mock_calc.call_args[0][0]["model"] == "tinfoil-glm-5-2"
mock_get_model.assert_called_once_with("GLM-5-2-20260415")
@pytest.mark.asyncio
async def test_configured_date_version_is_preserved_as_identity(self) -> None:
"""A date suffix in forwarded_model_id is meaningful and preserved."""
model_obj = MagicMock()
model_obj.id = "tinfoil-glm-5-2-20260415"
model_obj.forwarded_model_id = "glm-5-2-20260415"
with patch(
"routstr.proxy.get_model_instance"
) as mock_get_model, patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=GLM-5-2-20260415",
model_obj,
100_000,
)
assert "actual_model" not in result
assert (
mock_calc.call_args[0][0]["model"]
== "tinfoil-glm-5-2-20260415"
)
mock_get_model.assert_not_called()
@pytest.mark.asyncio
async def test_different_client_alias_same_upstream_identity(self) -> None:
"""A global alias winner from another provider is not a failover when
its forwarded model ID matches the requested upstream identity."""
model_obj = MagicMock()
model_obj.id = "tinfoil-glm-5-2"
model_obj.forwarded_model_id = "glm-5-2"
resolved_model_obj = MagicMock()
resolved_model_obj.id = "other-provider-glm-5-2"
resolved_model_obj.forwarded_model_id = "GLM-5-2"
with patch(
"routstr.proxy.get_model_instance",
return_value=resolved_model_obj,
) as mock_get_model, patch(
"routstr.upstream.ehbp.calculate_cost",
new_callable=AsyncMock,
) as mock_calc:
from routstr.payment.cost_calculation import CostData
mock_calc.return_value = CostData(
base_msats=0,
input_msats=5,
output_msats=10,
total_msats=15,
total_usd=0.0,
input_tokens=42,
output_tokens=10,
)
result = await _compute_ehbp_actual_cost(
"prompt=42,completion=10,total=52,model=provider-alias",
model_obj,
100_000,
)
mock_get_model.assert_called_once_with("provider-alias")
assert "actual_model" not in result
assert mock_calc.call_args[0][0]["model"] == "tinfoil-glm-5-2"
# ---------------------------------------------------------------------------
# TinfoilUpstreamProvider
# ---------------------------------------------------------------------------
class TestTinfoilUpstreamProvider:
def test_provider_type_and_defaults(self) -> None:
assert TinfoilUpstreamProvider.provider_type == "tinfoil"
assert (
TinfoilUpstreamProvider.default_base_url
== "https://inference.tinfoil.sh"
)
assert TinfoilUpstreamProvider.supports_ehbp is True
def test_transform_model_name(self) -> None:
provider = TinfoilUpstreamProvider(api_key="test")
assert provider.transform_model_name("tinfoil/llama3-3-70b") == "llama3-3-70b"
assert provider.transform_model_name("llama3-3-70b") == "llama3-3-70b"
def test_get_ehbp_forwarding_target_includes_usage_header(self) -> None:
provider = TinfoilUpstreamProvider(api_key="test")
model_obj = MagicMock()
model_obj.id = "llama3-3-70b"
model_obj.forwarded_model_id = "llama3-3-70b"
target = provider.get_ehbp_forwarding_target("v1/chat/completions", model_obj)
assert (
target.headers["X-Tinfoil-Request-Usage-Metrics"] == "true"
)
assert "v1/chat/completions" in target.url
def test_get_provider_metadata(self) -> None:
meta = TinfoilUpstreamProvider.get_provider_metadata()
assert meta["id"] == "tinfoil"
assert meta["name"] == "Tinfoil"
assert meta["fixed_base_url"] is True
def test_tinfoil_model_pricing_parses(self) -> None:
data = {
"id": "llama3-3-70b",
"context_window": 128000,
"created": 1721764788,
"pricing": {
"inputTokenPricePer1M": 1.75,
"outputTokenPricePer1M": 2.75,
"requestPrice": 0,
},
"endpoints": ["/v1/chat/completions"],
"type": "chat",
}
tf = TinfoilModel.parse_obj(data)
assert tf.id == "llama3-3-70b"
assert tf.pricing.inputTokenPricePer1M == 1.75
assert tf.pricing.outputTokenPricePer1M == 2.75
@pytest.mark.asyncio
async def test_fetch_models_parses_response(self) -> None:
provider = TinfoilUpstreamProvider(api_key="test")
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_response.json.return_value = {
"data": [
{
"id": "llama3-3-70b",
"context_window": 128000,
"created": 1721764788,
"multimodal": False,
"pricing": {
"inputTokenPricePer1M": 1.75,
"outputTokenPricePer1M": 2.75,
"requestPrice": 0,
},
"endpoints": ["/v1/chat/completions"],
"type": "chat",
}
]
}
with patch("routstr.upstream.tinfoil.httpx.AsyncClient") as mock_client_cls:
mock_client = MagicMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_response)
mock_client_cls.return_value = mock_client
models = await provider.fetch_models()
assert len(models) == 1
assert models[0].id == "llama3-3-70b"
assert models[0].pricing.prompt == 1.75 / 1_000_000
assert models[0].pricing.completion == 2.75 / 1_000_000
assert models[0].context_length == 128000
@pytest.mark.asyncio
async def test_fetch_models_handles_error(self) -> None:
provider = TinfoilUpstreamProvider(api_key="test")
with patch("routstr.upstream.tinfoil.httpx.AsyncClient") as mock_client_cls:
mock_client = MagicMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(side_effect=Exception("network error"))
mock_client_cls.return_value = mock_client
models = await provider.fetch_models()
assert models == []

View File

@@ -0,0 +1,138 @@
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
import pytest
from routstr.upstream.tinfoil_trailer import forward_with_trailer
class FakeReader:
def __init__(self, chunks: list[bytes]) -> None:
self._chunks = chunks
async def read(self, _size: int) -> bytes:
if self._chunks:
return self._chunks.pop(0)
return b""
class FakeWriter:
def __init__(self) -> None:
self.written = b""
self.drain = AsyncMock()
self.wait_closed = AsyncMock()
self.close = MagicMock()
def write(self, data: bytes) -> None:
self.written += data
@pytest.mark.asyncio
async def test_forward_with_trailer_captures_usage_trailer(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response = (
b"HTTP/1.1 200 OK\r\n"
b"Transfer-Encoding: chunked\r\n"
b"Trailer: X-Tinfoil-Usage-Metrics\r\n"
b"\r\n"
b"5\r\nhello\r\n"
b"0\r\n"
b"X-Tinfoil-Usage-Metrics: prompt=1,completion=2,total=3\r\n"
b"\r\n"
)
reader = FakeReader([response])
writer = FakeWriter()
open_connection = AsyncMock(return_value=(reader, writer))
monkeypatch.setattr(
"routstr.upstream.tinfoil_trailer.asyncio.open_connection", open_connection
)
result = await forward_with_trailer(
method="POST",
url="https://enclave.tinfoil.sh/v1/chat/completions?stream=true",
headers={"Authorization": "Bearer upstream"},
body=b"opaque",
)
assert result.status_code == 200
assert result.body == b"hello"
assert result.trailers == [
("x-tinfoil-usage-metrics", "prompt=1,completion=2,total=3")
]
assert b"Connection: close" in writer.written
writer.close.assert_called_once()
writer.wait_closed.assert_awaited_once()
@pytest.mark.asyncio
async def test_forward_with_trailer_strips_hop_by_hop_headers(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response = b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok"
reader = FakeReader([response])
writer = FakeWriter()
monkeypatch.setattr(
"routstr.upstream.tinfoil_trailer.asyncio.open_connection",
AsyncMock(return_value=(reader, writer)),
)
await forward_with_trailer(
method="POST",
url="https://enclave.tinfoil.sh/v1/chat/completions",
headers={
"Authorization": "Bearer upstream",
"Connection": "keep-alive, X-Client-Hop",
"Keep-Alive": "timeout=5",
"Proxy-Authenticate": "Basic",
"Proxy-Authorization": "Basic secret",
"TE": "trailers",
"Trailer": "X-Usage",
"Transfer-Encoding": "chunked",
"Upgrade": "websocket",
"X-Client-Hop": "remove-me",
"X-End-To-End": "preserve-me",
},
body=b"opaque",
)
serialized_headers = writer.written.split(b"\r\n\r\n", 1)[0].lower()
for name in (
b"keep-alive",
b"proxy-authenticate",
b"proxy-authorization",
b"te:",
b"trailer:",
b"transfer-encoding",
b"upgrade:",
b"x-client-hop",
):
assert name not in serialized_headers
assert b"connection: close" in serialized_headers
assert b"content-length: 6" in serialized_headers
assert b"x-end-to-end: preserve-me" in serialized_headers
@pytest.mark.asyncio
async def test_forward_with_trailer_enforces_response_size_limit(
monkeypatch: pytest.MonkeyPatch,
) -> None:
response = b"HTTP/1.1 200 OK\r\nContent-Length: 5\r\n\r\nhello"
reader = FakeReader([response])
writer = FakeWriter()
monkeypatch.setattr(
"routstr.upstream.tinfoil_trailer.asyncio.open_connection",
AsyncMock(return_value=(reader, writer)),
)
with pytest.raises(ValueError, match="EHBP response exceeded"):
await forward_with_trailer(
method="POST",
url="https://enclave.tinfoil.sh/v1/chat/completions",
headers={},
body=b"opaque",
max_response_bytes=4,
)
writer.close.assert_called_once()

View File

@@ -86,7 +86,7 @@ export function AddModelForm({
return (
<Dialog open={isOpen} onOpenChange={handleClose}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[600px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[600px]'>
<DialogHeader>
<DialogTitle className='flex items-center gap-2'>
<Plus className='h-5 w-5' />

View File

@@ -438,7 +438,7 @@ export function AddProviderModelDialog({
return (
<Dialog open={isOpen} onOpenChange={(open) => !open && onClose()}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[720px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[720px]'>
<DialogHeader>
<DialogTitle className='flex items-center gap-2'>
<Plus className='h-4 w-4' />

View File

@@ -177,7 +177,7 @@ export function CollectModelsDialog({
return (
<Dialog open={isOpen} onOpenChange={handleClose}>
<DialogContent className='max-h-[80vh] sm:max-w-[700px]'>
<DialogContent className='max-h-[80dvh] sm:max-w-[700px]'>
<DialogHeader>
<DialogTitle className='flex items-center gap-2'>
<Download className='h-5 w-5' />

View File

@@ -25,7 +25,7 @@ export function CostCalculatorDialog({
}: CostCalculatorDialogProps) {
return (
<Dialog open={isOpen} onOpenChange={onClose}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[700px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[700px]'>
<DialogHeader>
<DialogTitle className='flex items-center gap-2'>
<Calculator className='h-5 w-5' />

View File

@@ -100,7 +100,7 @@ export function EditGroupForm({
return (
<Dialog open={isOpen} onOpenChange={handleClose}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[700px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[700px]'>
<DialogHeader>
<DialogTitle className='flex items-center gap-2'>
<Users className='h-5 w-5' />

View File

@@ -213,7 +213,7 @@ export function ProviderCard({
</CardHeader>
<Dialog open={isKeyModalOpen} onOpenChange={setIsKeyModalOpen}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[500px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[500px]'>
<DialogHeader>
<DialogTitle>
{provider.api_key

View File

@@ -53,7 +53,7 @@ export function ProviderFormDialogContent({
availableMints,
}: ProviderFormDialogContentProps) {
return (
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-[500px]'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-[500px]'>
<DialogHeader>
<DialogTitle>{title}</DialogTitle>
<DialogDescription>{description}</DialogDescription>

View File

@@ -112,7 +112,7 @@ export function ProviderBalance({
</Button>
<Dialog open={isTopupDialogOpen} onOpenChange={handleCloseDialog}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-md'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-md'>
<DialogHeader>
<DialogTitle>Top Up Balance</DialogTitle>
<DialogDescription>

View File

@@ -239,7 +239,7 @@ export function RoutstrProviderCard({
</Card>
<Dialog open={isKeyDialogOpen} onOpenChange={setIsKeyDialogOpen}>
<DialogContent className='max-h-[90vh] overflow-y-auto sm:max-w-lg'>
<DialogContent className='max-h-[90dvh] overflow-y-auto sm:max-w-lg'>
<DialogHeader>
<DialogTitle>
{hasApiKey ? 'Create New Key on Upstream Node' : 'Create API Key'}

View File

@@ -626,7 +626,7 @@ export function TopModelsUsageChart({
className={cn(
'aspect-auto w-full',
isFullscreen
? 'h-[calc(100vh-220px)] min-h-[340px] sm:h-[calc(100vh-260px)] sm:min-h-[420px]'
? 'h-[calc(100dvh-220px)] min-h-[340px] sm:h-[calc(100dvh-260px)] sm:min-h-[420px]'
: 'h-[260px] sm:h-[340px]'
)}
onMouseLeave={() => {

View File

@@ -130,7 +130,7 @@ function DialogContent({
<DrawerPrimitive.Content
data-slot='dialog-content'
className={cn(
'border-border bg-card data-closed:animate-out data-open:animate-in data-closed:slide-out-to-bottom data-open:slide-in-from-bottom fixed inset-x-0 bottom-0 z-50 grid max-h-[90vh] w-full gap-4 rounded-t-xl border-t p-4 pb-[calc(1rem+env(safe-area-inset-bottom))] shadow-lg',
'border-border bg-card data-closed:animate-out data-open:animate-in data-closed:slide-out-to-bottom data-open:slide-in-from-bottom fixed inset-x-0 bottom-0 z-50 grid max-h-[90dvh] w-full gap-4 rounded-t-xl border-t p-4 pb-[calc(1rem+env(safe-area-inset-bottom))] shadow-lg',
className
)}
{...(props as React.ComponentProps<typeof DrawerPrimitive.Content>)}

View File

@@ -56,7 +56,7 @@ function DrawerContent({
<DrawerPrimitive.Content
data-slot='drawer-content'
className={cn(
'bg-background group/drawer-content fixed z-50 flex h-auto flex-col text-sm data-[vaul-drawer-direction=bottom]:inset-x-0 data-[vaul-drawer-direction=bottom]:bottom-0 data-[vaul-drawer-direction=bottom]:mt-24 data-[vaul-drawer-direction=bottom]:max-h-[80vh] data-[vaul-drawer-direction=bottom]:rounded-t-xl data-[vaul-drawer-direction=bottom]:border-t data-[vaul-drawer-direction=bottom]:pb-[calc(0.5rem+env(safe-area-inset-bottom))] data-[vaul-drawer-direction=left]:inset-y-0 data-[vaul-drawer-direction=left]:left-0 data-[vaul-drawer-direction=left]:w-3/4 data-[vaul-drawer-direction=left]:rounded-r-xl data-[vaul-drawer-direction=left]:border-r data-[vaul-drawer-direction=right]:inset-y-0 data-[vaul-drawer-direction=right]:right-0 data-[vaul-drawer-direction=right]:w-3/4 data-[vaul-drawer-direction=right]:rounded-l-xl data-[vaul-drawer-direction=right]:border-l data-[vaul-drawer-direction=top]:inset-x-0 data-[vaul-drawer-direction=top]:top-0 data-[vaul-drawer-direction=top]:mb-24 data-[vaul-drawer-direction=top]:max-h-[80vh] data-[vaul-drawer-direction=top]:rounded-b-xl data-[vaul-drawer-direction=top]:border-b data-[vaul-drawer-direction=left]:sm:max-w-sm data-[vaul-drawer-direction=right]:sm:max-w-sm',
'bg-background group/drawer-content fixed z-50 flex h-auto flex-col text-sm data-[vaul-drawer-direction=bottom]:inset-x-0 data-[vaul-drawer-direction=bottom]:bottom-0 data-[vaul-drawer-direction=bottom]:mt-24 data-[vaul-drawer-direction=bottom]:max-h-[80dvh] data-[vaul-drawer-direction=bottom]:rounded-t-xl data-[vaul-drawer-direction=bottom]:border-t data-[vaul-drawer-direction=bottom]:pb-[calc(0.5rem+env(safe-area-inset-bottom))] data-[vaul-drawer-direction=left]:inset-y-0 data-[vaul-drawer-direction=left]:left-0 data-[vaul-drawer-direction=left]:w-3/4 data-[vaul-drawer-direction=left]:rounded-r-xl data-[vaul-drawer-direction=left]:border-r data-[vaul-drawer-direction=right]:inset-y-0 data-[vaul-drawer-direction=right]:right-0 data-[vaul-drawer-direction=right]:w-3/4 data-[vaul-drawer-direction=right]:rounded-l-xl data-[vaul-drawer-direction=right]:border-l data-[vaul-drawer-direction=top]:inset-x-0 data-[vaul-drawer-direction=top]:top-0 data-[vaul-drawer-direction=top]:mb-24 data-[vaul-drawer-direction=top]:max-h-[80dvh] data-[vaul-drawer-direction=top]:rounded-b-xl data-[vaul-drawer-direction=top]:border-b data-[vaul-drawer-direction=left]:sm:max-w-sm data-[vaul-drawer-direction=right]:sm:max-w-sm',
className
)}
{...props}

View File

@@ -291,7 +291,7 @@ export function UsageMetricsChart({
className={cn(
'aspect-auto w-full',
isFullscreen
? 'h-[calc(100vh-220px)] min-h-[340px] sm:h-[calc(100vh-260px)] sm:min-h-[420px]'
? 'h-[calc(100dvh-220px)] min-h-[340px] sm:h-[calc(100dvh-260px)] sm:min-h-[420px]'
: 'h-[260px] sm:h-[340px]'
)}
config={chartConfig}

2
uv.lock generated
View File

@@ -2392,6 +2392,7 @@ dependencies = [
{ name = "cashu" },
{ name = "fastapi", extra = ["standard"] },
{ name = "greenlet" },
{ name = "h11" },
{ name = "httpx", extra = ["socks"] },
{ name = "litellm" },
{ name = "marshmallow" },
@@ -2426,6 +2427,7 @@ requires-dist = [
{ name = "cashu", specifier = ">=0.20" },
{ name = "fastapi", extras = ["standard"], specifier = ">=0.115" },
{ name = "greenlet", specifier = ">=3.2.1" },
{ name = "h11", specifier = ">=0.14" },
{ name = "httpx", extras = ["socks"], specifier = ">=0.25.2" },
{ name = "litellm", specifier = ">=1.55.0" },
{ name = "marshmallow", specifier = ">=3.13,<4.0" },