feat: Add test suite analysis documentation

Co-authored-by: db2002dominic <db2002dominic@gmail.com>

TEST_ANALYSIS_README.md

@@ -0,0 +1,367 @@
+# Test Suite Analysis - Documentation Overview
+
+This directory contains a comprehensive analysis of the Routstr test suite, prepared for a senior pytest engineer with full understanding of the codebase.
+
+## 📋 Documents Overview
+
+### 1. [TEST_SUITE_ANALYSIS_REPORT.md](./TEST_SUITE_ANALYSIS_REPORT.md)
+**The Complete Analysis** - ~1000 lines
+
+A comprehensive, in-depth analysis of the entire test suite covering:
+- Current test coverage overview (unit + integration)
+- Critical missing test coverage (detailed analysis)
+- Tests that may not be useful
+- Outdated tests that need updating
+- Test quality issues
+- Missing test infrastructure
+- Risk assessment
+- Detailed recommendations by priority
+
+**When to use:** When you need complete context, detailed explanations, or are planning the testing strategy.
+
+---
+
+### 2. [TEST_SUITE_ACTION_PLAN.md](./TEST_SUITE_ACTION_PLAN.md)
+**The Practical Guide** - Week-by-week action plan
+
+A hands-on, actionable plan with:
+- Immediate actions (this week)
+- Week 1-2 detailed plan
+- Week 3-4 core functionality tests
+- Week 5-6 advanced features
+- Specific test scenarios to implement
+- Code examples and templates
+- Success criteria per week
+- Tools and dependencies needed
+
+**When to use:** When you're ready to start implementing tests and need concrete steps.
+
+---
+
+### 3. [TEST_COVERAGE_MATRIX.md](./TEST_COVERAGE_MATRIX.md)
+**The Visual Reference** - Quick lookup tables
+
+Visual coverage matrix with:
+- Coverage status by component (✅/❌/⚠️)
+- Risk assessment by module (🔴 high risk)
+- Functionality coverage tables
+- Critical path testing status
+- Risk heat map
+- Quick action priorities
+- Coverage by file size
+
+**When to use:** When you need a quick reference or want to see the big picture at a glance.
+
+---
+
+## 🎯 Quick Start
+
+### If you want to understand the current state:
+1. Read **Section 1** of the Analysis Report (Current Test Coverage)
+2. Review the Coverage Matrix for visual overview
+3. Check the Risk Heat Map to see critical areas
+
+### If you want to start testing:
+1. Read the Action Plan's "Immediate Actions"
+2. Follow the Week 1-2 detailed plan
+3. Use the test scenarios as templates
+4. Track progress with weekly checklists
+
+### If you're prioritizing work:
+1. Check the Coverage Matrix's Critical Risk section
+2. Review the Action Plan's priorities
+3. See the Analysis Report's Risk Assessment (Section 11)
+4. Focus on items marked 🔴 in the matrix
+
+---
+
+## 📊 Key Findings Summary
+
+### Current State
+- **Overall Coverage:** ~40%
+- **Unit Tests:** ~200 tests, 25% coverage
+- **Integration Tests:** ~100 tests, 50% coverage
+- **Critical Gaps:** Upstream providers, admin, middleware, NIP-91
+
+### Biggest Problems
+1. **Upstream Providers:** 0% unit coverage (11 files, critical)
+2. **Admin Endpoints:** 0% coverage (critical)
+3. **Cost Calculation:** No direct tests (billing logic)
+4. **Middleware:** 0% coverage (logging/tracking)
+5. **Proxy Core:** No unit tests for main routing logic
+
+### Target State
+- **Overall Coverage:** 75%+
+- **Total Tests:** ~600 tests (+450 new)
+- **Time to Complete:** 4-6 weeks (1-2 engineers)
+- **Estimated Effort:** ~15,000 lines of test code
+
+---
+
+## 🚀 Implementation Roadmap
+
+### Week 1: Critical Foundation
+- Create upstream provider test infrastructure (50+ tests)
+- Add admin endpoint tests (40+ tests)
+- Add cost calculation tests (30+ tests)
+- Audit existing tests for reserved_balance
+
+**Deliverable:** Core test infrastructure, coverage → 55%
+
+### Week 2: Core Safety
+- Complete upstream provider tests
+- Add payment atomicity tests
+- Add authentication edge cases
+- Improve test fixtures
+
+**Deliverable:** Critical paths tested, coverage → 65%
+
+### Week 3-4: Core Functionality
+- Add proxy core tests (40 tests)
+- Add middleware tests (20 tests)
+- Add concurrency test suite (30 tests)
+- Add property-based tests
+
+**Deliverable:** Core routing tested, coverage → 70%
+
+### Week 5-6: Advanced Features
+- Add NIP-91 tests (40 tests)
+- Add discovery tests (30 tests)
+- Add performance tests
+- Add contract tests
+
+**Deliverable:** Full system tested, coverage → 75%+
+
+---
+
+## 📈 Success Metrics
+
+### Coverage Targets
+- Overall: 40% → 75%
+- Critical components: >70%
+- Upstream providers: 0% → 70%
+- Admin: 0% → 80%
+- Middleware: 0% → 75%
+
+### Quality Targets
+- Pass rate: 98% → 100%
+- Flaky tests: 2% → 0%
+- Execution time: 4.5min → <5min
+- Tests added: +450 tests
+
+### Risk Reduction
+- Critical risks: 5 → 0
+- High risks: 5 → 2
+- Medium risks: 4 → 2
+- Coverage gaps: Major → Minor
+
+---
+
+## 🔧 Tools Needed
+
+### Currently Used
+- ✅ pytest
+- ✅ pytest-asyncio
+- ✅ httpx (for mocking)
+
+### Should Add
+- pytest-xdist (parallel execution)
+- pytest-benchmark (performance)
+- pytest-cov (coverage in CI)
+- hypothesis (property testing)
+- factory-boy (test data)
+- mutmut (mutation testing)
+
+---
+
+## 💡 Best Practices
+
+### Writing Tests
+1. **Unit tests:** Mock external dependencies, test single units
+2. **Integration tests:** Use real services when possible
+3. **E2E tests:** Test complete flows with minimal mocking
+4. **Test names:** Descriptive, explain what's being tested
+5. **Fixtures:** Reusable, well-documented, focused
+
+### Test Organization
+1. Mirror source code structure
+2. Use consistent naming conventions
+3. Group related tests in classes
+4. Add comprehensive docstrings
+5. Use markers for test categories
+
+### Test Quality
+1. Test happy path + error cases
+2. Test edge cases and boundaries
+3. Test concurrent operations
+4. Test with realistic data
+5. Keep tests simple and focused
+
+---
+
+## ⚠️ Critical Warnings
+
+### DO NOT:
+- ❌ Skip testing upstream providers (biggest gap)
+- ❌ Ignore admin endpoint security
+- ❌ Forget to test concurrent operations
+- ❌ Over-mock integration tests
+- ❌ Test implementation details instead of behavior
+
+### DO:
+- ✅ Focus on critical paths first
+- ✅ Test error handling thoroughly
+- ✅ Use realistic test data
+- ✅ Document test scenarios
+- ✅ Track coverage trends
+
+---
+
+## 📞 Getting Help
+
+### Questions About:
+- **Analysis findings:** See full Analysis Report
+- **What to test next:** Check Action Plan's current week
+- **Coverage gaps:** Review Coverage Matrix
+- **Test examples:** Look at existing test files
+- **Best practices:** Review Action Plan's test utilities section
+
+### Debugging Issues:
+- Test failures: Check test logs and fixtures
+- Flaky tests: Add deterministic seeds, better isolation
+- Slow tests: Use pytest-xdist for parallelization
+- Coverage not improving: Use coverage report to find gaps
+
+---
+
+## 📅 Review Schedule
+
+### Weekly Reviews
+- **Monday:** Review previous week's progress
+- **Wednesday:** Check coverage metrics
+- **Friday:** Update action plan if needed
+
+### Deliverables
+- **Week 1:** Coverage report showing 55%+
+- **Week 2:** All critical tests passing
+- **Week 4:** Core functionality at 70%+
+- **Week 6:** Target coverage (75%+) achieved
+
+---
+
+## 🎓 Learning Resources
+
+### Pytest Documentation
+- Official docs: https://docs.pytest.org/
+- Best practices: https://docs.pytest.org/en/stable/goodpractices.html
+- Fixtures: https://docs.pytest.org/en/stable/fixture.html
+
+### Testing Patterns
+- AAA pattern (Arrange-Act-Assert)
+- Given-When-Then
+- Test pyramids
+- Test doubles (mocks, stubs, fakes)
+
+### Python Testing
+- unittest.mock documentation
+- asyncio testing patterns
+- FastAPI testing guide
+
+---
+
+## 📁 File Navigation
+
+```
+/workspace/
+├── TEST_ANALYSIS_README.md          ← You are here
+├── TEST_SUITE_ANALYSIS_REPORT.md    ← Full detailed analysis
+├── TEST_SUITE_ACTION_PLAN.md        ← Week-by-week action plan
+├── TEST_COVERAGE_MATRIX.md          ← Visual coverage matrix
+│
+├── tests/
+│   ├── unit/                        ← Unit tests (expand these)
+│   │   ├── test_algorithm.py        ✅ Good coverage
+│   │   ├── test_fee_consistency.py  ✅ Good coverage
+│   │   ├── test_image_tokens.py     ✅ Excellent
+│   │   ├── test_payment_helpers.py  ⚠️ Partial
+│   │   ├── test_settings.py         ⚠️ Minimal
+│   │   └── test_wallet.py           ✔️ Good
+│   │
+│   └── integration/                 ← Integration tests
+│       ├── test_wallet_*.py         ✅ Excellent
+│       ├── test_proxy_*.py          ✔️ Good
+│       ├── test_provider_*.py       ✔️ Good
+│       └── [Add more here]          ← Focus area
+│
+└── routstr/                         ← Source code
+    ├── core/                        ← Core functionality
+    │   ├── admin.py                 ❌ NO TESTS
+    │   ├── middleware.py            ❌ NO TESTS
+    │   └── [test these]
+    │
+    ├── upstream/                    ← Upstream providers
+    │   ├── base.py                  ❌ NO TESTS
+    │   ├── openai.py                ❌ NO TESTS
+    │   └── [11 files, all untested] ← CRITICAL GAP
+    │
+    ├── payment/
+    │   ├── cost_caculation.py       ❌ NO DIRECT TESTS
+    │   └── [test these]
+    │
+    └── [More files to test]
+```
+
+---
+
+## 🎯 Next Steps
+
+1. **Read the appropriate document for your need**
+   - Planning? → Analysis Report
+   - Implementing? → Action Plan
+   - Quick lookup? → Coverage Matrix
+
+2. **Start with Week 1 of the Action Plan**
+   - Upstream provider tests (CRITICAL)
+   - Admin endpoint tests (CRITICAL)
+   - Cost calculation tests (HIGH)
+
+3. **Track your progress**
+   - Update Coverage Matrix weekly
+   - Check off Action Plan items
+   - Monitor coverage metrics
+
+4. **Ask for help when needed**
+   - Review existing test files for patterns
+   - Check documentation for tools
+   - Discuss complex scenarios with team
+
+---
+
+## 📝 Document Maintenance
+
+These documents should be updated:
+- **Weekly:** During active test expansion
+- **Monthly:** After test expansion complete
+- **Quarterly:** For major codebase changes
+
+**Last Updated:** 2025-11-16  
+**Next Review:** End of Week 1 (2025-11-23)  
+**Prepared By:** AI Analysis System  
+**Target Audience:** Senior Pytest Engineer
+
+---
+
+## ✅ Checklist Before Starting
+
+- [ ] Read this overview document
+- [ ] Review Coverage Matrix for critical gaps
+- [ ] Read Week 1 of Action Plan
+- [ ] Set up development environment
+- [ ] Install recommended tools (pytest-xdist, etc.)
+- [ ] Review existing test patterns
+- [ ] Create feature branch for tests
+- [ ] Begin with upstream provider tests
+
+---
+
+**Ready to start? Go to [TEST_SUITE_ACTION_PLAN.md](./TEST_SUITE_ACTION_PLAN.md) → "Week 1-2 Detailed Plan"**

TEST_COVERAGE_MATRIX.md

@@ -0,0 +1,308 @@
+# Test Coverage Matrix
+
+Quick visual reference for test coverage across the Routstr codebase.
+
+## Legend
+- ✅ **Excellent:** >80% coverage, comprehensive tests
+- ✔️ **Good:** 60-80% coverage, most scenarios covered
+- ⚠️ **Partial:** 30-60% coverage, major gaps exist
+- ❌ **Critical Gap:** <30% coverage or no tests
+- 🔴 **High Risk:** Untested critical functionality
+
+---
+
+## Core Components
+
+| Component | File | Unit Tests | Integration Tests | Status | Risk |
+|-----------|------|------------|-------------------|--------|------|
+| **Proxy Router** | `proxy.py` | ❌ None | ⚠️ Indirect | ❌ | 🔴 |
+| Algorithm | `algorithm.py` | ✅ Excellent | ⚠️ Indirect | ✅ | ✅ |
+| Authentication | `auth.py` | ⚠️ Partial | ✔️ Good | ⚠️ | ⚠️ |
+| Balance | `balance.py` | ❌ None | ✔️ Good | ⚠️ | ⚠️ |
+| Discovery | `discovery.py` | ❌ None | ⚠️ Mocked | ❌ | ⚠️ |
+| NIP-91 | `nip91.py` | ❌ None | ❌ None | ❌ | 🔴 |
+
+## Core Modules
+
+| Module | File | Unit Tests | Integration Tests | Status | Risk |
+|--------|------|------------|-------------------|--------|------|
+| **Admin** | `core/admin.py` | ❌ None | ❌ None | ❌ | 🔴 |
+| Database | `core/db.py` | ⚠️ Partial | ✔️ Good | ✔️ | ✅ |
+| Settings | `core/settings.py` | ⚠️ Minimal | ⚠️ Basic | ⚠️ | ⚠️ |
+| Logging | `core/logging.py` | ❌ None | ❌ None | ❌ | ⚠️ |
+| **Middleware** | `core/middleware.py` | ❌ None | ❌ None | ❌ | 🔴 |
+| Main App | `core/main.py` | ❌ None | ✔️ Indirect | ⚠️ | ✅ |
+| Exceptions | `core/exceptions.py` | ❌ None | ⚠️ Indirect | ⚠️ | ✅ |
+
+## Payment System
+
+| Component | File | Unit Tests | Integration Tests | Status | Risk |
+|-----------|------|------------|-------------------|--------|------|
+| **Cost Calculation** | `payment/cost_caculation.py` | ❌ None | ⚠️ Indirect | ❌ | 🔴 |
+| Payment Helpers | `payment/helpers.py` | ✔️ Partial | ⚠️ Indirect | ⚠️ | ⚠️ |
+| Models | `payment/models.py` | ✔️ Good | ⚠️ Indirect | ✔️ | ✅ |
+| Price | `payment/price.py` | ❌ None | ⚠️ Mocked | ⚠️ | ⚠️ |
+| LNURL | `payment/lnurl.py` | ❌ None | ⚠️ Indirect | ⚠️ | ⚠️ |
+
+## Upstream Providers
+
+| Provider | File | Unit Tests | Integration Tests | Status | Risk |
+|----------|------|------------|-------------------|--------|------|
+| **Base Provider** | `upstream/base.py` | ❌ None | ⚠️ Indirect | ❌ | 🔴 |
+| **OpenAI** | `upstream/openai.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Anthropic** | `upstream/anthropic.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Azure** | `upstream/azure.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Groq** | `upstream/groq.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Perplexity** | `upstream/perplexity.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **XAI** | `upstream/xai.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Fireworks** | `upstream/fireworks.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **OpenRouter** | `upstream/openrouter.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Ollama** | `upstream/ollama.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| **Generic** | `upstream/generic.py` | ❌ None | ⚠️ Mocked | ❌ | 🔴 |
+| Helpers | `upstream/helpers.py` | ❌ None | ⚠️ Indirect | ❌ | ⚠️ |
+
+## Wallet & Authentication
+
+| Component | Functionality | Unit Tests | Integration Tests | Status | Risk |
+|-----------|--------------|------------|-------------------|--------|------|
+| Wallet Core | `wallet.py` | ✔️ Good | ✔️ Good | ✔️ | ✅ |
+| API Key Gen | Authentication | ⚠️ Partial | ✅ Excellent | ✔️ | ✅ |
+| Topup | Wallet balance | ⚠️ Partial | ✅ Excellent | ✔️ | ✅ |
+| Refund | Wallet balance | ⚠️ Partial | ✅ Excellent | ✔️ | ✅ |
+| Token Validation | Authentication | ⚠️ Partial | ✔️ Good | ⚠️ | ⚠️ |
+| Payment Flow | Balance deduction | ⚠️ Partial | ✔️ Good | ⚠️ | ⚠️ |
+| Cashu Integration | Token handling | ✔️ Mocked | ⚠️ Fallback | ⚠️ | ⚠️ |
+
+---
+
+## Functionality Coverage
+
+### Authentication & Authorization
+| Function | Unit | Integration | Notes |
+|----------|------|-------------|-------|
+| `validate_bearer_key()` | ❌ | ✔️ | Only integration tested |
+| Bearer token auth | ❌ | ✅ | Well covered in integration |
+| Cashu token auth | ⚠️ | ✔️ | Mocked in unit, fallback in integration |
+| API key creation | ❌ | ✅ | Excellent integration coverage |
+| Key expiry handling | ❌ | ⚠️ | Limited coverage |
+| Refund address | ❌ | ⚠️ | Partial coverage |
+
+### Payment & Billing
+| Function | Unit | Integration | Notes |
+|----------|------|-------------|-------|
+| `pay_for_request()` | ❌ | ⚠️ | Indirect testing only |
+| `revert_pay_for_request()` | ❌ | ⚠️ | Limited coverage |
+| `adjust_payment_for_tokens()` | ❌ | ⚠️ | Complex logic not unit tested |
+| `calculate_cost()` | ❌ | ⚠️ | No direct tests |
+| `get_max_cost_for_model()` | ✔️ | ⚠️ | Basic unit tests |
+| Reserved balance handling | ❌ | ⚠️ | New feature, needs tests |
+| Concurrent payments | ❌ | ❌ | Critical gap |
+
+### Routing & Proxy
+| Function | Unit | Integration | Notes |
+|----------|------|-------------|-------|
+| `initialize_upstreams()` | ❌ | ⚠️ | Indirect only |
+| `refresh_model_maps()` | ❌ | ⚠️ | Background task not tested |
+| `create_model_mappings()` | ✅ | ❌ | Good unit coverage |
+| `should_prefer_model()` | ✅ | ❌ | Good unit coverage |
+| `proxy()` handler | ❌ | ⚠️ | Main proxy logic not unit tested |
+| Model resolution | ✔️ | ⚠️ | Algorithm tested, routing not |
+| Provider selection | ✔️ | ⚠️ | Algorithm tested, integration partial |
+
+### Discovery & NIP-91
+| Function | Unit | Integration | Notes |
+|----------|------|-------------|-------|
+| `query_nostr_relay_for_providers()` | ❌ | ⚠️ | Mocked in integration |
+| `parse_provider_announcement()` | ❌ | ⚠️ | Indirect coverage |
+| `create_nip91_event()` | ❌ | ❌ | No tests |
+| `publish_to_relay()` | ❌ | ❌ | No tests |
+| `announce_provider()` | ❌ | ❌ | Background task not tested |
+| `fetch_provider_health()` | ❌ | ⚠️ | Mocked in integration |
+
+### Admin & Management
+| Endpoint | Unit | Integration | Notes |
+|----------|------|-------------|-------|
+| List models | ❌ | ❌ | No coverage |
+| Update model | ❌ | ❌ | No coverage |
+| Disable model | ❌ | ❌ | No coverage |
+| List providers | ❌ | ❌ | No coverage |
+| Add provider | ❌ | ❌ | No coverage |
+| Update provider | ❌ | ❌ | No coverage |
+| Delete provider | ❌ | ❌ | No coverage |
+| Update settings | ❌ | ❌ | No coverage |
+
+---
+
+## Test Quality Metrics
+
+### Unit Tests
+| Metric | Current | Target | Gap |
+|--------|---------|--------|-----|
+| Total unit tests | ~200 | ~450 | +250 tests |
+| Avg assertions/test | 2.5 | 3.5 | +1 |
+| Avg lines/test | 15 | 20 | +5 |
+| Fixtures used | 15 | 30 | +15 |
+| Mocking quality | Fair | Good | Improve |
+
+### Integration Tests
+| Metric | Current | Target | Gap |
+|--------|---------|--------|-----|
+| Total integration tests | ~100 | ~150 | +50 tests |
+| Real services used | Few | More | Improve |
+| Mock overuse | High | Medium | Reduce |
+| E2E scenarios | 0 | 10 | +10 tests |
+
+### Coverage
+| Area | Current | Target | Priority |
+|------|---------|--------|----------|
+| **Overall** | **40%** | **75%** | **High** |
+| Wallet | 85% | 90% | Low |
+| Authentication | 65% | 85% | Medium |
+| Proxy/Routing | 35% | 80% | **Critical** |
+| Upstream Providers | 20% | 65% | **Critical** |
+| Payment/Cost | 50% | 75% | High |
+| Discovery | 30% | 70% | Medium |
+| Admin | 0% | 80% | **Critical** |
+| NIP-91 | 0% | 60% | High |
+| Middleware | 0% | 75% | **Critical** |
+
+---
+
+## Critical Path Testing Status
+
+### Request Flow (Authenticated)
+```
+Request → Middleware → Auth → Routing → Upstream → Response → Payment
+   ❌         ❌         ⚠️      ❌         ❌          ⚠️        ⚠️
+```
+
+### Request Flow (Cashu)
+```
+Request → Middleware → Cashu Validate → Routing → Upstream → Response → Payment
+   ❌         ❌            ⚠️              ❌         ❌          ⚠️        ⚠️
+```
+
+### Wallet Operations
+```
+Create Key → Validate → Topup → Pay → Refund
+    ✅          ⚠️       ✅     ⚠️     ✅
+```
+
+### Model Selection
+```
+Request → Parse Model → Find Provider → Check Pricing → Route
+   ⚠️         ⚠️            ❌              ❌           ❌
+```
+
+---
+
+## Risk Heat Map
+
+### 🔴 Critical Risk (Test Immediately)
+1. **Upstream Provider Logic** - 0% unit coverage, 11 files
+2. **Admin Endpoints** - 0% coverage, critical functionality
+3. **Proxy Core Routing** - No unit tests for main handler
+4. **Middleware** - No tests for logging/tracking
+5. **Cost Calculation** - No direct tests for billing logic
+
+### ⚠️ High Risk (Test This Sprint)
+6. **Payment Atomicity** - Race conditions not tested
+7. **NIP-91 Discovery** - Background tasks not tested
+8. **Reserved Balance** - New feature needs thorough testing
+9. **Concurrent Operations** - Limited concurrency testing
+10. **Error Handling** - Edge cases not systematically tested
+
+### ✅ Low Risk (Maintain Coverage)
+11. Wallet operations - Good coverage
+12. Algorithm logic - Well tested
+13. API key generation - Comprehensive tests
+14. Token validation - Decent coverage
+15. Basic integration flows - Working
+
+---
+
+## Test Execution Statistics
+
+### Current Performance
+- **Total Tests:** ~300
+- **Unit Tests:** ~200 (30 seconds)
+- **Integration Tests:** ~100 (4 minutes)
+- **Total Time:** ~4.5 minutes
+- **Pass Rate:** 98%+
+- **Flaky Tests:** ~2%
+
+### Target Performance
+- **Total Tests:** ~600
+- **Unit Tests:** ~450 (<1 minute with parallel)
+- **Integration Tests:** ~150 (<4 minutes)
+- **Total Time:** <5 minutes
+- **Pass Rate:** 100%
+- **Flaky Tests:** 0%
+
+---
+
+## Quick Action Priorities
+
+### Week 1 (Must Do)
+1. ✅ Create upstream provider test infrastructure
+2. ✅ Add admin endpoint tests
+3. ✅ Add cost calculation tests
+4. ✅ Audit reserved balance handling
+
+### Week 2 (Should Do)
+5. Add proxy core unit tests
+6. Add middleware tests
+7. Add auth edge case tests
+8. Add concurrency test suite
+
+### Week 3-4 (Important)
+9. Add NIP-91 tests
+10. Add discovery tests
+11. Improve test fixtures
+12. Add property-based tests
+
+### Week 5-6 (Nice to Have)
+13. Add performance tests
+14. Add contract tests
+15. Add mutation testing
+16. Improve test documentation
+
+---
+
+## Coverage by File Size
+
+| File | LOC | Tests | Coverage | Tests Needed |
+|------|-----|-------|----------|--------------|
+| `proxy.py` | 300+ | 0 unit | 35% | ~40 tests |
+| `nip91.py` | 575 | 0 | 0% | ~40 tests |
+| `auth.py` | 662 | 4 unit | 65% | ~30 tests |
+| `discovery.py` | 400 | 0 unit | 30% | ~30 tests |
+| `algorithm.py` | 301 | 15 unit | 85% | ~5 tests |
+| `balance.py` | 244 | 0 unit | 70% | ~20 tests |
+| `middleware.py` | 127 | 0 | 0% | ~20 tests |
+| `cost_caculation.py` | 157 | 0 | 50% | ~30 tests |
+| `price.py` | 163 | 0 | 30% | ~15 tests |
+
+**Total new tests needed:** ~450 tests  
+**Estimated effort:** 4-6 weeks (1-2 engineers)
+
+---
+
+## Testing Tools Status
+
+| Tool | Status | Usage | Notes |
+|------|--------|-------|-------|
+| pytest | ✅ Used | Core test runner | Working well |
+| pytest-asyncio | ✅ Used | Async testing | Working well |
+| pytest-cov | ⚠️ Partial | Coverage reports | Not in CI |
+| pytest-xdist | ❌ Not used | Parallel execution | Should add |
+| pytest-benchmark | ❌ Not used | Performance testing | Should add |
+| hypothesis | ❌ Not used | Property testing | Should add |
+| factory-boy | ❌ Not used | Test data | Should add |
+| mutmut | ❌ Not used | Mutation testing | Should add |
+
+---
+
+**Last Updated:** 2025-11-16  
+**Review Schedule:** Weekly during test expansion  
+**Target Completion:** End of Week 6

TEST_SUITE_ACTION_PLAN.md

@@ -0,0 +1,479 @@
+# Test Suite Action Plan - Quick Reference
+
+## Immediate Actions (This Week)
+
+### 1. Create Upstream Provider Test Foundation
+**Priority:** CRITICAL  
+**Effort:** 2 days  
+**Files to create:**
+- `tests/unit/upstream/__init__.py`
+- `tests/unit/upstream/test_base_provider.py`
+- `tests/unit/upstream/test_openai_provider.py`
+- `tests/unit/upstream/test_anthropic_provider.py`
+
+**Test scenarios:**
+```python
+# test_openai_provider.py
+def test_prepare_headers_with_api_key()
+def test_prepare_headers_removes_cashu_headers()
+def test_transform_request_body()
+def test_handle_streaming_response()
+def test_handle_error_response_401()
+def test_handle_error_response_429()
+def test_model_list_caching()
+```
+
+### 2. Add Admin Endpoint Tests  
+**Priority:** CRITICAL  
+**Effort:** 2 days  
+**File to create:**
+- `tests/integration/test_admin_endpoints.py`
+
+**Test scenarios:**
+```python
+def test_admin_list_models()
+def test_admin_update_model_pricing()
+def test_admin_disable_model()
+def test_admin_enable_model()
+def test_admin_add_upstream_provider()
+def test_admin_update_upstream_provider()
+def test_admin_delete_upstream_provider()
+def test_admin_unauthorized_access()
+def test_admin_update_settings()
+```
+
+### 3. Add Cost Calculation Unit Tests
+**Priority:** HIGH  
+**Effort:** 1 day  
+**File to create:**
+- `tests/unit/test_cost_calculation.py`
+
+**Test scenarios:**
+```python
+def test_calculate_cost_with_usage_data()
+def test_calculate_cost_without_usage_data()
+def test_calculate_cost_fixed_pricing_mode()
+def test_calculate_cost_dynamic_pricing_mode()
+def test_calculate_cost_invalid_model()
+def test_calculate_cost_missing_pricing()
+def test_calculate_cost_zero_tokens()
+def test_calculate_cost_large_tokens()
+def test_calculate_cost_rounding()
+```
+
+### 4. Audit Existing Tests for Reserved Balance
+**Priority:** HIGH  
+**Effort:** 1 day  
+**Files to update:**
+- `tests/integration/test_wallet_topup.py`
+- `tests/integration/test_proxy_post_endpoints.py`
+- `tests/integration/test_wallet_authentication.py`
+
+**Changes needed:**
+```python
+# BEFORE:
+assert wallet_response.json()["balance"] == expected_balance
+
+# AFTER:
+wallet_data = wallet_response.json()
+assert wallet_data["balance"] == expected_available_balance
+assert wallet_data["reserved"] == expected_reserved_balance
+# Verify total_balance = balance + reserved
+```
+
+---
+
+## Week 1-2 Detailed Plan
+
+### Day 1-2: Upstream Provider Tests
+1. Create test infrastructure:
+   ```bash
+   mkdir -p tests/unit/upstream
+   touch tests/unit/upstream/__init__.py
+   touch tests/unit/upstream/test_base_provider.py
+   ```
+
+2. Write base provider tests (20 tests):
+   - Test header preparation
+   - Test request transformation
+   - Test response handling
+   - Test error mapping
+   - Test streaming support
+
+3. Write OpenAI provider tests (15 tests):
+   - Test API key handling
+   - Test model name transformation
+   - Test response structure validation
+   - Test error codes mapping
+
+4. Write Anthropic provider tests (15 tests):
+   - Test x-api-key header
+   - Test Claude-specific transformations
+   - Test streaming format
+
+### Day 3-4: Admin Tests
+1. Create admin test file:
+   ```bash
+   touch tests/integration/test_admin_endpoints.py
+   ```
+
+2. Write model management tests (15 tests):
+   - CRUD operations
+   - Validation tests
+   - Permission tests
+
+3. Write provider management tests (15 tests):
+   - Add/update/delete providers
+   - Fee configuration
+   - Provider URL validation
+
+4. Write settings management tests (10 tests):
+   - Update settings
+   - Validation
+   - Database vs env precedence
+
+### Day 5: Cost Calculation Tests
+1. Create cost calculation test file:
+   ```bash
+   touch tests/unit/test_cost_calculation.py
+   ```
+
+2. Write comprehensive tests (30 tests):
+   - All pricing modes
+   - Edge cases
+   - Rounding behavior
+   - Error scenarios
+
+---
+
+## Week 3-4: Core Functionality
+
+### Proxy Core Tests (40 tests)
+**File:** `tests/unit/test_proxy_core.py`
+
+```python
+# Model mapping tests
+def test_initialize_upstreams_success()
+def test_initialize_upstreams_empty_db()
+def test_initialize_upstreams_invalid_provider()
+def test_refresh_model_maps_with_overrides()
+def test_refresh_model_maps_with_disabled_models()
+def test_get_model_instance_existing()
+def test_get_model_instance_missing()
+def test_get_provider_for_model_existing()
+def test_get_provider_for_model_missing()
+
+# Routing tests
+def test_route_to_cheapest_provider()
+def test_route_with_exact_model_match()
+def test_route_with_alias_match()
+def test_route_with_provider_penalty()
+def test_route_with_disabled_model()
+
+# Error handling tests
+def test_invalid_model_error_response()
+def test_no_provider_error_response()
+def test_upstream_failure_handling()
+def test_concurrent_model_refresh()
+```
+
+### Middleware Tests (20 tests)
+**File:** `tests/unit/test_middleware.py`
+
+```python
+def test_request_id_generation()
+def test_request_id_propagation()
+def test_request_logging_structure()
+def test_response_logging_structure()
+def test_error_logging()
+def test_request_timing()
+def test_context_variable_lifecycle()
+def test_sensitive_data_redaction()
+def test_body_size_logging()
+def test_header_filtering()
+```
+
+### Concurrency Tests (30 tests)
+**File:** `tests/integration/test_concurrency.py`
+
+```python
+# Payment concurrency
+def test_concurrent_payments_same_key()
+def test_concurrent_topups_same_key()
+def test_concurrent_refunds()
+def test_payment_reversal_race_condition()
+
+# Cache concurrency
+def test_concurrent_model_map_refresh()
+def test_concurrent_provider_cache_access()
+def test_concurrent_settings_updates()
+
+# Database concurrency
+def test_concurrent_balance_updates()
+def test_concurrent_api_key_creation()
+def test_atomic_payment_with_concurrent_topup()
+```
+
+---
+
+## Week 5-6: Discovery & Advanced Features
+
+### NIP-91 Tests (40 tests)
+**File:** `tests/unit/test_nip91.py`
+
+```python
+# Event creation tests
+def test_create_nip91_event_basic()
+def test_create_nip91_event_with_mints()
+def test_create_nip91_event_signing()
+
+# Parsing tests
+def test_parse_valid_nip91_event()
+def test_parse_invalid_nip91_event()
+def test_events_semantically_equal()
+
+# Relay interaction tests (mocked)
+def test_query_nip91_events_success()
+def test_query_nip91_events_timeout()
+def test_publish_to_relay_success()
+def test_publish_to_relay_failure()
+def test_publish_with_backoff()
+
+# Discovery tests
+def test_discover_onion_url_from_tor()
+def test_discover_onion_url_not_found()
+def test_determine_provider_id()
+```
+
+### Discovery Tests (30 tests)
+**File:** `tests/unit/test_discovery.py`
+
+```python
+# Provider parsing
+def test_parse_provider_announcement_nip91()
+def test_parse_provider_announcement_invalid()
+def test_parse_provider_announcement_missing_fields()
+
+# Cache management
+def test_refresh_providers_cache()
+def test_providers_cache_concurrent_access()
+def test_providers_cache_expiry()
+
+# Health checks
+def test_fetch_provider_health_onion()
+def test_fetch_provider_health_clearnet()
+def test_fetch_provider_health_timeout()
+def test_fetch_provider_health_with_tor_proxy()
+```
+
+---
+
+## Test Quality Improvements
+
+### Add Test Markers
+**File:** `pyproject.toml` or `pytest.ini`
+
+```ini
+[tool.pytest.ini_options]
+markers =
+    unit: Unit tests
+    integration: Integration tests
+    e2e: End-to-end tests
+    slow: Slow-running tests
+    concurrency: Concurrency tests
+    security: Security-related tests
+    wallet: Wallet component tests
+    proxy: Proxy component tests
+    auth: Authentication tests
+    admin: Admin functionality tests
+    discovery: Provider discovery tests
+    payment: Payment/billing tests
+```
+
+### Improve Test Fixtures
+**File:** `tests/conftest.py`
+
+```python
+# Split TestmintWallet into smaller fixtures
+@pytest.fixture
+def mint_url():
+    """Get configured mint URL"""
+    return os.environ.get("MINT", "http://localhost:3338")
+
+@pytest.fixture
+def cashu_token_generator(mint_url):
+    """Factory for generating cashu tokens"""
+    def _generate(amount: int) -> str:
+        # Simplified token generation
+        ...
+    return _generate
+
+@pytest.fixture
+def mock_upstream_provider():
+    """Factory for creating mock upstream providers"""
+    def _create(provider_type: str, **kwargs) -> Mock:
+        # Create mock provider with sensible defaults
+        ...
+    return _create
+```
+
+### Add Test Utilities
+**File:** `tests/utils/factories.py`
+
+```python
+"""Test data factories"""
+from factory import Factory, Faker
+from factory.alchemy import SQLAlchemyModelFactory
+
+class ApiKeyFactory(SQLAlchemyModelFactory):
+    class Meta:
+        model = ApiKey
+        sqlalchemy_session = None
+    
+    hashed_key = Faker('sha256')
+    balance = 10000000
+    reserved_balance = 0
+    total_spent = 0
+    total_requests = 0
+
+class ModelRowFactory(SQLAlchemyModelFactory):
+    class Meta:
+        model = ModelRow
+        sqlalchemy_session = None
+    
+    id = Faker('uuid4')
+    name = Faker('word')
+    enabled = True
+    pricing = '{"prompt": 0.001, "completion": 0.002}'
+```
+
+---
+
+## Metrics & Monitoring
+
+### Coverage Targets
+```bash
+# Add to CI pipeline
+pytest --cov=routstr --cov-report=html --cov-report=term --cov-fail-under=75
+
+# Per-module targets
+routstr/upstream/: 70%
+routstr/core/admin.py: 80%
+routstr/payment/cost_caculation.py: 80%
+routstr/proxy.py: 75%
+routstr/auth.py: 80%
+```
+
+### Performance Monitoring
+```python
+# Add to tests
+@pytest.mark.benchmark
+def test_model_mapping_performance(benchmark):
+    result = benchmark(refresh_model_maps)
+    assert result < 0.1  # Should complete in <100ms
+```
+
+### Test Execution Time
+```bash
+# Current target: All tests < 5 minutes
+# Unit tests: < 30 seconds
+# Integration tests: < 4 minutes
+```
+
+---
+
+## Tools & Dependencies
+
+### Add to requirements.txt
+```
+pytest-benchmark==4.0.0  # Performance testing
+pytest-xdist==3.3.1  # Parallel test execution
+factory-boy==3.3.0  # Test data factories
+hypothesis==6.92.0  # Property-based testing
+mutmut==2.4.4  # Mutation testing
+pytest-timeout==2.2.0  # Prevent hanging tests
+pytest-asyncio==0.23.0  # Already present, ensure latest
+```
+
+### Configure pytest-xdist
+```bash
+# Run tests in parallel
+pytest -n auto
+
+# Or specify workers
+pytest -n 4
+```
+
+---
+
+## Weekly Review Checklist
+
+### End of Week 1
+- [ ] Upstream provider tests created (50+ tests)
+- [ ] Admin endpoint tests created (40+ tests)
+- [ ] Cost calculation tests created (30+ tests)
+- [ ] Existing tests audited for reserved_balance
+- [ ] Coverage increased from 40% to ~55%
+
+### End of Week 2
+- [ ] All CRITICAL tests completed
+- [ ] CI pipeline updated with coverage checks
+- [ ] Test documentation improved
+- [ ] Coverage increased from 55% to ~65%
+
+### End of Week 4
+- [ ] Proxy core tests completed (40 tests)
+- [ ] Middleware tests completed (20 tests)
+- [ ] Concurrency tests completed (30 tests)
+- [ ] Coverage increased from 65% to ~70%
+
+### End of Week 6
+- [ ] NIP-91 tests completed (40 tests)
+- [ ] Discovery tests completed (30 tests)
+- [ ] Property-based tests added (10 tests)
+- [ ] Coverage target achieved: 75%+
+- [ ] All high-priority gaps closed
+
+---
+
+## Success Criteria
+
+### Code Quality
+- [ ] All tests pass consistently
+- [ ] No flaky tests
+- [ ] Test execution time < 5 minutes
+- [ ] Coverage > 75% overall
+- [ ] All CRITICAL components > 70% coverage
+
+### Documentation
+- [ ] All test modules have docstrings
+- [ ] Test scenarios are clearly documented
+- [ ] Test data is well-documented
+- [ ] README updated with testing guide
+
+### Automation
+- [ ] Coverage reports in CI
+- [ ] Automated test quality checks
+- [ ] Performance regression detection
+- [ ] Test result trends tracked
+
+---
+
+## Getting Help
+
+### Questions?
+- Review full analysis: `TEST_SUITE_ANALYSIS_REPORT.md`
+- Check existing tests for patterns
+- Ask team for test data sources
+- Review pytest documentation
+
+### Blocked?
+- Can't mock something? Consider using real service
+- Test too slow? Use pytest-xdist
+- Test too complex? Split into smaller tests
+- Coverage not improving? Use coverage report to find gaps
+
+---
+
+**Last Updated:** 2025-11-16  
+**Next Review:** End of Week 1

TEST_SUITE_ANALYSIS_REPORT.md

@@ -0,0 +1,808 @@
+# Test Suite Analysis Report
+
+**Date:** 2025-11-16  
+**Codebase:** Routstr - Privacy-preserving AI Proxy  
+**Prepared for:** Senior Pytest Engineer
+
+---
+
+## Executive Summary
+
+This report provides a comprehensive analysis of the test suite for the Routstr codebase. The analysis reveals that while the test suite has good coverage for wallet and authentication flows, there are significant gaps in testing core routing logic, upstream providers, middleware, admin functionality, and NIP-91 discovery features.
+
+### Key Findings:
+- **Test Coverage:** ~40% of critical codebase functionality
+- **Missing Critical Tests:** Upstream providers, algorithm logic, middleware, admin endpoints
+- **Outdated Tests:** Some integration tests don't reflect current proxy payment flow
+- **Recommended Action:** Expand test coverage by ~60% with focus on core routing and provider logic
+
+---
+
+## 1. Current Test Coverage Overview
+
+### 1.1 Unit Tests (`tests/unit/`)
+
+| Test File | Lines | Coverage Status | Notes |
+|-----------|-------|----------------|-------|
+| `test_algorithm.py` | 200 | ✅ Excellent | Tests model prioritization algorithm thoroughly |
+| `test_fee_consistency.py` | 156 | ✅ Good | Tests model-to-row payload conversion and fee handling |
+| `test_image_tokens.py` | 190 | ✅ Excellent | Comprehensive image token calculation tests |
+| `test_payment_helpers.py` | 128 | ⚠️ Partial | Only tests `get_max_cost_for_model` function |
+| `test_settings.py` | 38 | ⚠️ Minimal | Basic settings initialization tests only |
+| `test_wallet.py` | 134 | ✅ Good | Tests wallet operations with mocking |
+
+**Unit Test Score: 6/10**
+
+### 1.2 Integration Tests (`tests/integration/`)
+
+| Test File | Lines | Coverage Status | Notes |
+|-----------|-------|----------------|-------|
+| `test_wallet_authentication.py` | 581 | ✅ Excellent | Comprehensive API key generation and validation |
+| `test_wallet_topup.py` | 525 | ✅ Excellent | Thorough topup flow testing |
+| `test_wallet_refund.py` | 579 | ✅ Excellent | Comprehensive refund scenarios |
+| `test_proxy_post_endpoints.py` | 904 | ✅ Good | Tests proxy POST forwarding |
+| `test_proxy_get_endpoints.py` | ? | ⚠️ Minimal | Limited GET endpoint testing |
+| `test_provider_management.py` | 700 | ✅ Good | Tests provider discovery endpoint |
+| `test_background_tasks.py` | ? | ⚠️ Unknown | Not analyzed |
+| `test_database_consistency.py` | ? | ⚠️ Unknown | Not analyzed |
+| `test_error_handling_edge_cases.py` | ? | ⚠️ Unknown | Not analyzed |
+
+**Integration Test Score: 7/10**
+
+---
+
+## 2. Critical Missing Test Coverage
+
+### 2.1 HIGH PRIORITY - Core Routing Logic
+
+**File:** `routstr/proxy.py` (300+ lines)
+
+**Missing Tests:**
+```python
+# CRITICAL: No tests for core proxy routing logic
+- ✗ initialize_upstreams()
+- ✗ reinitialize_upstreams()
+- ✗ refresh_model_maps()
+- ✗ refresh_model_maps_periodically()
+- ✗ get_bearer_token_key()
+- ✗ parse_request_body_json()
+- ✗ proxy() main handler (only indirectly tested via integration)
+```
+
+**Impact:** Core routing decisions, model selection, and request forwarding are not unit tested. Integration tests cover some flows but don't test edge cases or error conditions systematically.
+
+**Recommended Tests:**
+- Test model mapping refresh with various provider configurations
+- Test proxy routing with invalid/missing models
+- Test error handling in upstream initialization
+- Test concurrent request handling
+- Test model alias resolution
+
+### 2.2 HIGH PRIORITY - Upstream Providers
+
+**Files:** `routstr/upstream/*.py` (13 provider implementations)
+
+**Missing Tests:**
+```python
+# CRITICAL: Zero unit tests for upstream provider implementations
+- ✗ BaseUpstreamProvider class
+- ✗ OpenAI provider (openai.py)
+- ✗ Anthropic provider (anthropic.py)
+- ✗ Azure provider (azure.py)
+- ✗ Groq provider (groq.py)
+- ✗ Perplexity provider (perplexity.py)
+- ✗ XAI provider (xai.py)
+- ✗ Fireworks provider (fireworks.py)
+- ✗ Generic provider (generic.py)
+- ✗ OpenRouter provider (openrouter.py)
+- ✗ Ollama provider (ollama.py)
+- ✗ Provider helpers (helpers.py)
+```
+
+**Impact:** No coverage of:
+- Provider-specific request/response transformations
+- Header preparation for each provider
+- Streaming response handling
+- Error response mapping
+- Model discovery and caching
+- Provider fee application
+
+**Recommended Tests:**
+- Unit tests for each provider's request transformation
+- Test streaming vs non-streaming response handling
+- Test error mapping for provider-specific errors
+- Test header preparation with various scenarios
+- Test model caching and refresh logic
+
+### 2.3 HIGH PRIORITY - Payment Cost Calculation
+
+**File:** `routstr/payment/cost_caculation.py` (157 lines)
+
+**Missing Tests:**
+```python
+# CRITICAL: No dedicated tests for cost calculation logic
+- ✗ calculate_cost() with various response types
+- ✗ CostData vs MaxCostData scenarios
+- ✗ Error handling for invalid pricing data
+- ✗ Fixed vs dynamic pricing modes
+- ✗ Token-based cost calculation edge cases
+```
+
+**Impact:** Core billing logic not directly tested. Only indirectly covered through integration tests which don't explore edge cases.
+
+**Recommended Tests:**
+- Test cost calculation with missing usage data
+- Test fixed pricing vs dynamic pricing modes
+- Test model-specific pricing vs global pricing
+- Test cost rounding and precision
+- Test invalid pricing data handling
+
+### 2.4 MEDIUM PRIORITY - Authentication & Balance Management
+
+**File:** `routstr/auth.py` (662 lines)
+
+**Missing Tests:**
+```python
+# Some coverage but gaps remain:
+- ⚠️ validate_bearer_key() - only integration tested
+- ✗ pay_for_request() - insufficient edge case coverage
+- ✗ revert_pay_for_request() - not directly tested
+- ✗ adjust_payment_for_tokens() - complex logic not unit tested
+- ✗ Concurrent payment scenarios
+- ✗ Race condition handling in atomic updates
+```
+
+**Impact:** Critical payment flows lack isolated unit tests. Race conditions and edge cases not systematically tested.
+
+**Recommended Tests:**
+- Test concurrent payment attempts with same key
+- Test payment reversal in various failure scenarios
+- Test adjustment logic with different cost differentials
+- Test atomic update failures and rollback
+- Test cashu token validation edge cases
+
+### 2.5 MEDIUM PRIORITY - NIP-91 Nostr Discovery
+
+**File:** `routstr/nip91.py` (575 lines)
+
+**Missing Tests:**
+```python
+# CRITICAL: Zero tests for Nostr/NIP-91 functionality
+- ✗ nsec_to_keypair()
+- ✗ create_nip91_event()
+- ✗ events_semantically_equal()
+- ✗ query_nip91_events()
+- ✗ discover_onion_url_from_tor()
+- ✗ publish_to_relay()
+- ✗ announce_provider()
+```
+
+**Impact:** No coverage of provider announcement system. Critical for network discovery but completely untested.
+
+**Recommended Tests:**
+- Test NIP-91 event creation and validation
+- Test relay querying with various responses
+- Test event semantic comparison
+- Test onion URL discovery
+- Test publish retry logic and backoff
+- Mock websocket connections for testing
+
+### 2.6 MEDIUM PRIORITY - Discovery System
+
+**File:** `routstr/discovery.py` (400 lines)
+
+**Missing Tests:**
+```python
+# Limited coverage via integration tests only:
+- ⚠️ query_nostr_relay_for_providers() - integration only
+- ⚠️ parse_provider_announcement() - integration only
+- ✗ refresh_providers_cache() - not directly tested
+- ✗ providers_cache_refresher() - background task not tested
+- ✗ fetch_provider_health() - not directly tested
+- ✗ Cache lock mechanism not tested
+```
+
+**Impact:** Provider discovery and health checking not systematically tested.
+
+**Recommended Tests:**
+- Test provider announcement parsing edge cases
+- Test cache refresh with various relay responses
+- Test health check with timeout scenarios
+- Test concurrent cache access
+- Test Tor proxy usage for onion URLs
+
+### 2.7 MEDIUM PRIORITY - Middleware
+
+**File:** `routstr/core/middleware.py` (127 lines)
+
+**Missing Tests:**
+```python
+# CRITICAL: Zero tests for middleware
+- ✗ LoggingMiddleware
+- ✗ Request ID generation and propagation
+- ✗ Request/response logging
+- ✗ Error handling in middleware
+- ✗ Context variable management
+```
+
+**Impact:** Request logging, tracking, and error handling not tested.
+
+**Recommended Tests:**
+- Test request ID generation and propagation
+- Test request/response logging
+- Test error logging and exception handling
+- Test context variable lifecycle
+- Test sensitive data redaction in logs
+
+### 2.8 HIGH PRIORITY - Admin Endpoints
+
+**File:** `routstr/core/admin.py`
+
+**Missing Tests:**
+```python
+# CRITICAL: Zero tests for admin functionality
+- ✗ All admin endpoints (/admin/*)
+- ✗ Model management (enable/disable/update)
+- ✗ Provider management (add/update/delete)
+- ✗ Settings management
+- ✗ Authentication for admin endpoints
+```
+
+**Impact:** No coverage of admin functionality which can modify critical system configuration.
+
+**Recommended Tests:**
+- Test admin authentication
+- Test model CRUD operations
+- Test provider CRUD operations
+- Test settings updates
+- Test validation of admin inputs
+- Test authorization checks
+
+### 2.9 LOW PRIORITY - Balance Router
+
+**File:** `routstr/balance.py` (244 lines)
+
+**Missing Tests:**
+```python
+# Some coverage but gaps:
+- ⚠️ Refund cache mechanism
+- ✗ _refund_cache_get() / _refund_cache_set()
+- ✗ donate() endpoint
+- ✗ Cache TTL expiration
+- ✗ Concurrent refund request handling
+```
+
+**Impact:** Refund caching and edge cases not tested.
+
+**Recommended Tests:**
+- Test refund cache hit/miss scenarios
+- Test cache expiration
+- Test concurrent refund attempts with caching
+- Test donate endpoint
+
+### 2.10 LOW PRIORITY - Payment Helpers
+
+**File:** `routstr/payment/helpers.py`
+
+**Missing Tests:**
+```python
+# Very limited coverage:
+- ✗ check_token_balance()
+- ✗ create_error_response()
+- ✗ calculate_discounted_max_cost()
+- ⚠️ Image token estimation (good coverage)
+```
+
+**Impact:** Helper functions not directly tested, only through integration.
+
+---
+
+## 3. Tests That May Not Be Testing Anything Useful
+
+### 3.1 Potentially Redundant Tests
+
+**File:** `tests/integration/test_example.py`
+- **Status:** Likely a placeholder/example file
+- **Recommendation:** Remove if it's just an example template
+
+### 3.2 Over-Mocked Tests
+
+**File:** `tests/unit/test_wallet.py`
+- **Issue:** Heavy mocking may not catch real integration issues
+- **Example:** `test_credit_balance()` mocks everything, doesn't test actual cashu wallet integration
+- **Recommendation:** Keep for unit testing, but ensure integration tests cover real scenarios
+
+### 3.3 Tests with Unclear Value
+
+**File:** `tests/integration/test_database_timestamp_accuracy()`
+- **Location:** `test_wallet_authentication.py:551`
+- **Issue:** Comments indicate the ApiKey model doesn't have a timestamp field, so test doesn't validate timestamps
+- **Recommendation:** Either add timestamp field or remove test
+
+---
+
+## 4. Outdated Tests
+
+### 4.1 Payment Flow Changes
+
+**Issue:** Integration tests may not reflect the current `reserved_balance` payment flow introduced in recent migration `042f6b77d69d_introduce_reserved_balance.py`
+
+**Affected Tests:**
+- Tests that check balance changes may need updates to account for reserved vs available balance
+- Tests should verify both `balance` and `reserved_balance` fields
+
+**Files:**
+- `test_proxy_post_endpoints.py` - May not verify reserved balance correctly
+- `test_wallet_topup.py` - Should test reserved balance handling
+
+**Recommendation:** Audit all tests that check balance to ensure they account for the two-phase payment system (reserve → finalize).
+
+### 4.2 Model Provider Changes
+
+**Issue:** The algorithm now prefers cheaper providers and applies provider penalties. Some tests may assume old behavior.
+
+**Affected Tests:**
+- Tests that assume specific model→provider mappings may fail with algorithm updates
+- Tests should be more flexible about provider selection
+
+**Recommendation:** Review tests that make assumptions about which provider serves which model.
+
+### 4.3 Settings Management
+
+**Issue:** Migration `a1b2c3d4e5f6_add_settings_table.py` changed how settings are managed (DB vs env variables).
+
+**Affected Tests:**
+- `test_settings.py` tests basic functionality but doesn't test DB vs env precedence thoroughly
+- Integration tests may make wrong assumptions about settings source
+
+**Recommendation:** Expand `test_settings.py` to cover all settings sources and precedence rules.
+
+---
+
+## 5. Test Quality Issues
+
+### 5.1 Insufficient Error Case Coverage
+
+**Issue:** Most tests focus on happy path scenarios.
+
+**Examples:**
+- Limited testing of malformed requests
+- Limited testing of upstream failures
+- Limited testing of concurrent access edge cases
+- Limited testing of database transaction failures
+
+**Recommendation:** Add negative test cases for each major flow.
+
+### 5.2 Mock Overuse in Integration Tests
+
+**Issue:** Integration tests use heavy mocking which defeats their purpose.
+
+**Example:** `test_proxy_post_endpoints.py` mocks `httpx.AsyncClient.send` which means actual HTTP forwarding is never tested.
+
+**Recommendation:** 
+- Keep unit tests heavily mocked
+- Make integration tests use real (local) services where possible
+- Add E2E test category for testing with real external providers (manual/CI only)
+
+### 5.3 Lack of Property-Based Testing
+
+**Issue:** No property-based tests for complex logic.
+
+**Candidates:**
+- Cost calculation (various token combinations should never overflow)
+- Model alias resolution (should always resolve to valid model)
+- Payment adjustments (total cost should always match usage)
+
+**Recommendation:** Add hypothesis tests for critical algorithms.
+
+### 5.4 Missing Concurrency Tests
+
+**Issue:** Limited testing of concurrent operations.
+
+**Gaps:**
+- Concurrent topups to same key
+- Concurrent payments from same key
+- Concurrent model map refreshes
+- Concurrent provider cache updates
+
+**Recommendation:** Add dedicated concurrency test suite using asyncio.gather().
+
+### 5.5 Test Fixtures Need Improvement
+
+**Issue:** `conftest.py` has a large TestmintWallet class (340 lines) that's quite complex.
+
+**Problems:**
+- Hard to understand what's being mocked
+- Fallback token creation doesn't match real cashu tokens
+- Mix of mock and real behavior is confusing
+
+**Recommendation:** 
+- Split into smaller, focused fixtures
+- Document what's mocked vs real
+- Consider using real testmint for integration tests
+
+---
+
+## 6. Missing Test Infrastructure
+
+### 6.1 Performance/Load Tests
+
+**Status:** `test_performance_load.py` exists but coverage unknown
+
+**Gaps:**
+- No systematic load testing
+- No latency benchmarking
+- No throughput testing
+- No resource usage monitoring
+
+**Recommendation:** 
+- Add benchmark tests using pytest-benchmark
+- Add load tests for critical paths
+- Monitor test execution time trends
+
+### 6.2 Contract Tests
+
+**Status:** No contract tests
+
+**Need:**
+- Tests for upstream provider API contracts
+- Tests for expected response formats
+- Tests for error response formats
+
+**Recommendation:** Add contract tests using pact or similar.
+
+### 6.3 Mutation Testing
+
+**Status:** Not used
+
+**Recommendation:** Run mutation testing (mutmut) to find untested code paths.
+
+---
+
+## 7. Test Organization Issues
+
+### 7.1 Test File Naming
+
+**Issue:** Some test files don't follow consistent naming:
+- `test_wallet_authentication.py` ✓ Good
+- `test_example.py` ✗ Unclear purpose
+- `test_fee_consistency.py` ⚠️ Tests model payload conversion, not fees
+
+**Recommendation:** Rename files to match what they actually test.
+
+### 7.2 Test Markers
+
+**Current Usage:**
+- `@pytest.mark.integration`
+- `@pytest.mark.asyncio`
+- `@pytest.mark.slow`
+- `@pytest.mark.skip`
+
+**Missing Markers:**
+- `@pytest.mark.unit`
+- `@pytest.mark.e2e`
+- `@pytest.mark.concurrency`
+- `@pytest.mark.security`
+- Component markers (`@pytest.mark.wallet`, `@pytest.mark.proxy`, etc.)
+
+**Recommendation:** Add comprehensive marker system for better test organization.
+
+### 7.3 Missing Test Documentation
+
+**Issue:** Many test files lack module docstrings explaining what they test and why.
+
+**Recommendation:** Add comprehensive docstrings to all test modules.
+
+---
+
+## 8. Recommended Test Additions by Priority
+
+### 8.1 CRITICAL (Do Immediately)
+
+1. **Upstream Provider Tests** (200+ tests needed)
+   - Unit tests for each provider class
+   - Test request/response transformation
+   - Test error handling per provider
+
+2. **Admin Endpoint Tests** (50+ tests needed)
+   - Full coverage of admin API
+   - Authorization tests
+   - CRUD operation tests
+
+3. **Cost Calculation Tests** (30+ tests needed)
+   - Comprehensive cost calculation coverage
+   - Edge cases and rounding
+   - Fixed vs dynamic pricing
+
+4. **Proxy Core Tests** (40+ tests needed)
+   - Model mapping tests
+   - Routing decision tests
+   - Error handling tests
+
+### 8.2 HIGH (Do This Sprint)
+
+5. **Authentication Tests** (30+ tests needed)
+   - Concurrent payment tests
+   - Race condition tests
+   - Payment reversal tests
+
+6. **Middleware Tests** (20+ tests needed)
+   - Request logging tests
+   - Error handling tests
+   - Context management tests
+
+7. **NIP-91 Tests** (40+ tests needed)
+   - Event creation/parsing tests
+   - Relay interaction tests (mocked)
+   - Onion discovery tests
+
+### 8.3 MEDIUM (Next Sprint)
+
+8. **Discovery Tests** (30+ tests needed)
+   - Provider parsing edge cases
+   - Cache management tests
+   - Health check tests
+
+9. **Payment Helper Tests** (20+ tests needed)
+   - Helper function coverage
+   - Error response creation
+   - Balance checking
+
+10. **Concurrency Tests** (30+ tests needed)
+    - Add concurrency test suite
+    - Test all critical paths under load
+
+### 8.4 LOW (Backlog)
+
+11. **Property-Based Tests** (10+ tests)
+    - Add hypothesis tests
+    - Focus on algorithm logic
+
+12. **Contract Tests** (20+ tests)
+    - Provider API contracts
+    - Response format validation
+
+13. **Performance Tests** (10+ tests)
+    - Benchmark critical paths
+    - Load testing
+
+---
+
+## 9. Test Coverage Metrics
+
+### Current Estimated Coverage
+
+| Component | Unit Tests | Integration Tests | Overall |
+|-----------|------------|-------------------|---------|
+| Wallet | 70% | 90% | 85% |
+| Authentication | 40% | 80% | 65% |
+| Proxy/Routing | 10% | 50% | 35% |
+| Upstream Providers | 0% | 30% | 20% |
+| Payment/Cost | 30% | 60% | 50% |
+| Discovery | 5% | 40% | 30% |
+| Admin | 0% | 0% | 0% |
+| NIP-91 | 0% | 0% | 0% |
+| Middleware | 0% | 0% | 0% |
+| **Overall** | **25%** | **50%** | **40%** |
+
+### Target Coverage
+
+| Component | Target Unit | Target Integration | Target Overall |
+|-----------|-------------|-------------------|----------------|
+| Wallet | 85% | 95% | 90% |
+| Authentication | 80% | 90% | 85% |
+| Proxy/Routing | 75% | 80% | 80% |
+| Upstream Providers | 70% | 60% | 65% |
+| Payment/Cost | 80% | 70% | 75% |
+| Discovery | 65% | 70% | 70% |
+| Admin | 80% | 80% | 80% |
+| NIP-91 | 70% | 50% | 60% |
+| Middleware | 80% | 70% | 75% |
+| **Overall** | **75%** | **75%** | **75%** |
+
+### Effort Required
+
+- **Estimated New Tests Needed:** ~450 tests
+- **Estimated Lines of Test Code:** ~15,000 lines
+- **Estimated Engineering Time:** 4-6 weeks (1-2 engineers)
+
+---
+
+## 10. Actionable Recommendations
+
+### 10.1 Immediate Actions (This Week)
+
+1. **Create upstream provider test suite foundation**
+   - Create `tests/unit/upstream/` directory
+   - Add base test class for provider testing
+   - Write tests for 2-3 providers as examples
+
+2. **Add admin endpoint tests**
+   - Create `tests/integration/test_admin_endpoints.py`
+   - Cover all admin CRUD operations
+   - Add authorization tests
+
+3. **Add cost calculation unit tests**
+   - Create comprehensive `test_cost_calculation.py`
+   - Cover all edge cases and modes
+
+4. **Audit existing tests for reserved_balance**
+   - Update tests to verify two-phase payment
+   - Ensure balance checking is correct
+
+### 10.2 Short-term Actions (This Sprint)
+
+5. **Add proxy core unit tests**
+   - Test model mapping logic
+   - Test routing decisions
+   - Test error handling
+
+6. **Add middleware tests**
+   - Test request logging
+   - Test error handling
+   - Test context management
+
+7. **Improve test fixtures**
+   - Simplify TestmintWallet
+   - Add reusable provider fixtures
+   - Document fixture behavior
+
+8. **Add concurrency test suite**
+   - Test concurrent payments
+   - Test concurrent topups
+   - Test cache access patterns
+
+### 10.3 Medium-term Actions (Next Sprint)
+
+9. **Add NIP-91 and discovery tests**
+   - Mock websocket connections
+   - Test event parsing
+   - Test provider health checks
+
+10. **Add property-based tests**
+    - Use hypothesis for algorithm tests
+    - Test cost calculation properties
+    - Test payment invariants
+
+11. **Improve test organization**
+    - Add comprehensive markers
+    - Document test suite structure
+    - Create test categories
+
+12. **Add contract tests**
+    - Test upstream API contracts
+    - Test response format validation
+
+### 10.4 Long-term Actions (Backlog)
+
+13. **Add E2E test suite**
+    - Test with real external providers
+    - Use testmint for real cashu tests
+    - Run in dedicated CI environment
+
+14. **Add performance test suite**
+    - Benchmark critical paths
+    - Monitor performance trends
+    - Add load testing
+
+15. **Set up mutation testing**
+    - Run mutmut on codebase
+    - Fix identified gaps
+    - Add to CI pipeline
+
+16. **Improve test infrastructure**
+    - Add test database seeding
+    - Add test data factories
+    - Improve test isolation
+
+---
+
+## 11. Risk Assessment
+
+### High Risk Areas (Untested)
+
+1. **Upstream Provider Logic** (0% unit coverage)
+   - **Risk:** Provider-specific bugs not caught before production
+   - **Impact:** Failed requests, incorrect billing, data corruption
+   - **Mitigation:** CRITICAL - Add provider tests immediately
+
+2. **Admin Functionality** (0% coverage)
+   - **Risk:** Configuration changes can break system
+   - **Impact:** Service outage, data corruption, security issues
+   - **Mitigation:** HIGH - Add admin tests this week
+
+3. **Payment Atomicity** (Limited testing)
+   - **Risk:** Race conditions cause incorrect billing
+   - **Impact:** Revenue loss, customer complaints
+   - **Mitigation:** HIGH - Add concurrency tests
+
+4. **NIP-91 Discovery** (0% coverage)
+   - **Risk:** Provider announcement failures go unnoticed
+   - **Impact:** Reduced discoverability, network issues
+   - **Mitigation:** MEDIUM - Add discovery tests next sprint
+
+### Medium Risk Areas (Partial Coverage)
+
+5. **Cost Calculation** (50% coverage)
+   - **Risk:** Incorrect pricing calculations
+   - **Impact:** Revenue loss or overcharging
+   - **Mitigation:** HIGH - Add comprehensive cost tests
+
+6. **Authentication** (65% coverage)
+   - **Risk:** Auth bypass or token issues
+   - **Impact:** Security vulnerability, service abuse
+   - **Mitigation:** MEDIUM - Add auth edge case tests
+
+7. **Middleware** (0% coverage)
+   - **Risk:** Logging or tracking issues
+   - **Impact:** Debugging difficulties, audit trail gaps
+   - **Mitigation:** MEDIUM - Add middleware tests
+
+---
+
+## 12. Testing Best Practices Not Followed
+
+### 12.1 Test Independence
+- Some integration tests may have ordering dependencies
+- Tests should be runnable in any order
+- **Action:** Audit for test interdependencies
+
+### 12.2 Test Data Management
+- Hard-coded test data in tests
+- No test data factories
+- **Action:** Add factory_boy or similar
+
+### 12.3 Test Clarity
+- Some tests have unclear assertions
+- Missing descriptive docstrings
+- **Action:** Improve test documentation
+
+### 12.4 Test Maintenance
+- No automated test quality checks
+- No coverage tracking in CI
+- **Action:** Add coverage reports to CI
+
+---
+
+## 13. Conclusion
+
+The Routstr test suite provides good coverage for wallet and authentication flows but has significant gaps in core routing logic, upstream providers, admin functionality, and discovery features. The following priorities should guide test suite improvements:
+
+### Top 3 Priorities:
+
+1. **Add Upstream Provider Tests** (0% → 70% coverage)
+   - Critical for ensuring correct request handling
+   - Prevents provider-specific bugs in production
+   - Estimated effort: 2 weeks
+
+2. **Add Admin Endpoint Tests** (0% → 80% coverage)
+   - Critical for system stability
+   - Prevents configuration errors
+   - Estimated effort: 1 week
+
+3. **Add Cost Calculation Tests** (50% → 80% coverage)
+   - Critical for revenue integrity
+   - Prevents billing errors
+   - Estimated effort: 1 week
+
+### Success Metrics:
+
+- Overall test coverage: 40% → 75%
+- Unit test coverage: 25% → 75%
+- Integration test coverage: 50% → 75%
+- Tests passing: 100% (maintain)
+- Test execution time: <5 minutes (target)
+
+### Timeline:
+
+- **Week 1-2:** Upstream provider tests + Admin tests
+- **Week 3-4:** Cost calculation + Proxy core tests
+- **Week 5-6:** Middleware + NIP-91 + Discovery tests
+
+With systematic implementation of these recommendations, the test suite will provide robust protection against regressions and enable confident refactoring and feature development.
+
+---
+
+**Report End**
+
+*For questions or clarifications, please contact the analysis team.*