Restore to commit 74e578279624c6045ca440a3459ebfa1f8d54191
This commit is contained in:
southseact-3d
233
TOKEN_USAGE_IMPLEMENTATION.md
Normal file
233
TOKEN_USAGE_IMPLEMENTATION.md
Normal file
@@ -0,0 +1,233 @@
|
||||
# Token Usage Tracking - Implementation Guide
|
||||
|
||||
## Overview
|
||||
This document explains how token usage tracking works in the application and how to test it.
|
||||
|
||||
## Architecture
|
||||
|
||||
### Token Recording Flow
|
||||
1. **AI Request Made** → User sends a message (plan or build)
|
||||
2. **Token Consumption** → AI provider processes request and reports usage
|
||||
3. **Recording** → Server records tokens via `recordUserTokens(userId, tokens)`
|
||||
4. **Persistence** → Tokens stored in `tokenUsage` object and persisted to disk
|
||||
5. **UI Update** → Frontend fetches updated usage via `/api/account/usage`
|
||||
6. **Display** → Progress bar updates to show new percentage
|
||||
|
||||
### Key Components
|
||||
|
||||
#### Server Side (`chat/server.js`)
|
||||
|
||||
**Token Storage Structure:**
|
||||
```javascript
|
||||
tokenUsage[userId] = {
|
||||
month: '2026-01', // Current month key
|
||||
usage: 15000, // Tokens used this month
|
||||
addOns: 50000 // Bonus tokens purchased
|
||||
}
|
||||
```
|
||||
|
||||
**Core Functions:**
|
||||
- `recordUserTokens(userId, tokens)` - Records token usage
|
||||
- `getTokenUsageSummary(userId, plan)` - Returns usage summary
|
||||
- `ensureTokenUsageBucket(userId)` - Ensures bucket exists for user
|
||||
- `persistTokenUsage()` - Saves to disk
|
||||
|
||||
**API Endpoints:**
|
||||
- `GET /api/account/usage` - Returns current usage summary
|
||||
- `POST /api/test/simulate-tokens` - Test endpoint for simulating usage
|
||||
- Body: `{ "tokens": 1000 }`
|
||||
- Returns: Updated usage summary
|
||||
|
||||
#### Client Side (`chat/public/builder.js`)
|
||||
|
||||
**Core Functions:**
|
||||
- `loadUsageSummary()` - Fetches usage from `/api/account/usage`
|
||||
- `updateUsageProgressBar(summary)` - Updates UI with new usage data
|
||||
|
||||
**Update Timing:**
|
||||
- After message completion (2 second delay for server persistence)
|
||||
- After plan message (immediate)
|
||||
- On page load
|
||||
- On manual refresh
|
||||
- While OpenCode is running: poll every 60 seconds
|
||||
|
||||
**HTML Elements:**
|
||||
- `#usage-meter-title` - "Usage" label
|
||||
- `#usage-meter-percent` - "X% used" text
|
||||
- `#usage-meter-fill` - Progress bar fill element
|
||||
- `#usage-meter-track` - Progress bar container
|
||||
|
||||
## Testing
|
||||
|
||||
### Method 1: Test Endpoint
|
||||
Use the test endpoint to simulate token consumption without running actual AI models.
|
||||
|
||||
**Using curl:**
|
||||
```bash
|
||||
# Simulate 1000 tokens
|
||||
curl -X POST http://localhost:4000/api/test/simulate-tokens \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"tokens": 1000}'
|
||||
|
||||
# Simulate 5000 tokens
|
||||
curl -X POST http://localhost:4000/api/test/simulate-tokens \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"tokens": 5000}'
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```json
|
||||
{
|
||||
"ok": true,
|
||||
"message": "Simulated 1000 tokens",
|
||||
"tokensAdded": 1000,
|
||||
"summary": {
|
||||
"month": "2026-01",
|
||||
"used": 15000,
|
||||
"limit": 50000,
|
||||
"remaining": 35000,
|
||||
"percent": 30,
|
||||
"addOn": 0,
|
||||
"plan": "hobby"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Method 2: Test HTML Page
|
||||
Open `http://localhost:4000/test_token_usage.html` in your browser.
|
||||
|
||||
**Features:**
|
||||
- Visual progress bar matching builder page
|
||||
- Input field to specify token amount
|
||||
- Real-time usage statistics
|
||||
- Debug output showing raw API response
|
||||
- Automatic refresh after simulation
|
||||
|
||||
**Usage:**
|
||||
1. Navigate to the test page
|
||||
2. Enter desired token amount (default: 1000)
|
||||
3. Click "Simulate Token Usage"
|
||||
4. Watch the progress bar update
|
||||
5. Click "Refresh Usage" to manually fetch latest data
|
||||
|
||||
### Method 3: Real AI Usage
|
||||
Test with actual AI requests on the builder page:
|
||||
|
||||
1. Open builder page: `http://localhost:4000/builder?session=<session-id>`
|
||||
2. Send a plan message (uses OpenRouter)
|
||||
3. Wait for response
|
||||
4. Observe usage bar update after ~2 seconds
|
||||
5. Send build message (uses OpenCode)
|
||||
6. Wait for completion
|
||||
7. Observe usage bar update again
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
### Token Recording
|
||||
- [x] `recordUserTokens()` accepts userId and tokens
|
||||
- [x] Tokens are rounded up (ceil)
|
||||
- [x] Tokens are added to bucket.usage
|
||||
- [x] Changes are persisted immediately
|
||||
- [x] Console logs confirm recording
|
||||
|
||||
### Usage Summary
|
||||
- [x] Summary includes: month, used, limit, remaining, percent, addOn, plan
|
||||
- [x] Percent calculation: `(used / limit) * 100`
|
||||
- [x] Remaining calculation: `limit - used`
|
||||
- [x] Handles zero/null limits gracefully
|
||||
|
||||
### API Endpoints
|
||||
- [x] `/api/account/usage` returns summary
|
||||
- [x] `/api/test/simulate-tokens` records and returns summary
|
||||
- [x] Endpoints handle missing userId
|
||||
- [x] Responses include all expected fields
|
||||
|
||||
### UI Updates
|
||||
- [x] Progress bar width updates based on percent
|
||||
- [x] Percent text shows "X% used"
|
||||
- [x] Tooltip shows "used / limit tokens"
|
||||
- [x] Update occurs after AI completion
|
||||
- [x] Update occurs after simulation
|
||||
|
||||
## Token Limits by Plan
|
||||
|
||||
```javascript
|
||||
const USER_PLANS = {
|
||||
hobby: {
|
||||
tokens: 50000, // 50k tokens/month
|
||||
multiplier: 1
|
||||
},
|
||||
starter: {
|
||||
tokens: 2500000, // 2.5M tokens/month
|
||||
multiplier: 1
|
||||
},
|
||||
business: {
|
||||
tokens: 5000000, // 5M tokens/month
|
||||
multiplier: 1
|
||||
},
|
||||
enterprise: {
|
||||
tokens: 10000000, // 10M tokens/month
|
||||
multiplier: 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Server Logs
|
||||
Look for these log patterns:
|
||||
```
|
||||
[USAGE] Recorded 1000 tokens for user-abc123. New total: 15000
|
||||
[USAGE] Usage summary loaded: { month: '2026-01', used: 15000, limit: 50000, ... }
|
||||
```
|
||||
|
||||
### Client Console
|
||||
Look for these log patterns in browser console:
|
||||
```
|
||||
[USAGE] Usage summary loaded: { month: "2026-01", plan: "hobby", ... }
|
||||
[TEST] Simulation response: { ok: true, tokensAdded: 1000, ... }
|
||||
```
|
||||
|
||||
### Common Issues
|
||||
|
||||
**Issue: Usage bar doesn't update**
|
||||
- Check network tab for `/api/account/usage` request
|
||||
- Verify response contains `summary` object
|
||||
- Check console for JavaScript errors
|
||||
- Verify element IDs match: `usage-meter-fill`, `usage-meter-percent`, etc.
|
||||
|
||||
**Issue: Percentage is wrong**
|
||||
- Verify token recording: check server logs
|
||||
- Check calculation: `(used / limit) * 100`
|
||||
- Ensure limit > 0
|
||||
- Check for integer overflow (unlikely)
|
||||
|
||||
**Issue: Test endpoint returns 401**
|
||||
- Ensure user is logged in
|
||||
- Check cookie/session
|
||||
- Verify `X-User-Id` header or `chat_user` cookie exists
|
||||
|
||||
## Security Notes
|
||||
|
||||
The `/api/test/simulate-tokens` endpoint:
|
||||
- Should be disabled in production
|
||||
- Currently accepts any token amount
|
||||
- No authentication beyond user session
|
||||
- Can be used to artificially inflate usage
|
||||
|
||||
**Production Considerations:**
|
||||
- Remove or restrict test endpoint
|
||||
- Add rate limiting
|
||||
- Add admin-only flag
|
||||
- Log all simulations for auditing
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
1. **Real-time Updates** - WebSocket/SSE for live usage updates
|
||||
2. **Granular Tracking** - Track by model/provider/feature
|
||||
3. **Historical Data** - Store and display usage trends
|
||||
4. **Alerts** - Notify when approaching limit
|
||||
5. **Usage Analytics** - Dashboard with charts and insights
|
||||
6. **Token Estimation** - Show estimated cost before sending
|
||||
7. **Batch Operations** - Test multiple scenarios at once
|
||||
8. **Usage Export** - Download usage data as CSV/JSON
|
||||
Reference in New Issue
Block a user