# Token Usage Tracking - Implementation Guide

## Overview
This document explains how token usage tracking works in the application and how to test it.

## Architecture

### Token Recording Flow
1. **AI Request Made** → User sends a message (plan or build)
2. **Token Consumption** → AI provider processes request and reports usage
3. **Recording** → Server records tokens via `recordUserTokens(userId, tokens)`
4. **Persistence** → Tokens stored in `tokenUsage` object and persisted to disk
5. **UI Update** → Frontend fetches updated usage via `/api/account/usage`
6. **Display** → Progress bar updates to show new percentage

### Key Components

#### Server Side (`chat/server.js`)

**Token Storage Structure:**
```javascript
tokenUsage[userId] = {
  month: '2026-01',     // Current month key
  usage: 15000,         // Tokens used this month
  addOns: 50000        // Bonus tokens purchased
}
```

**Core Functions:**
- `recordUserTokens(userId, tokens)` - Records token usage
- `getTokenUsageSummary(userId, plan)` - Returns usage summary
- `ensureTokenUsageBucket(userId)` - Ensures bucket exists for user
- `persistTokenUsage()` - Saves to disk

**API Endpoints:**
- `GET /api/account/usage` - Returns current usage summary
- `POST /api/test/simulate-tokens` - Test endpoint for simulating usage
  - Body: `{ "tokens": 1000 }`
  - Returns: Updated usage summary

#### Client Side (`chat/public/builder.js`)

**Core Functions:**
- `loadUsageSummary()` - Fetches usage from `/api/account/usage`
- `updateUsageProgressBar(summary)` - Updates UI with new usage data

**Update Timing:**
- After message completion (2 second delay for server persistence)
- After plan message (immediate)
- On page load
- On manual refresh
- While OpenCode is running: poll every 60 seconds

**HTML Elements:**
- `#usage-meter-title` - "Usage" label
- `#usage-meter-percent` - "X% used" text
- `#usage-meter-fill` - Progress bar fill element
- `#usage-meter-track` - Progress bar container

## Testing

### Method 1: Test Endpoint
Use the test endpoint to simulate token consumption without running actual AI models.

**Using curl:**
```bash
# Simulate 1000 tokens
curl -X POST http://localhost:4000/api/test/simulate-tokens \
  -H "Content-Type: application/json" \
  -d '{"tokens": 1000}'

# Simulate 5000 tokens
curl -X POST http://localhost:4000/api/test/simulate-tokens \
  -H "Content-Type: application/json" \
  -d '{"tokens": 5000}'
```

**Response:**
```json
{
  "ok": true,
  "message": "Simulated 1000 tokens",
  "tokensAdded": 1000,
  "summary": {
    "month": "2026-01",
    "used": 15000,
    "limit": 50000,
    "remaining": 35000,
    "percent": 30,
    "addOn": 0,
    "plan": "hobby"
  }
}
```

### Method 2: Test HTML Page
Open `http://localhost:4000/test_token_usage.html` in your browser.

**Features:**
- Visual progress bar matching builder page
- Input field to specify token amount
- Real-time usage statistics
- Debug output showing raw API response
- Automatic refresh after simulation

**Usage:**
1. Navigate to the test page
2. Enter desired token amount (default: 1000)
3. Click "Simulate Token Usage"
4. Watch the progress bar update
5. Click "Refresh Usage" to manually fetch latest data

### Method 3: Real AI Usage
Test with actual AI requests on the builder page:

1. Open builder page: `http://localhost:4000/builder?session=<session-id>`
2. Send a plan message (uses OpenRouter)
3. Wait for response
4. Observe usage bar update after ~2 seconds
5. Send build message (uses OpenCode)
6. Wait for completion
7. Observe usage bar update again

## Verification Checklist

### Token Recording
- [x] `recordUserTokens()` accepts userId and tokens
- [x] Tokens are rounded up (ceil) 
- [x] Tokens are added to bucket.usage
- [x] Changes are persisted immediately
- [x] Console logs confirm recording

### Usage Summary
- [x] Summary includes: month, used, limit, remaining, percent, addOn, plan
- [x] Percent calculation: `(used / limit) * 100`
- [x] Remaining calculation: `limit - used`
- [x] Handles zero/null limits gracefully

### API Endpoints
- [x] `/api/account/usage` returns summary
- [x] `/api/test/simulate-tokens` records and returns summary
- [x] Endpoints handle missing userId
- [x] Responses include all expected fields

### UI Updates
- [x] Progress bar width updates based on percent
- [x] Percent text shows "X% used"
- [x] Tooltip shows "used / limit tokens"
- [x] Update occurs after AI completion
- [x] Update occurs after simulation

## Token Limits by Plan

```javascript
const USER_PLANS = {
  hobby: {
    tokens: 50000,        // 50k tokens/month
    multiplier: 1
  },
  starter: {
    tokens: 2500000,      // 2.5M tokens/month
    multiplier: 1
  },
  business: {
    tokens: 5000000,      // 5M tokens/month
    multiplier: 1
  },
  enterprise: {
    tokens: 10000000,     // 10M tokens/month
    multiplier: 1
  }
}
```

## Debugging

### Server Logs
Look for these log patterns:
```
[USAGE] Recorded 1000 tokens for user-abc123. New total: 15000
[USAGE] Usage summary loaded: { month: '2026-01', used: 15000, limit: 50000, ... }
```

### Client Console
Look for these log patterns in browser console:
```
[USAGE] Usage summary loaded: { month: "2026-01", plan: "hobby", ... }
[TEST] Simulation response: { ok: true, tokensAdded: 1000, ... }
```

### Common Issues

**Issue: Usage bar doesn't update**
- Check network tab for `/api/account/usage` request
- Verify response contains `summary` object
- Check console for JavaScript errors
- Verify element IDs match: `usage-meter-fill`, `usage-meter-percent`, etc.

**Issue: Percentage is wrong**
- Verify token recording: check server logs
- Check calculation: `(used / limit) * 100`
- Ensure limit > 0
- Check for integer overflow (unlikely)

**Issue: Test endpoint returns 401**
- Ensure user is logged in
- Check cookie/session
- Verify `X-User-Id` header or `chat_user` cookie exists

## Security Notes

The `/api/test/simulate-tokens` endpoint:
- Should be disabled in production
- Currently accepts any token amount
- No authentication beyond user session
- Can be used to artificially inflate usage

**Production Considerations:**
- Remove or restrict test endpoint
- Add rate limiting
- Add admin-only flag
- Log all simulations for auditing

## Future Enhancements

1. **Real-time Updates** - WebSocket/SSE for live usage updates
2. **Granular Tracking** - Track by model/provider/feature
3. **Historical Data** - Store and display usage trends
4. **Alerts** - Notify when approaching limit
5. **Usage Analytics** - Dashboard with charts and insights
6. **Token Estimation** - Show estimated cost before sending
7. **Batch Operations** - Test multiple scenarios at once
8. **Usage Export** - Download usage data as CSV/JSON