shopify-ai-backup/TOKEN_USAGE_IMPLEMENTATION.md

Token Usage Tracking - Implementation Guide

Overview

This document explains how token usage tracking works in the application and how to test it.

Architecture

Token Recording Flow

1. AI Request Made → User sends a message (plan or build)
2. Token Consumption → AI provider processes request and reports usage
3. Recording → Server records tokens via recordUserTokens(userId, tokens)
4. Persistence → Tokens stored in tokenUsage object and persisted to disk
5. UI Update → Frontend fetches updated usage via /api/account/usage
6. Display → Progress bar updates to show new percentage

Key Components

Server Side (chat/server.js)

Token Storage Structure:

tokenUsage[userId] = {
  month: '2026-01',     // Current month key
  usage: 15000,         // Tokens used this month
  addOns: 50000        // Bonus tokens purchased
}

Core Functions:

• recordUserTokens(userId, tokens) - Records token usage
• getTokenUsageSummary(userId, plan) - Returns usage summary
• ensureTokenUsageBucket(userId) - Ensures bucket exists for user
• persistTokenUsage() - Saves to disk

API Endpoints:

• GET /api/account/usage - Returns current usage summary
• POST /api/test/simulate-tokens - Test endpoint for simulating usage
  • Body: { "tokens": 1000 }
  • Returns: Updated usage summary

Client Side (chat/public/builder.js)

Core Functions:

• loadUsageSummary() - Fetches usage from /api/account/usage
• updateUsageProgressBar(summary) - Updates UI with new usage data

Update Timing:

• After message completion (2 second delay for server persistence)
• After plan message (immediate)
• On page load
• On manual refresh
• While OpenCode is running: poll every 60 seconds

HTML Elements:

• #usage-meter-title - "Usage" label
• #usage-meter-percent - "X% used" text
• #usage-meter-fill - Progress bar fill element
• #usage-meter-track - Progress bar container

Testing

Method 1: Test Endpoint

Use the test endpoint to simulate token consumption without running actual AI models.

Using curl:

# Simulate 1000 tokens
curl -X POST http://localhost:4000/api/test/simulate-tokens \
  -H "Content-Type: application/json" \
  -d '{"tokens": 1000}'

# Simulate 5000 tokens
curl -X POST http://localhost:4000/api/test/simulate-tokens \
  -H "Content-Type: application/json" \
  -d '{"tokens": 5000}'

Response:

{
  "ok": true,
  "message": "Simulated 1000 tokens",
  "tokensAdded": 1000,
  "summary": {
    "month": "2026-01",
    "used": 15000,
    "limit": 50000,
    "remaining": 35000,
    "percent": 30,
    "addOn": 0,
    "plan": "hobby"
  }
}

Method 2: Test HTML Page

Open http://localhost:4000/test_token_usage.html in your browser.

Features:

• Visual progress bar matching builder page
• Input field to specify token amount
• Real-time usage statistics
• Debug output showing raw API response
• Automatic refresh after simulation

Usage:

1. Navigate to the test page
2. Enter desired token amount (default: 1000)
3. Click "Simulate Token Usage"
4. Watch the progress bar update
5. Click "Refresh Usage" to manually fetch latest data

Method 3: Real AI Usage

Test with actual AI requests on the builder page:

1. Open builder page: http://localhost:4000/builder?session=<session-id>
2. Send a plan message (uses OpenRouter)
3. Wait for response
4. Observe usage bar update after ~2 seconds
5. Send build message (uses OpenCode)
6. Wait for completion
7. Observe usage bar update again

Verification Checklist

Token Recording

• recordUserTokens() accepts userId and tokens
• Tokens are rounded up (ceil)
• Tokens are added to bucket.usage
• Changes are persisted immediately
• Console logs confirm recording

Usage Summary

• Summary includes: month, used, limit, remaining, percent, addOn, plan
• Percent calculation: (used / limit) * 100
• Remaining calculation: limit - used
• Handles zero/null limits gracefully

API Endpoints

• /api/account/usage returns summary
• /api/test/simulate-tokens records and returns summary
• Endpoints handle missing userId
• Responses include all expected fields

UI Updates

• Progress bar width updates based on percent
• Percent text shows "X% used"
• Tooltip shows "used / limit tokens"
• Update occurs after AI completion
• Update occurs after simulation

Token Limits by Plan

const USER_PLANS = {
  hobby: {
    tokens: 50000,        // 50k tokens/month
    multiplier: 1
  },
  starter: {
    tokens: 2500000,      // 2.5M tokens/month
    multiplier: 1
  },
  business: {
    tokens: 5000000,      // 5M tokens/month
    multiplier: 1
  },
  enterprise: {
    tokens: 10000000,     // 10M tokens/month
    multiplier: 1
  }
}

Debugging

Server Logs

Look for these log patterns:

[USAGE] Recorded 1000 tokens for user-abc123. New total: 15000
[USAGE] Usage summary loaded: { month: '2026-01', used: 15000, limit: 50000, ... }

Client Console

Look for these log patterns in browser console:

[USAGE] Usage summary loaded: { month: "2026-01", plan: "hobby", ... }
[TEST] Simulation response: { ok: true, tokensAdded: 1000, ... }

Common Issues

Issue: Usage bar doesn't update

• Check network tab for /api/account/usage request
• Verify response contains summary object
• Check console for JavaScript errors
• Verify element IDs match: usage-meter-fill, usage-meter-percent, etc.

Issue: Percentage is wrong

• Verify token recording: check server logs
• Check calculation: (used / limit) * 100
• Ensure limit > 0
• Check for integer overflow (unlikely)

Issue: Test endpoint returns 401

• Ensure user is logged in
• Check cookie/session
• Verify X-User-Id header or chat_user cookie exists

Security Notes

The /api/test/simulate-tokens endpoint:

• Should be disabled in production
• Currently accepts any token amount
• No authentication beyond user session
• Can be used to artificially inflate usage

Production Considerations:

• Remove or restrict test endpoint
• Add rate limiting
• Add admin-only flag
• Log all simulations for auditing

Future Enhancements

1. Real-time Updates - WebSocket/SSE for live usage updates
2. Granular Tracking - Track by model/provider/feature
3. Historical Data - Store and display usage trends
4. Alerts - Notify when approaching limit
5. Usage Analytics - Dashboard with charts and insights
6. Token Estimation - Show estimated cost before sending
7. Batch Operations - Test multiple scenarios at once
8. Usage Export - Download usage data as CSV/JSON