shopify-ai-backup/TOKEN_TRACKING_FIXES.md

Token Tracking Fixes - Summary

Issues Fixed

1. Plan Messages Now Use 1x Multiplier

Location: chat/server.js:4013

• Problem: Plan messages were using getModelTier(model || providerName) which could return 'plus' or 'pro' tier, causing plan tokens to be counted at 2x or 3x multiplier
• Fix: Changed to getModelTier(model || providerName, 'free') to force plan messages to use 'free' tier (1x multiplier)
• Impact: Plan messages (OpenRouter, Mistral, Google, Groq, NVIDIA) now always counted at 1x multiplier regardless of the model used

2. Missing Tier Information Defaults to 1x Usage

Location: chat/server.js:2508-2523

• Problem: Models without configured tiers could have inconsistent tier detection
• Fix: Enhanced getModelTier() function with:
  • Optional forceTier parameter to override tier when needed
  • Better validation to check if tier is in VALID_TIERS before using it
  • Default to 'free' (1x) if tier is not configured
• Impact: All messages with missing tier info now consistently use 1x multiplier

3. Both Plan and OpenCode Tokens Counted in Overall Usage

Status: ✅ Already Working

• Location: chat/public/builder.js:356-390
• Explanation: Builder's usage meter uses 'aggregate' tier which sums up usage across all tiers (free, plus, pro)
• Result: Plan tokens (from OpenRouter etc.) and OpenCode tokens are both included in the total usage display

4. Enhanced Debugging

Location: chat/server.js:2640

• Enhancement: Added source tracking to recordUserTokens() logging
• New Log Format: [USAGE] recording tokens for user=${userId} tier=${safeTier} raw=${tokens} rounded=${roundedTokens} effective=${effectiveTokens} source=${caller_file}
• Benefit: Helps identify where token recording is coming from (plan vs opencode)

Technical Details

Tier Multipliers

TIER_USAGE_MULTIPLIER = {
  free: 1,   // 1x multiplier
  plus: 2,   // 2x multiplier
  pro: 3      // 3x multiplier
}

Token Recording Flow

Plan Messages (1x multiplier)

1. User sends message to /api/plan
2. handlePlanMessage() processes request
3. Provider (OpenRouter, Mistral, etc.) generates response
4. extractTokenUsageFromResult() extracts actual token count
5. recordUserTokens(userId, 'free', tokensUsed) records at 1x multiplier
6. Tokens go to bucket.usage.free

OpenCode Messages (model tier multiplier)

1. User sends message to /api/sessions/{id}/messages
2. processMessage() handles OpenCode execution
3. getModelTier(message.model) determines tier from model configuration
4. extractTokenUsageFromResult() or estimateTokensFromMessages() gets token count
5. recordUserTokens(userId, tier, tokensUsed) records at appropriate multiplier
6. Tokens go to bucket.usage[tier] (free, plus, or pro)

Usage Summary Calculation

// For each tier (free, plus, pro):
const used = bucket.usage[tier];
const limit = getPlanTokenLimits(plan, userId)[tier];
const remaining = limit - used;
const percent = (used / limit) * 100;

Testing Checklist

• Create a plan message and verify it's counted at 1x multiplier
• Create an OpenCode message and verify it's counted at model's configured tier
• Check that both plan and OpenCode tokens appear in the aggregate usage meter
• Verify logs show proper tier assignment for each message type
• Test with a model that has no tier configured (should default to 1x)
• Test with a model that has 'plus' or 'pro' tier configured

How to Verify Token Tracking

Check Server Logs

Look for these log patterns:

[PLAN] Recording tokens for successful plan: user=xxx tokens=1000 provider=openrouter model=gpt-4o
[USAGE] recording tokens for user=xxx tier=free raw=1000 rounded=1000 effective=1000 source=handlePlanMessage

[USAGE] processMessage: recording tokens user=xxx tier=plus raw=500 rounded=500 effective=1000 source=processMessage
[USAGE] Persisted token usage. New total for xxx/free: 15000
[USAGE] Persisted token usage. New total for xxx/plus: 2000

Check Client Usage Meter

Open browser DevTools Console and look for:

[USAGE] Usage summary loaded:
{
  month: "2025-01",
  plan: "hobby",
  tiers: {
    free: { used: 15000, limit: 50000, remaining: 35000, percent: 30, ... },
    plus: { used: 2000, limit: 2500000, remaining: 2498000, percent: 0, ... },
    pro: { used: 0, limit: 5000000, remaining: 5000000, percent: 0, ... }
  }
}

Files Modified

• chat/server.js - Enhanced getModelTier() function, fixed plan message recording, enhanced logging
• chat/public/builder.js - Already had aggregate tier support (no changes needed)