shopify-ai-backup/PLAN_PROVIDER_PREFERENCE.md

Provider Preference per Model - Implementation Plan

Overview

This document outlines the implementation plan for allowing users to specify preferred providers for specific models when using OpenRouter via the OpenCode CLI. For example, ensuring Claude models only route through Anthropic's direct API rather than third-party providers.

---

Part 1: OpenRouter API Research

How Provider Routing Works

OpenRouter supports a provider object in the request body for Chat Completions that controls routing behavior:

{
  "model": "anthropic/claude-sonnet-4",
  "messages": [...],
  "provider": {
    "order": ["anthropic"],
    "allow_fallbacks": false
  }
}

Key Provider Routing Parameters

Parameter	Type	Description
order	string[]	List of provider slugs to try in order (e.g., ["anthropic", "openai"])
allow_fallbacks	boolean	Whether to allow backup providers when primary is unavailable (default: true)
only	string[]	List of provider slugs to allow for this request (whitelist)
ignore	string[]	List of provider slugs to skip for this request (blacklist)
sort	string | object	Sort providers by "price", "throughput", or "latency"
require_parameters	boolean	Only use providers that support all request parameters
data_collection	"allow" | "deny"	Control whether to use providers that may store data
zdr	boolean	Restrict to Zero Data Retention endpoints

Common Use Cases

Force specific provider only:

{
  "provider": {
    "order": ["anthropic"],
    "allow_fallbacks": false
  }
}

Prefer provider but allow fallbacks:

{
  "provider": {
    "order": ["anthropic"]
  }
}

Whitelist multiple providers:

{
  "provider": {
    "only": ["anthropic", "openai", "google"]
  }
}

Blacklist specific providers:

{
  "provider": {
    "ignore": ["some-provider", "another-provider"]
  }
}

Available Provider Slugs (Examples)

• anthropic - Anthropic direct
• openai - OpenAI direct
• google - Google AI Studio
• azure - Azure OpenAI
• together - Together AI
• fireworks - Fireworks AI
• groq - Groq
• deepinfra - DeepInfra
• bedrock - AWS Bedrock
• vertex-ai - Google Vertex AI

---

Part 2: Current Architecture Analysis

Relevant Files

File	Purpose
packages/console/core/src/model.ts	ZenData model configuration with providers array
packages/console/app/src/routes/zen/util/handler.ts	Request handler with provider selection logic
packages/console/app/src/routes/workspace/[id]/model-section.tsx	Admin UI for model settings
packages/app/src/components/settings-models.tsx	Desktop app model visibility settings
packages/opencode/src/provider/models.ts	CLI model definitions from models.dev

Current Provider Selection Logic

The selectProvider function in handler.ts (lines 340-398) currently selects providers based on:

1. BYOK (Bring Your Own Key) - Uses provider credentials if configured
2. Trial mode - Routes to trial provider if applicable
3. Sticky provider - Maintains same provider across session
4. Weighted random selection - Distributes across providers based on weight

Model Configuration Schema

From packages/console/core/src/model.ts:

const ModelSchema = z.object({
  name: z.string(),
  cost: ModelCostSchema,
  providers: z.array(z.object({
    id: z.string(),          // Provider slug
    model: z.string(),       // Model name for provider
    weight: z.number().optional(),
    disabled: z.boolean().optional(),
  })),
  stickyProvider: z.enum(["strict", "prefer"]).optional(),
  fallbackProvider: z.string().optional(),
  byokProvider: z.enum(["openai", "anthropic", "google"]).optional(),
  // ...
})

---

Part 3: Implementation Options

Option A: Model-Level Provider Preference (Server-Side)

Add provider preference to model configuration in ZenData, allowing workspace admins to set preferred providers.

Pros:

• Centralized control
• Applied automatically to all requests
• No client-side changes required

Cons:

• Requires server deployment for changes
• Not immediately visible to end users

Option B: Request-Level Provider Preference (Client-Side)

Allow clients to specify provider preferences per request via headers or request body.

Pros:

• Flexible for different use cases
• No server deployment needed for preference changes

Cons:

• Requires client library updates
• More complex API surface

Option C: Hybrid Approach (Recommended)

Combine both: server-level defaults with client-side overrides.

---

Part 4: Implementation Details

4.1 Database Schema Changes

Add new table for model provider preferences:

-- packages/console/core/src/schema/model-provider.sql.ts

import { text, integer, mysqlTable } from "drizzle-orm/mysql-core"

export const ModelProviderTable = mysqlTable("model_provider", {
  id: text("id").primaryKey(),
  workspace_id: text("workspace_id").notNull(),
  model: text("model").notNull(),
  provider_preference: text("provider_preference").notNull(), // JSON: { order: [], allow_fallbacks: bool, only: [], ignore: [] }
  time_created: integer("time_created").notNull(),
  time_updated: integer("time_updated"),
})

4.2 API Changes

Add new functions to model.ts:

export namespace Model {
  // Existing functions...
  
  export const setProviderPreference = fn(
    z.object({
      model: z.string(),
      preference: z.object({
        order: z.array(z.string()).optional(),
        only: z.array(z.string()).optional(),
        ignore: z.array(z.string()).optional(),
        allow_fallbacks: z.boolean().optional(),
      }),
    }),
    ({ model, preference }) => {
      Actor.assertAdmin()
      // Store preference in database
    }
  )

  export const getProviderPreference = fn(
    z.object({ model: z.string() }),
    ({ model }) => {
      // Retrieve preference from database
    }
  )
}

4.3 Handler Changes

Modify handler.ts to apply provider preferences:

function selectProvider(
  reqModel: string,
  zenData: ZenData,
  authInfo: AuthInfo,
  modelInfo: ModelInfo,
  sessionId: string,
  isTrial: boolean,
  retry: RetryOptions,
  stickyProvider: string | undefined,
  userPreference: ProviderPreference | undefined, // NEW
) {
  // Apply user preference first
  if (userPreference?.order && userPreference.order.length > 0) {
    const preferredProvider = userPreference.order
      .map(slug => modelInfo.providers.find(p => p.id === slug))
      .find(p => p && !p.disabled)
    
    if (preferredProvider) {
      if (!userPreference.allow_fallbacks) {
        return preferredProvider
      }
      // Continue with fallback logic...
    }
  }
  
  // Existing logic...
}

4.4 Request Body Forwarding

For OpenRouter integration, pass provider preferences in the request body:

const reqBody = JSON.stringify({
  ...createBodyConverter(opts.format, providerInfo.format)(body),
  model: providerInfo.model,
  // Add provider preferences if using OpenRouter
  ...(providerInfo.id === 'openrouter' && userPreference && {
    provider: {
      order: userPreference.order,
      allow_fallbacks: userPreference.allow_fallbacks ?? true,
      only: userPreference.only,
      ignore: userPreference.ignore,
    }
  }),
})

---

Part 5: Admin UI Options

Option 1: Inline Provider Dropdown (Recommended)

Add a dropdown next to each model in the existing table:

<tr data-slot="model-row">
  <td data-slot="model-name">
    <div>{/* Model icon and name */}</div>
  </td>
  <td data-slot="model-provider">
    <select 
      onChange={(e) => updateProviderPreference(model.id, e.target.value)}
      disabled={!userInfo()?.isAdmin}
    >
      <option value="">Auto (default)</option>
      <option value="anthropic">Anthropic only</option>
      <option value="anthropic-fallback">Anthropic (allow fallbacks)</option>
      <option value="openai">OpenAI only</option>
      {/* ... other providers */}
    </select>
  </td>
  <td data-slot="model-toggle">
    {/* Enable/disable toggle */}
  </td>
</tr>

Option 2: Expandable Model Details

Add expandable row with detailed provider settings:

<tr data-slot="model-row">
  {/* Basic info */}
</tr>
<tr data-slot="model-details" class="hidden">
  <td colspan="3">
    <div class="provider-settings">
      <h4>Provider Preferences</h4>
      <div class="preference-option">
        <label>Routing Strategy</label>
        <select>
          <option>Auto (load balanced)</option>
          <option>Prefer specific provider</option>
          <option>Require specific provider</option>
        </select>
      </div>
      <div class="preference-option">
        <label>Primary Provider</label>
        <select>
          <option value="anthropic">Anthropic</option>
          <option value="openai">OpenAI</option>
          {/* ... */}
        </select>
      </div>
      <div class="preference-option">
        <label>Allow Fallbacks</label>
        <Switch checked={true} />
      </div>
    </div>
  </td>
</tr>

Option 3: Dedicated Provider Settings Page

Create a new tab/modal for advanced provider configuration:

// New route: /workspace/[id]/models/[modelId]/providers

export function ModelProviderSettings() {
  const params = useParams()
  const modelInfo = createAsync(() => getModelInfo(params.id, params.modelId))
  
  return (
    <div class="provider-settings-page">
      <header>
        <h2>{modelInfo()?.name} - Provider Settings</h2>
      </header>
      
      <section class="routing-strategy">
        <h3>Routing Strategy</h3>
        <RadioGroup value={strategy} onChange={setStrategy}>
          <Radio value="auto">Automatic (OpenRouter optimized)</Radio>
          <Radio value="preferred">Prefer provider</Radio>
          <Radio value="required">Require provider</Radio>
        </RadioGroup>
      </section>
      
      <section class="provider-order">
        <h3>Provider Priority</h3>
        <DragDropList items={providers} onReorder={setProviderOrder}>
          {(provider) => (
            <div class="provider-item">
              <ProviderIcon id={provider.id} />
              <span>{provider.name}</span>
              <DragHandle />
            </div>
          )}
        </DragDropList>
      </section>
      
      <section class="fallback-settings">
        <h3>Fallback Behavior</h3>
        <Switch 
          checked={allowFallbacks} 
          onChange={setAllowFallbacks}
        >
          Allow fallback to other providers
        </Switch>
      </section>
      
      <section class="provider-blacklist">
        <h3>Excluded Providers</h3>
        <TagInput 
          value={ignoredProviders} 
          onChange={setIgnoredProviders}
          suggestions={availableProviders}
        />
      </section>
    </div>
  )
}

Option 4: Compact Status Indicators

Add visual indicators for provider preference status:

<td data-slot="model-provider-status">
  <div class="provider-indicators">
    <Show when={providerPreference()}>
      <span class="badge" title={`Using ${providerPreference().order[0]}`}>
        <ProviderIcon id={providerPreference().order[0]} size="sm" />
        {providerPreference().allow_fallbacks ? '' : ' (strict)'}
      </span>
    </Show>
    <Show when={!providerPreference()}>
      <span class="badge auto">Auto</span>
    </Show>
  </div>
</td>

Option 5: Bulk Configuration Modal

Allow setting preferences for multiple models at once:

export function BulkProviderModal() {
  const [selectedModels, setSelectedModels] = createSignal<string[]>([])
  const [preference, setPreference] = createSignal({})
  
  return (
    <Dialog title="Bulk Provider Configuration">
      <ModelSelector 
        models={allModels}
        selected={selectedModels()}
        onChange={setSelectedModels}
      />
      <ProviderPreferenceForm 
        value={preference()}
        onChange={setPreference}
      />
      <DialogActions>
        <Button onClick={applyBulk}>Apply to {selectedModels().length} models</Button>
      </DialogActions>
    </Dialog>
  )
}

---

Part 6: UI/UX Recommendations

Recommended Approach: Combination of Options 1 + 4

1. Main Table View: Add compact provider indicator (Option 4) to show current preference
2. Quick Change: Dropdown for common presets (Option 1)
3. Advanced Settings: Link to detailed configuration page (Option 3)

Visual Design

┌─────────────────────────────────────────────────────────────────────────┐
│ Models                                                                   │
│                                                                          │
│ ┌─────────────────────────────────────────────────────────────────────┐ │
│ │ Model          │ Provider      │ Status  │ Enabled                  │ │
│ ├─────────────────────────────────────────────────────────────────────┤ │
│ │ 🟣 Claude 4    │ [Anthropic ▼] │ ●       │ [====]                   │ │
│ │ 🟢 GPT-4o      │ [Auto ▼]      │ Auto    │ [====]                   │ │
│ │ 🔵 Gemini 2    │ [Google ▼]    │ ●       │ [====]                   │ │
│ │ 🟡 Grok 2      │ [Auto ▼]      │ Auto    │ [====]                   │ │
│ └─────────────────────────────────────────────────────────────────────┘ │
│                                                                          │
│ 💡 Click provider dropdown to change routing preferences                │
└─────────────────────────────────────────────────────────────────────────┘

Dropdown Options

Each model's provider dropdown should include:

1. Auto (default) - Use OpenRouter's load balancing
2. [Provider] only - Strict provider requirement (no fallbacks)
3. Prefer [Provider] - Prefer this provider with fallbacks
4. Advanced... - Open detailed settings modal

---

Part 7: Implementation Phases

Phase 1: Backend Foundation (Week 1)

• Create database schema for provider preferences
• Add API endpoints for CRUD operations
• Modify request handler to apply preferences
• Add tests for provider selection logic

Phase 2: Basic UI (Week 2)

• Add provider column to model settings table
• Implement dropdown for common presets
• Add visual indicators for current preference
• Wire up API calls to save preferences

Phase 3: Advanced Features (Week 3)

• Create detailed provider settings page
• Implement drag-drop provider ordering
• Add bulk configuration modal
• Add provider blacklist/whitelist UI

Phase 4: Polish & Testing (Week 4)

• Add loading states and error handling
• Implement optimistic updates
• Add comprehensive tests
• Documentation updates

---

Part 8: Testing Strategy

Unit Tests

// packages/console/core/src/model.test.ts

describe("Model.setProviderPreference", () => {
  it("should store provider preference for model", async () => {
    const result = await Model.setProviderPreference({
      model: "claude-sonnet-4",
      preference: {
        order: ["anthropic"],
        allow_fallbacks: false,
      },
    })
    expect(result).toBeDefined()
  })
})

describe("Provider selection with preference", () => {
  it("should use preferred provider when set", () => {
    const provider = selectProvider(/* ... */, {
      order: ["anthropic"],
      allow_fallbacks: false,
    })
    expect(provider.id).toBe("anthropic")
  })
  
  it("should fallback when allow_fallbacks is true", () => {
    // Mock anthropic provider as disabled
    const provider = selectProvider(/* ... */, {
      order: ["anthropic"],
      allow_fallbacks: true,
    })
    expect(provider.id).not.toBe("anthropic")
  })
})

Integration Tests

// packages/console/app/src/routes/zen/v1/chat/completions.test.ts

describe("Chat completions with provider preference", () => {
  it("should route to preferred provider", async () => {
    // Set preference
    await Model.setProviderPreference({
      model: "claude-sonnet-4",
      preference: { order: ["anthropic"], allow_fallbacks: false },
    })
    
    // Make request
    const response = await fetch("/api/v1/chat/completions", {
      method: "POST",
      body: JSON.stringify({
        model: "claude-sonnet-4",
        messages: [{ role: "user", content: "Hello" }],
      }),
    })
    
    // Verify it went to Anthropic
    expect(response.status).toBe(200)
  })
})

---

Part 9: Security Considerations

Access Control

• Only workspace admins can modify provider preferences
• Validate provider slugs against allowed list
• Prevent injection of malicious provider IDs

Data Validation

const ALLOWED_PROVIDERS = [
  "anthropic", "openai", "google", "azure", "together",
  "fireworks", "groq", "deepinfra", "bedrock", "vertex-ai",
  // ... add as needed
]

const ProviderPreferenceSchema = z.object({
  order: z.array(z.enum(ALLOWED_PROVIDERS)).optional(),
  only: z.array(z.enum(ALLOWED_PROVIDERS)).optional(),
  ignore: z.array(z.enum(ALLOWED_PROVIDERS)).optional(),
  allow_fallbacks: z.boolean().optional(),
})

---

Part 10: Documentation Needs

1. User Documentation

   • How to set provider preferences
   • When to use strict vs. fallback mode
   • Provider-specific considerations

2. API Documentation

   • New endpoints for provider preferences
   • Request/response schemas
   • Error codes

3. Internal Documentation

   • Architecture decisions
   • Provider selection algorithm
   • Database schema

---

Appendix A: Provider Slug Reference

Provider	Slug	Notes
Anthropic	anthropic	Direct API
OpenAI	openai	Direct API
Google AI Studio	google	Direct API
Google Vertex AI	vertex-ai	Enterprise
AWS Bedrock	bedrock	Enterprise
Azure OpenAI	azure	Enterprise
Together AI	together	Third-party
Fireworks AI	fireworks	Third-party
Groq	groq	High-speed
DeepInfra	deepinfra	Cost-effective
Replicate	replicate	Various models
Perplexity	perplexity	Online models

---

Appendix B: Related Files

Files to Modify

• packages/console/core/src/model.ts - Add preference functions
• packages/console/core/src/schema/model-provider.sql.ts - New schema
• packages/console/app/src/routes/zen/util/handler.ts - Apply preferences
• packages/console/app/src/routes/workspace/[id]/model-section.tsx - UI changes
• packages/console/app/src/i18n/en.ts - Add translations

Files to Create

• packages/console/core/src/schema/model-provider.sql.ts - Database schema
• packages/console/app/src/routes/workspace/[id]/model-provider-settings.tsx - Advanced UI
• packages/console/app/src/component/provider-preference-dropdown.tsx - Reusable component

---

Appendix C: OpenRouter API Reference

Full provider object schema from OpenRouter API:

{
  "provider": {
    "order": ["string"],
    "allow_fallbacks": true,
    "require_parameters": false,
    "data_collection": "allow",
    "zdr": false,
    "enforce_distillable_text": false,
    "only": ["string"],
    "ignore": ["string"],
    "quantizations": ["int4", "int8", "fp8", "fp16", "bf16", "fp32"],
    "sort": "price | throughput | latency",
    "preferred_min_throughput": 0,
    "preferred_max_latency": 0,
    "max_price": {
      "prompt": 0,
      "completion": 0
    }
  }
}