Portkey model routing

Portkey routing chains models and providers with automatic failover — configure cheap models first so capable tiers only run on escalation.

Hub: Tokenminning with Portkey. Pattern: Model routing.

Fallback chain

{
  "strategy": {
    "mode": "fallback"
  },
  "targets": [
    { "provider": "openai", "model": "gpt-4o-mini" },
    { "provider": "openai", "model": "gpt-4o" }
  ]
}

Portkey tries gpt-4o-mini first. On rate limit or error, falls through to gpt-4o. You pay only for the successful attempt.

Config overrides per environment

Attach configs to API keys — budgets.

Client wiring

from portkey_ai import Portkey
 
client = Portkey(
    api_key="pk-...",
    config="pc-..."  # config with fallback chain
)
 
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello"}]
)

Tier selection

Spending guardrails

• First target in fallback chain must be cheapest viable tier
• Limit fallback depth — three targets max for agent loops
• Log which target succeeded in your middleware
• Avoid load-balanced random routing until you understand cost distribution

Related

• Portkey hub
• Budgets
• Observability