Providers Overview

Connect to AI models from any provider

ai-query supports multiple AI providers through a unified interface. Each provider has its own factory function that creates a LanguageModel instance.

Supported Providers

OpenAI

GPT-5, GPT-4o, and o-series reasoning models

Anthropic

Claude 3.5 Sonnet, Claude 3 Opus, and more

Google

Gemini 2.0, Gemini 1.5 Pro and Flash

OpenRouter

Access any model through one API

DeepSeek

DeepSeek Chat and Reasoner models

Groq

Groq LPU Inference

Workers AI

Cloudflare Workers AI via OpenAI-compatible endpoints

Custom Providers

Build your own provider adapters and wrappers

Meta Llama

Official Meta Llama API

xAI

Grok Models via xAI API

Cloud Providers

Bedrock

AWS Bedrock - managed foundation models

Bedrock requires an extra dependency: pip install ai-query[bedrock] or uv add ai-query[bedrock]

Quick Start

Each provider requires an API key set as an environment variable:

# OpenAI
export OPENAI_API_KEY="sk-..."

# Anthropic
export ANTHROPIC_API_KEY="sk-ant-..."

# Google
export GOOGLE_API_KEY="..."

# OpenRouter
export OPENROUTER_API_KEY="sk-or-v1-..."

# DeepSeek
export DEEPSEEK_API_KEY="sk-..."

# Groq
export GROQ_API_KEY="gsk-..."

# Workers AI
export CLOUDFLARE_ACCOUNT_ID="..."
export CLOUDFLARE_API_TOKEN="..."

Usage

All providers follow the same pattern - import the factory function and use it with generate_text() or stream_text():

from ai_query import generate_text
from ai_query.providers import openai, anthropic, google, openrouter, deepseek, groq, workers_ai

# OpenAI
result = await generate_text(model=openai("gpt-4o"), prompt="Hello!")

# Anthropic  
result = await generate_text(model=anthropic("claude-sonnet-4-20250514"), prompt="Hello!")

# Google
result = await generate_text(model=google("gemini-2.0-flash"), prompt="Hello!")

# OpenRouter (access any model)
result = await generate_text(model=openrouter("openai/gpt-4o"), prompt="Hello!")

# DeepSeek
result = await generate_text(model=deepseek("deepseek-chat"), prompt="Hello!")

# Groq (ultra-fast inference)
result = await generate_text(model=groq("llama-3.3-70b-versatile"), prompt="Hello!")

# Workers AI
result = await generate_text(model=workers_ai("@cf/moonshotai/kimi-k2.5"), prompt="Hello!")

Switching Providers

One of the key benefits of ai-query is how easy it is to switch between providers:

from ai_query import generate_text
from ai_query.providers import openai, anthropic

# Just change the model - everything else stays the same!
async def generate_response(prompt: str, use_openai: bool = True):
    model = openai("gpt-4o") if use_openai else anthropic("claude-sonnet-4-20250514")
    
    result = await generate_text(
        model=model,
        prompt=prompt,
        tools=my_tools  # Same tools work with any provider
    )
    return result.text

Provider Options

Each provider supports specific options through the provider_options parameter:

result = await generate_text(
    model=openai("gpt-4o"),
    prompt="Be creative!",
    provider_options={
        "openai": {
            "temperature": 0.9,
            "max_tokens": 1000
        }
    }
)

See individual provider pages for available options.

OpenAI-Compatible Providers

OpenRouter, DeepSeek, Groq, and Workers AI use OpenAI-compatible APIs internally. This means:

• They share the same request/response format
• Provider options still use their own provider keys ("openrouter", "deepseek", "groq", "workers_ai")
• Tool calling works identically

# All four use the same underlying format
result = await generate_text(
    model=openrouter("openai/gpt-4o"),  # or deepseek(...), groq(...), workers_ai(...)
    prompt="Hello!",
    provider_options={
        "workers_ai": {"temperature": 0.7}
    }
)