> For the complete documentation index, see [llms.txt](https://docs.mozilla.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mozilla.ai/quickstart.md).

# Quickstart

Install any-llm and make your first API call in 5 minutes

### Requirements

* Python 3.11 or newer
* API keys for your chosen LLM provider

### Installation

```bash
pip install any-llm-sdk[all]  # Install with all provider support
```

#### Installing Specific Providers

If you want to install a specific provider from our [supported providers](/providers.md):

```bash
pip install any-llm-sdk[mistral]  # For Mistral provider
pip install any-llm-sdk[ollama]   # For Ollama provider
# install multiple providers
pip install any-llm-sdk[mistral,ollama]
```

#### Library Integration

If you're building a library, install just the base package (`pip install any-llm-sdk`) and let your users install provider dependencies.

> **API Keys:** Set your provider's API key as an environment variable (e.g., `export MISTRAL_API_KEY="your-key"`) or pass it directly using the `api_key` parameter.

### APIs

#### Using the AnyLLM Class

For applications making multiple requests with the same provider, use the `AnyLLM` class to avoid repeated provider instantiation:

```python
import os

from any_llm import AnyLLM

# Make sure you have the appropriate API key set
api_key = os.environ.get('MISTRAL_API_KEY')
if not api_key:
    raise ValueError("Please set MISTRAL_API_KEY environment variable")

llm = AnyLLM.create("mistral")

response = llm.completion(
    model="mistral-small-latest",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

metadata = llm.get_provider_metadata()
print(f"Supports streaming: {metadata.streaming}")
print(f"Supports tools: {metadata.completion}")
```

#### API Call

```python
import os

from any_llm import completion

# Make sure you have the appropriate API key set
api_key = os.environ.get('MISTRAL_API_KEY')
if not api_key:
    raise ValueError("Please set MISTRAL_API_KEY environment variable")

# Recommended: separate provider and model parameters
response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```

#### When to Choose Which Approach

**Use Direct API Functions (`completion`, `acompletion`) when:**

* Making simple, one-off requests
* Prototyping or writing quick scripts
* You want the simplest possible interface

**Use Provider Class (`AnyLLM.create`) when:**

* Building applications that make multiple requests with the same provider
* You want to avoid repeated provider instantiation overhead

**Finding model names:** Check the [providers page](/providers.md) for provider IDs, or use the [`list_models`](/api-reference/list-models.md) API to see available models for your provider.

### Custom OpenAI-compatible Endpoints

If your gateway or server speaks the OpenAI API but is not one of the [supported providers](/providers.md), point any-llm at it directly with `AnyLLM.create_openai_compatible`. The provider reports the name you give it rather than reporting itself as `openai`, and is used exactly like any other provider instance:

```python
from any_llm import AnyLLM

llm = AnyLLM.create_openai_compatible(
    name="mygateway",
    api_base="https://mygateway.example/v1",
    api_key="your-key",  # optional for keyless local servers
)

response = llm.completion(
    model="some-model",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)
```

Capability flags follow the OpenAI-compatible defaults. A capability the endpoint does not implement fails either locally with `NotImplementedError` (for flag-gated capabilities such as batch) or with the endpoint's own error for calls that any-llm forwards. Use this whenever you need an OpenAI-compatible endpoint that any-llm does not ship a dedicated provider for.

### Streaming

For the [providers that support streaming](/providers.md), you can enable it by passing `stream=True`:

```python
output = ""
for chunk in completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True
):
    chunk_content = chunk.choices[0].delta.content or ""
    print(chunk_content)
    output += chunk_content
```

### Reasoning

For [providers that support reasoning](/providers.md), you can request thinking traces alongside the response using `reasoning_effort`:

```python
from any_llm import completion

response = completion(
    model="claude-sonnet-4-5-20250929",
    provider="anthropic",
    messages=[{"role": "user", "content": "How many r's are in strawberry?"}],
    reasoning_effort="high",
)

# Access the model's thinking trace
if response.choices[0].message.reasoning:
    print(response.choices[0].message.reasoning.content)

# The final answer
print(response.choices[0].message.content)
```

Reasoning also works with streaming — each chunk may include `chunk.choices[0].delta.reasoning`.

### Embeddings

`embedding` and `aembedding` allow you to create vector embeddings from text using the same unified interface across providers.

Not all providers support embeddings - check the [providers documentation](/providers.md) to see which ones do.

```python
from any_llm import embedding

result = embedding(
    model="text-embedding-3-small",
    provider="openai",
    inputs="Hello, world!" # can be either string or list of strings
)

# Access the embedding vector
embedding_vector = result.data[0].embedding
print(f"Embedding vector length: {len(embedding_vector)}")
print(f"Tokens used: {result.usage.total_tokens}")
```

### Moderation

`moderation` and `amoderation` run a content-safety classifier against input (or output) text and return a normalized, OpenAI-compatible result.

Not all providers support moderation; calling an unsupported provider raises `NotImplementedError`. Today, **OpenAI** and **Mistral** implement the API.

```python
from any_llm import moderation

result = moderation(
    model="omni-moderation-latest",
    provider="openai",
    input="I want to hurt someone",
)

print(result.results[0].flagged)       # True
print(result.results[0].categories)    # {"violence": True, ...}
```

Pass `include_raw=True` to populate `ModerationResult.provider_raw` with the untouched provider response (useful for debugging or provider-specific fields).

### Tools

`any-llm` supports tool calling for providers that support it. You can pass a list of tools where each tool is either:

1. **Python callable** - Functions with proper docstrings and type annotations
2. **OpenAI Format tool dict** - Already in OpenAI tool format

```python
from any_llm import completion

def get_weather(location: str, unit: str = "F") -> str:
    """Get weather information for a location.

    Args:
        location: The city or location to get weather for
        unit: Temperature unit, either 'C' or 'F'

    Returns:
        Current weather description
    """
    return f"Weather in {location} is sunny and 75{unit}!"

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "What's the weather in Pittsburgh PA?"}],
    tools=[get_weather]
)
```

any-llm automatically converts your Python functions to OpenAI tools format. Functions must have:

* A docstring describing what the function does
* Type annotations for all parameters
* A return type annotation

### Exception Handling

The `any-llm` package provides a unified exception hierarchy that works consistently across all LLM providers.

#### Enabling Unified Exceptions

{% hint style="info" %}
**Opt-in Feature:** Unified exception handling is currently **opt-in**. Set the `ANY_LLM_UNIFIED_EXCEPTIONS` environment variable to enable it:
{% endhint %}

```bash
export ANY_LLM_UNIFIED_EXCEPTIONS=1
```

When enabled, provider-specific exceptions are automatically converted to `any-llm` exception types. When disabled (default), the original provider exceptions are raised with a deprecation warning.

#### Basic Usage

```python
from any_llm import completion
from any_llm.exceptions import (
    AnyLLMError,
    AuthenticationError,
    InvalidRequestError,
    ModelNotFoundError,
    ProviderError,
    RateLimitError,
)

try:
    response = completion(
        model="gpt-4",
        provider="openai",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except ModelNotFoundError as e:
    print(f"Model not found: {e.message}")
except RateLimitError as e:
    print(f"Rate limited: {e.message}")
except AuthenticationError as e:
    print(f"Auth failed: {e.message}")
except InvalidRequestError as e:
    print(f"Invalid request: {e.message}")
except ProviderError as e:
    print(f"Provider error: {e.message}")
except AnyLLMError as e:
    print(f"Error: {e.message}")
```

#### Accessing Original Exceptions

All unified exceptions preserve the original provider exception for debugging:

```python
from any_llm.exceptions import RateLimitError

messages = [{"role": "user", "content": "Hello!"}]

try:
    response = completion(model="gpt-4", provider="openai", messages=messages)
except RateLimitError as e:
    print(f"Provider: {e.provider_name}")
    print(f"Original exception: {type(e.original_exception)}")
```

#### Structured Error Metadata

Unified exceptions also carry the structured HTTP fields the provider reported, so you can classify a failure without unwrapping `original_exception` and coupling to a specific SDK's attribute layout:

| Attribute     | Meaning                                                             |
| ------------- | ------------------------------------------------------------------- |
| `status_code` | HTTP status the provider returned                                   |
| `code`        | Provider-specific error code from the response body                 |
| `param`       | Request field the provider flagged as the cause                     |
| `error_type`  | Provider-specific error category, such as `"invalid_request_error"` |

```python
from any_llm.exceptions import InvalidRequestError

try:
    response = completion(model="gpt-4", provider="openai", messages=messages)
except InvalidRequestError as e:
    if e.status_code == 400 and e.param == "reasoning_effort":
        print("Retry with reasoning_effort set to 'none'")
```

These fields are populated best-effort from the shapes provider SDKs expose: an attribute on the exception, or the parsed response body. Coverage varies by provider, so treat every field as optional. Anything `any-llm` cannot recover is `None`, including `status_code` for a non-HTTP failure such as a timeout or connection error.

`status_code` also drives which exception type you get. See [Exceptions](/api-reference/exceptions.md) for the full status-to-type mapping.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.mozilla.ai/quickstart.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.