For the complete documentation index, see llms.txt. This page is also available as Markdown.

Video Generation

Generate videos from a prompt and images as a job you poll

Overview

The video API turns a text prompt, optionally with a first frame, a last frame or reference images, into a short video. Rendering takes from tens of seconds to several minutes, so every provider returns a job first:

  1. video_generation() starts the job and returns a VideoJob, usually queued or in_progress.

  2. retrieve_video() reads the job again. poll_video() does that for you until the job is completed or failed.

  3. download_video() returns the finished video as bytes, whether the provider handed back a URL or inline data.

Each function has an async twin with an a prefix: avideo_generation(), aretrieve_video(), apoll_video() and adownload_video().

Supported Providers

Provider

SUPPORTS_VIDEO_GENERATION

Notes

DeepInfra

Yes

POST /v1/inference/{model} on DeepInfra's native API, for models such as ByteDance/Seedance-2.0 and Wan-AI/Wan3.0-Video. DeepInfra renders inside the request and has no endpoint to look a request up afterwards, so video_generation blocks for the whole render (up to 900 seconds by default; pass timeout to change it) and returns a job that is already completed or failed. retrieve_video and poll_video read that job back from memory, which keeps the newest 1000 jobs started in the same process; any other id raises InvalidRequestError. Pass DeepInfra's webhook option (a URL you serve) and video_generation instead returns at once the queued job with DeepInfra's request_id; retrieve_video answers queued for it until you pass the body DeepInfra posts to your webhook to complete_video_job("deepinfra", payload), which returns the finished job and stores it for retrieve_video. The callback does not name the model, so pass model when the job was started in another process. Your HTTP request can then time out freely without losing the job or its charge. Every DeepInfra video model has its own request fields, so the provider fetches the model's input schema from GET /models/{model} once per process and maps each any-llm field to whatever name that schema gives it: seconds to duration, size to a resolution tier (by short side, 720p or 720P) plus the nearest listed aspect_ratio/ratio, or to an exact size such as 1280*720; first_frame/last_frame to first_frame_image/first_frame fields or to typed media items; reference_images and reference_videos to fields of the same name or to media items; reference_audios to reference_audios, to media items, or (one clip only) to a single-clip field such as Wan 2.6's audio_url; seed and negative_prompt to the same names; generate_audio to the model's boolean generate_audio, generate_audio_switch (Pixverse 6) or audio (Wan 3.0) field. A keyword argument or extra_body value that sets a field one of these any-llm fields set, to another value, raises InvalidRequestError; the same value passes. A field the schema has no name for raises UnsupportedParameterError, and a size the model does not render, or whose shape it has no field for, raises InvalidRequestError, both before any render is billed. Model-only fields such as Wan's media pass through as keyword arguments (media items merge with the ones any-llm produces, but items of a kind any-llm produced, such as first_frame or reference_image, must be the same ones the any-llm field gives). The finished video is a public URL. The job carries what DeepInfra billed: cost in USD from the reply's inference_status, and tokens from its out_tokens.

Gemini

Yes

Gemini Omni Flash (gemini-omni-1.1-flash) through the Interactions API, created with background=True and read back with interactions.get. size maps to Omni's 16:9/9:16 aspect ratio and 360p/720p/1080p/4k resolution, so the short side must be 360, 720, 1080 or 2160 pixels and the shape must be exactly 16:9 or 9:16; anything else raises InvalidRequestError before a render is billed. first_frame and last_frame go in as image parts before the prompt, in that order, which is how Omni assigns their roles; reference_images follow them. seed goes in generation_config. A response_format or generation_config keyword argument, as a dict or an SDK model, merges with the one any-llm builds (for a list of response formats, with its video format), and one that sets aspect_ratio, resolution or seed to another value than the shared fields raises InvalidRequestError. seconds, negative_prompt, reference_audios, reference_videos and generate_audio raise UnsupportedParameterError: put what to avoid in the prompt, and describe the sound track there too (or ask for none), since Omni always tries to add its own audio, has no field to turn it off, and documents audio references as unsupported. Omni reads a video part as a clip to edit, extend or reference by what the prompt asks, so any-llm does not map reference_videos to it yet. The finished video arrives inline (base64) or as a Files API URI, which download_video fetches with your API key.

Together

Yes

POST /v2/videos and GET /v2/videos/{id} through the Together SDK, for models such as ByteDance/Seedance-2.0. size maps to width/height, resolution and aspect_ratio to resolution and ratio, seconds to the string Together expects, and generate_audio to generate_audio. Image, audio and video inputs go in media: first_frame/last_frame as media.frame_images entries, reference_images as media.reference_images, reference_audios as media.audio_inputs, and reference_videos as media.reference_videos with {"video": url} items. A media you pass, as a keyword argument or in extra_body, merges with these; where both set a key, the keyword argument wins. Together-only options such as fps, steps, guidance_scale and output_format pass through as keyword arguments. A keyword argument, extra_body or media value, or first or last media.frame_images entry that sets a field one of these any-llm fields (or prompt) set, to another value, raises InvalidRequestError. Seedance renders fixed resolutions only (for example 864x496, 1280x720, 1920x1080, and their portrait twins); a size that is not WIDTHxHEIGHT raises InvalidRequestError before any request is sent, and any other WIDTHxHEIGHT is accepted, but the job then fails with code unsupportedModelResolution and the full list in error.message. The finished video is a public URL.

All other providers return SUPPORTS_VIDEO_GENERATION = False and raise NotImplementedError if called. OpenAI is among them: OpenAI removed the Sora 2 models and the Videos API on September 24, 2026, with no replacement.

Installation

Each video provider needs its own extra:

pip install any-llm-sdk[deepinfra]
pip install any-llm-sdk[gemini]
pip install any-llm-sdk[together]

Quick Start

Write video to a file with the extension its mime_type implies, .mp4 for video/mp4.

The same call works against Gemini by changing the model string and dropping seconds, which Omni Flash does not take:

Shape

A video's shape is two facts: how sharp, and landscape or portrait. Pass them as one size, or as a resolution tier plus an aspect_ratio:

Each provider turns either form into its own fields. The tier form is what most models bill by; the exact form is for models such as Wan 2.6 that list fixed sizes. Passing size together with either of the other two raises InvalidRequestError.

Frames and references

Images are URLs, provider file URIs, or data: URLs with base64 content. A provider that cannot take one of them says so with UnsupportedParameterError.

Audio

Audio must be a hosted URL the provider can download; no provider documents base64 audio. reference_videos takes video the same way, as a URL the provider can download, or as a file URI on a provider that has one. The same call works against Together by changing the model string. How a model refers to its references inside the prompt is the model's own convention: Seedance 2.0 uses @Image1 and @Audio1, Wan 3.0 uses Image 1, and Seedance 2.5 takes plain prose.

Async

API Reference

video_generation() / avideo_generation()

Parameter
Type
Default
Description

model

str

required

Provider-prefixed model ID (e.g. "together:ByteDance/Seedance-2.0")

prompt

str

required

What the video shows

seconds

int | None

None

Length of the video in seconds

size

str | None

None

WIDTHxHEIGHT, such as "1280x720". Says both how sharp and what shape; not combined with the next two

resolution

str | None

None

How sharp, by short side: "480p", "720p", "1080p" or "4k"

aspect_ratio

str | None

None

What shape, as WIDTH:HEIGHT such as "16:9" or "9:16"

seed

int | None

None

Seed for reproducible generation

negative_prompt

str | None

None

What the video should not contain

reference_images

list[str] | None

None

Images the video takes its subjects or style from

reference_audios

list[str] | None

None

Audio URLs the video follows, for lip sync, beat-matched motion or a sound track

reference_videos

list[str] | None

None

Video URLs or provider file URIs the video takes its motion, subjects or style from

first_frame

str | None

None

Image the video starts from

last_frame

str | None

None

Image the video ends on

generate_audio

bool | None

None

Whether the video carries a sound track. None leaves the provider's default.

provider

str | None

None

Explicit provider name. Inferred from model if omitted.

api_key

str | None

None

Provider API key. Falls back to the environment variable.

api_base

str | None

None

Provider API base URL. Falls back to the environment variable.

client_args

dict | None

None

Additional arguments for the provider client constructor

Any other keyword argument is passed to the provider's API call unchanged, unless it sets a request field one of the parameters above already set, to another value. That raises InvalidRequestError before any render is billed, because a gateway prices a render from these parameters. The same value passes.

retrieve_video() / aretrieve_video()

Take provider and video_id and return the job as it is now.

poll_video() / apoll_video()

Take provider and video_id, retrieve the job every poll_interval seconds (default 5) and return it once it is completed or failed. A failed job is returned, not raised, so its error can be read. After timeout seconds (default 600; None waits forever) they raise TimeoutError.

download_video() / adownload_video()

Take provider and one entry of VideoJob.outputs and return the video as bytes. Inline data is decoded; a URL is fetched, with the provider's credentials where its URLs need them.

complete_video_job()

Take provider and the JSON body the provider posted to your webhook, and return the job it describes, usually completed or failed. Pass model when the callback does not name the model. retrieve_video then reads the job back in the same process. Nothing is sent to the provider, so there is no async variant. Providers that do not deliver video jobs to a webhook raise NotImplementedError.

VideoJob

Field
Type
Description

id

str

Provider-assigned job ID; pass it to retrieve_video

model

str

Model that renders the video

status

"queued" | "in_progress" | "completed" | "failed"

Where the job is. Providers' own words map onto these four.

done

bool

True once status is completed or failed

created_at

int | None

Unix timestamp of the request

completed_at

int | None

Unix timestamp of completion, when the provider reports it

progress

int | None

Percent complete, when the provider reports it

seconds

int | None

Length of the video, when the provider reports it

size

str | None

WIDTHxHEIGHT of the video, when the provider reports it

cost

float | None

What the provider charged for the job, in USD, when it reports it

tokens

int | None

The units the provider bills the job by, when it bills video by tokens and reports them

error

VideoError | None

Why a failed job failed

outputs

list[VideoOutput]

The rendered videos; empty until completed

VideoOutput

Field
Type
Description

url

str | None

Where the video can be fetched

b64_data

str | None

The video, base64-encoded, when the provider returns it inline

mime_type

str

Media type of the video, video/mp4 unless the provider says otherwise

Last updated