> For the complete documentation index, see [llms.txt](https://docs.mozilla.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mozilla.ai/any-guardrail/api-reference/index/content-safety/openai-moderation.md).

# OpenAI Moderation

Hosted moderation API flagging content across 13 harm categories with calibrated scores.

Wraps OpenAI's hosted moderation classifier (default model: `omni-moderation-latest`) to flag content across 13 harm sub-categories: hate, hate/threatening, harassment, harassment/threatening, self-harm, self-harm/intent, self-harm/instructions, sexual, sexual/minors, violence, violence/graphic, illicit, and illicit/violent. The classifier returns a calibrated per-category probability score alongside a boolean `flagged` verdict.

Expected input: a single string (or a list of strings, screened one at a time via the batched `ThreeStageGuardrail.validate`).

`GuardrailOutput` mapping: - `valid` is `False` when OpenAI flags the content **or** when the maximum per-category score exceeds `threshold` (otherwise `True`). - `score` is the maximum per-category probability (higher = riskier). - `categories` is the full per-category breakdown: each `CategoryResult` carries the calibrated `score` and a `triggered` flag (set when OpenAI flagged that category or its score exceeds `threshold`). - `raw` is the full OpenAI SDK moderation response object.

The current `omni-moderation` model is a GPT-4o-derived multimodal classifier; the original methodology (taxonomy, active-learning loop, calibration) is described in Markov et al. 2023 (AAAI 2023). The Moderation API is free and does not count toward standard usage quotas.

For more information, see:

* [Moderation guide (usage)](https://platform.openai.com/docs/guides/moderation)
* [Upgrading the Moderation API with our new multimodal moderation model](https://openai.com/index/upgrading-the-moderation-api-with-our-new-multimodal-moderation-model/)
* [A Holistic Approach to Undesired Content Detection in the Real World (arXiv:2208.03274)](https://arxiv.org/abs/2208.03274)

## Supported Models

* `omni-moderation-latest`
* `omni-moderation-2024-09-26`
* `text-moderation-latest`

## Constructor

| Parameter   | Type    | Required | Default | Description                                                                                                                        |
| ----------- | ------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `model_id`  | \`str   | None\`   | No      | `None`                                                                                                                             |
| `api_key`   | \`str   | None\`   | No      | `None`                                                                                                                             |
| `base_url`  | \`str   | None\`   | No      | `None`                                                                                                                             |
| `threshold` | `float` | No       | `0.5`   | Maximum per-category score above which content is considered invalid even if OpenAI did not explicitly flag it. Defaults to `0.5`. |

Initialize the OpenAI Moderation guardrail.

## validate

Default validation pipeline: preprocess -> inference -> postprocess.

**Parameters**

| Parameter    | Type  | Required     | Default | Description |
| ------------ | ----- | ------------ | ------- | ----------- |
| `input_text` | \`str | list\[str]\` | Yes     | —           |

**Returns:** `GuardrailOutput | list[GuardrailOutput]`

## Benchmarks

No benchmark results recorded yet. See the [benchmark methodology](/any-guardrail/api-reference/benchmarks.md) for how numbers are harvested (published) or measured and added.

## License

* **Vendor:** OpenAI
* **Default license:** `proprietary` (of the default model/service)


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.mozilla.ai/any-guardrail/api-reference/index/content-safety/openai-moderation.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.