> For the complete documentation index, see [llms.txt](https://docs.mozilla.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mozilla.ai/any-guardrail/api-reference/index/content-safety/duo-guard.md).

# DuoGuard

Multilingual multi-label safety classifier scoring text across 12 harm categories including jailbreak prompts.

DuoGuard is a compact (0.5B-1.5B) classifier built on Qwen 2.5 and Llama 3.2 backbones, trained with a two-player reinforcement-learning framework in which a generator and the guardrail co-evolve to synthesize multilingual safety data. It emits an independent probability (sigmoid) for each of the 12 categories in `DUOGUARD_CATEGORIES` — the MLCommons-style hazard taxonomy (violent crimes, non-violent crimes, sex-related crimes, child sexual exploitation, specialized advice, privacy, intellectual property, indiscriminate weapons, hate, suicide and self-harm, sexual content) plus a jailbreak-prompt category.

The models are fine-tuned primarily for English, French, German, and Spanish, with broader coverage inherited from the Qwen 2.5 / Llama 3.2 base models.

Verdict mapping onto `GuardrailOutput`:

* `categories` carries all 12 categories, each with its sigmoid probability and a `triggered` flag (probability strictly above `threshold`).
* `valid` is `True` only when no category is triggered.
* `score` (canonical risk: higher = riskier) is the maximum category probability.

Expected inputs: a single text string, or a `list[str]` for batched classification (the inherited `ThreeStageGuardrail.validate` handles list input). It screens a single body of text; it does not take a separate prompt / response pair.

For more information, see:

* [DuoGuard model collection](https://huggingface.co/collections/DuoGuard/duoguard-models-67a29ad8bd579a404e504d21).
* [DuoGuard-0.5B model card](https://huggingface.co/DuoGuard/DuoGuard-0.5B) (default).
* [DuoGuard-1B-Llama-3.2-transfer model card](https://huggingface.co/DuoGuard/DuoGuard-1B-Llama-3.2-transfer).
* [DuoGuard-1.5B-transfer model card](https://huggingface.co/DuoGuard/DuoGuard-1.5B-transfer).
* [DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails (arXiv:2502.05163)](https://arxiv.org/abs/2502.05163).

## Supported Models

* `DuoGuard/DuoGuard-0.5B`
* `DuoGuard/DuoGuard-1B-Llama-3.2-transfer`
* `DuoGuard/DuoGuard-1.5B-transfer`

## Constructor

| Parameter   | Type                                                 | Required | Default | Description                                                                                                                                                                                                                                                                                             |
| ----------- | ---------------------------------------------------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `model_id`  | \`str                                                | None\`   | No      | `None`                                                                                                                                                                                                                                                                                                  |
| `threshold` | `float`                                              | No       | `0.5`   | Per-category probability strictly above which that category is flagged (and the text becomes invalid). Defaults to 0.5, from the model card.                                                                                                                                                            |
| `provider`  | `Optional[Provider[dict[str, Any], dict[str, Any]]]` | No       | `None`  | Optional pre-configured provider. If `None`, a default `HuggingFaceProvider` is built with `multi_label=True` and the matching base tokenizer (see `MODELS_TO_TOKENIZER`), then the model is loaded eagerly. A pad token is set from the EOS token because the Qwen-family tokenizers ship without one. |

Initialize the DuoGuard guardrail.

## validate

Default validation pipeline: preprocess -> inference -> postprocess.

**Parameters**

| Parameter    | Type  | Required     | Default | Description |
| ------------ | ----- | ------------ | ------- | ----------- |
| `input_text` | \`str | list\[str]\` | Yes     | —           |

**Returns:** `GuardrailOutput | list[GuardrailOutput]`

## Benchmarks

### Content Safety

| Dataset (rev)                    | Metric | Threshold    | Value    | Harness                 | Source                           | Contam. |
| -------------------------------- | ------ | ------------ | -------- | ----------------------- | -------------------------------- | ------- |
| openai\_moderation (unspecified) | f1     | native-valid | 0.789668 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| xstest (unspecified)             | fpr    | native-valid | 0.028    | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| wildguardmix (unspecified)       | f1     | native-valid | 0.875    | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| aegis (unspecified)              | f1     | native-valid | 0.810997 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| jbb (unspecified)                | f1     | native-valid | 0.793814 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| orbench (unspecified)            | fpr    | native-valid | 0.242105 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |

### Prompt Injection

| Dataset (rev)             | Metric | Threshold    | Value     | Harness                 | Source                           | Contam. |
| ------------------------- | ------ | ------------ | --------- | ----------------------- | -------------------------------- | ------- |
| deepset\_pi (unspecified) | f1     | native-valid | 0.475     | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| notinject (unspecified)   | fpr    | native-valid | 0.0105263 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |
| gandalf (unspecified)     | recall | native-valid | 0.473214  | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |

### Toxicity

| Dataset (rev)                | Metric | Threshold    | Value    | Harness                 | Source                           | Contam. |
| ---------------------------- | ------ | ------------ | -------- | ----------------------- | -------------------------------- | ------- |
| real\_toxicity (unspecified) | f1     | native-valid | 0.308571 | guardrail-bench+ag0.7.4 | measured:guardrail-bench+ag0.7.4 |         |

## License

* **Vendor:** DuoGuard
* **Default license:** `apache-2.0` (of the default model/service)

| Model variant                             | License      |
| ----------------------------------------- | ------------ |
| `DuoGuard/DuoGuard-0.5B`                  | `apache-2.0` |
| `DuoGuard/DuoGuard-1.5B-transfer`         | `apache-2.0` |
| `DuoGuard/DuoGuard-1B-Llama-3.2-transfer` | `llama-3.2`  |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.mozilla.ai/any-guardrail/api-reference/index/content-safety/duo-guard.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
