> For the complete documentation index, see [llms.txt](https://docs.mozilla.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.mozilla.ai/any-guardrail/api-reference/index/flowjudge.md).

# FlowJudge

Wrapper around FlowJudge, allowing for custom guardrailing based on user defined criteria, metrics, and rubric.

Please see the model card for more information: [FlowJudge](https://huggingface.co/flowaicom/Flow-Judge-v0.1).

Two ways to specify the evaluation. Either supply the convenience fields (`name`/`criteria`/`rubric`/`required_inputs`/`required_output`) to build a metric, or pass a prebuilt `metric` — a `flow_judge` `Metric` / `CustomMetric` or one of the library's preset metrics (e.g. `RESPONSE_FAITHFULNESS_3POINT`).

Args: name: User defined metric name (convenience path). criteria: User defined question that they want answered by FlowJudge model (convenience path). rubric: A scoring rubric in a likert scale fashion, providing an integer score and then a description of what the value means (convenience path). required\_inputs: A list of what is required for the judge to consider (convenience path). required\_output: What is the expected output from the judge (convenience path). pass\_threshold: The rubric score at which the text counts as passing. `valid` is `rubric_score >= pass_threshold` (or `<=` when `higher_is_better` is False). higher\_is\_better: Whether higher rubric scores mean better/passing text. Set to False for rubrics where higher scores mean worse text. metric: A prebuilt `flow_judge` `Metric` / `CustomMetric` (e.g. a preset). When given, the convenience fields are not required and are ignored. model: A prebuilt `flow_judge` backend (`Hf`, `Vllm`, `Llamafile`, `Baseten`). Defaults to `Hf(flash_attn=False)`. Use this to pick a faster/quantized backend (install the matching `flow-judge[vllm|llamafile|baseten]` extra yourself). generation\_params: Generation parameters (`temperature`, `top_p`, `max_new_tokens`, `do_sample`) for the default `Hf` backend. Ignored when `model` is supplied.

Raises: ImportError: When the `flowjudge` extra is not installed. ValueError: When neither `metric` nor the full convenience field set is provided.

## Constructor

| Parameter           | Type              | Required | Default |
| ------------------- | ----------------- | -------- | ------- |
| `name`              | \`str             | None\`   | No      |
| `criteria`          | \`str             | None\`   | No      |
| `rubric`            | \`dict\[int, str] | None\`   | No      |
| `required_inputs`   | \`list\[str]      | None\`   | No      |
| `required_output`   | \`str             | None\`   | No      |
| `pass_threshold`    | `int`             | No       | `3`     |
| `higher_is_better`  | `bool`            | No       | `True`  |
| `metric`            | \`Any             | None\`   | No      |
| `model`             | \`Any             | None\`   | No      |
| `generation_params` | \`dict\[str, Any] | None\`   | No      |

Initialize the FlowJudgeClass.

## validate

Classifies the desired input and output according to the associated metric provided to the judge.

**Parameters**

| Parameter | Type                   | Required | Default |
| --------- | ---------------------- | -------- | ------- |
| `inputs`  | `list[dict[str, str]]` | Yes      | —       |
| `output`  | `dict[str, str]`       | Yes      | —       |

**Returns:** `GuardrailOutput`


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.mozilla.ai/any-guardrail/api-reference/index/flowjudge.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.