> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openreward.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat Backends

> A composable channel for grader model calls

## Goals

* Learn how to build composable chat backends for environments, for use cases like grading and user simulations
* Understand why this traffic should not silently follow `OPENAI_*` environment variables

## Prerequisites

* `openreward>=0.1.153`
* Familiarity with [LLM graders](/environments/using-llm-graders)

## Introduction

Environments can use a regular provider SDK (OpenAI, Anthropic, Google, and so on) directly, as the [LLM graders](/environments/using-llm-graders) tutorial shows. For a model call that only ever runs against one provider, that works fine. This applies to any model the environment itself calls: judges, rubric graders, simulated users, and scripted opponents.

The problem appears when your environment runs inside a larger system that *reconfigures* provider clients through the process environment. Training harnesses commonly inject `OPENAI_BASE_URL` (and `OPENAI_API_KEY`) into the environment container so that **policy** traffic is served by an internal inference gateway. The `openai` SDK reads those variables implicitly, so a grader built as a plain `openai.AsyncClient(api_key=...)` gets silently repointed too. If the gateway does not serve your judge model, the judge fails, the failure is usually swallowed by retry logic, and every rollout quietly scores zero.

`openreward.chat_backends` makes grader clients **compose** with that kind of orchestration instead of fighting it:

* the *environment author* writes one call (`resolve_backend(secrets=secrets)`) and never hardcodes a provider client;
* the *operator* decides at runtime where grader traffic goes, separately from policy traffic, using `CHAT_*` settings;
* the *provider* is swappable (`CHAT_PROVIDER`, `register_backend`) without touching grading code.

## Basic usage

Build the grader client in your environment's `__init__` and call it in your grading code:

```python theme={null}
from openreward.chat_backends import resolve_backend

class MyEnvironment(Environment):
    def __init__(self, task_spec: JSONObject = {}, secrets: dict[str, str] = {}):
        super().__init__(task_spec)
        # Raises ValueError if no key is available: a grader without
        # credentials should fail loudly at session start.
        self.judge_client = resolve_backend(secrets=secrets)
```

```python theme={null}
resp = await self.judge_client.create(
    model="gpt-5-mini",
    messages=messages,                  # OpenAI chat format, including image_url blocks
    max_completion_tokens=1200,
)
verdict = parse(resp.text)              # normalised ChatCompletion: .text, .model,
                                        # .finish_reason, .usage, .raw
```

`create()` accepts extra provider kwargs verbatim (for example `response_format={"type": "json_object"}`) and raises provider exceptions unchanged, so your existing retry and fail-closed logic keeps working.

## How resolution works

The OpenAI backend has two modes, decided by whether **any** `CHAT_*` setting is present:

| Mode      | Trigger                                                       | Base URL                                              | API key                                                                      |
| --------- | ------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------- |
| Legacy    | No `CHAT_*` setting anywhere                                  | `OPENAI_BASE_URL` (or the OpenAI default)             | `openai_api_key` secret, then `OPENAI_API_KEY`                               |
| Dedicated | `chat_api_key` secret, `CHAT_API_KEY`, or `CHAT_BASE_URL` set | `CHAT_BASE_URL` (default `https://api.openai.com/v1`) | `chat_api_key` secret, then `CHAT_API_KEY`, then the `openai_api_key` secret |

With nothing configured, behaviour is identical to a plain client: existing sessions keep working unchanged. The moment an operator sets one `CHAT_*` variable, the grader ignores the `OPENAI_*` environment entirely. For a training run that overrides `OPENAI_BASE_URL`, a single extra override is enough to send judges back to the real endpoint:

```bash theme={null}
CHAT_BASE_URL=https://api.openai.com/v1    # key still comes from the openai_api_key secret
```

Session secrets always beat process environment variables, and the host-pinned `(value, [allowed_hosts])` secret form is understood.

## Swapping and adding providers

`CHAT_PROVIDER` selects the backend by name (default `openai`). An unrecognised name logs a warning and falls back to the default rather than raising: a config typo should degrade the provider choice, not kill a rollout mid-flight.

To add a provider, implement the small ABC and register it:

```python theme={null}
from openreward.chat_backends import ChatBackend, ChatCompletion, register_backend

class AnthropicBackend(ChatBackend):
    name = "anthropic"

    def __init__(self, *, config=None, secrets=None):
        ...

    async def create(self, *, model, messages, max_completion_tokens=None, **kwargs):
        ...
        return ChatCompletion(text=..., raw=...)

register_backend("anthropic", AnthropicBackend)
```

Grading code written against `create()` and `.text` now runs against either provider, selected per deployment with `CHAT_PROVIDER=anthropic`, with no code change. The same pattern works for any model role the environment owns: a simulated user or opponent built on `create()` can be pointed at a different provider or endpoint per deployment in exactly the same way.

## Escape hatches

If you need the full provider surface (streaming, embeddings), `OpenAIChatBackend.raw_client` exposes the underlying pinned `AsyncOpenAI` client. For synchronous scripts, `OpenAIChatConfig.from_env(secrets=...).client_kwargs()` gives you the resolved `api_key`/`base_url` to construct any client with the same resolution rules.

## Next Steps

<Card title="Using LLM Graders" icon="school" href="/environments/using-llm-graders">
  Learn how to integrate language model graders into an environment
</Card>