> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openreward.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Multi-turn Conversations

> Serving a conversation history as the prompt

## Goals

* Learn how to serve a task's prompt as a role-tagged conversation instead of a single block of text
* Understand how harnesses consume conversation-shaped prompts, and how the feature degrades gracefully everywhere else

## Prerequisites

* `openreward>=0.1.157`
* Completing the [Your First Environment](/environments/your-first-environment) tutorial

## Introduction

Most environments serve their prompt with `get_prompt()`: a list of content blocks that every harness wraps in a single `{"role": "user"}` message. That remains the default — and for most tasks, the right — way to serve a prompt.

Some tasks, though, *are* conversations. A dialogue dataset where the model must write the next reply. A preference task built on chat transcripts. A support conversation the policy must continue. For those, the task carries the conversation so far — user turns, assistant turns, maybe a system prompt — and squeezing it into one user message loses the structure the model was trained on.

`get_messages()` is an additional, opt-in prompt surface for exactly these tasks. The environment returns role-tagged messages, the harness feeds them to the model as a real conversation, and the model writes the next assistant turn. Everything else about the episode — the tool loop, the terminal answer, rewards — proceeds exactly as it always has.

<Note>
  This is **not** user simulation. The conversation is task data, served once at the start of the episode. The environment does not generate new user turns mid-episode. (If you need a simulated user, see the pattern in [Chat Backends](/environments/chat-backends) — a simulated user is a model role the environment owns, called from inside a tool.)
</Note>

## Serving a conversation

Implement `get_messages()` instead of `get_prompt()` — one of the two is required, and each defaults to delegating to the other:

```python theme={null}
from openreward.environments import Environment, JSONObject, Message, TextBlock

class DialogueEnv(Environment):
    def __init__(self, task_spec: JSONObject = {}, secrets: dict[str, str] = {}):
        super().__init__(task_spec)
        self.conversation = task_spec["conversation"]

    def get_messages(self) -> list[Message]:
        return [
            Message(role=turn["role"], blocks=[TextBlock(text=turn["text"])])
            for turn in self.conversation
        ]

    @classmethod
    def list_tasks(cls, split: str) -> list[JSONObject]:
        return [
            {
                "id": "support-1",
                "conversation": [
                    {"role": "system", "text": "You are a support agent for Acme."},
                    {"role": "user", "text": "My order hasn't arrived."},
                    {"role": "assistant", "text": "Sorry to hear that — what's the order number?"},
                    {"role": "user", "text": "It's #4411, ordered two weeks ago."},
                ],
            }
        ]
```

A `Message` has a `role` (`"system"`, `"user"`, or `"assistant"`) and content `blocks` — `TextBlock` / `ImageBlock`, plus the [rich block types](#rich-conversations-reasoning-and-tool-history) below, so multimodal and agentic conversations work unchanged.

Note that the `{"role": ..., "text": ...}` dicts in this task spec are just this example's own convention — task specs are plain JSON, and the SDK doesn't prescribe how a conversation is stored in them. `get_messages()` is the translation point where your task data becomes `Message` objects; store whatever shape suits your data and build the blocks there.

Rewards are unaffected: grade the model's next turn however you would grade any response — a submit tool, a [terminal tool](/environments/using-terminal-tools), an [LLM grader](/environments/using-llm-graders) or [rubrics](/environments/using-rubrics).

## Consuming a conversation

On the client, `session.get_messages()` returns the conversation, with optional provider formatting:

```python theme={null}
with environment.session(task=task) as session:
    # Provider-shaped, ready to send:
    input_messages = session.get_messages(format="openai")
    response = oai_client.chat.completions.create(
        model="gpt-5.2", messages=input_messages, tools=tools,
    )
```

The `format` argument mirrors `list_tools(format=...)`:

| Format                      | Returns                                                                                                          |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `None`                      | `list[Message]` — the raw conversation                                                                           |
| `"openai"` / `"openrouter"` | a chat-completions message list                                                                                  |
| `"anthropic"`               | `{"system": Optional[str], "messages": [...]}` — leading system turns hoisted into the separate system parameter |
| `"google"`                  | `{"system_instruction": Optional[str], "contents": [...]}` — the assistant role mapped to `"model"`              |

## Rich conversations: reasoning and tool history

<Note>Requires `openreward>=0.1.157`.</Note>

A conversation the policy must continue often contains more than text: past reasoning (interleaved-thinking models carry reasoning across turns, and OpenReward reasoning is plain text meant to be passed back in) and tool interactions (an agent transcript is a conversation). Three further block types cover this, following **Anthropic message semantics** — everything is a content block, and no `role: "tool"` exists:

| Block             | Lives in          | Fields                                                                              |
| ----------------- | ----------------- | ----------------------------------------------------------------------------------- |
| `ReasoningBlock`  | `assistant` turns | `text`, optional `signature` (passthrough for provider-signed thinking)             |
| `ToolCallBlock`   | `assistant` turns | `id`, `name`, `input`                                                               |
| `ToolResultBlock` | `user` turns      | `tool_call_id`, `blocks` (ordinary text/image content — images welcome), `is_error` |

Placement is validated on the `Message` model itself — a `tool_result` in an assistant turn fails at construction with a clear error, not at the provider.

```python theme={null}
from openreward.environments import (Message, ReasoningBlock, TextBlock,
                                     ToolCallBlock, ToolResultBlock)

def get_messages(self) -> list[Message]:
    return [
        Message(role="user", blocks=[TextBlock(text="My order #4411 hasn't arrived.")]),
        Message(role="assistant", blocks=[
            ReasoningBlock(text="Two weeks late — check the carrier first."),
            ToolCallBlock(id="tc_1", name="lookup_order", input={"id": "4411"}),
        ]),
        Message(role="user", blocks=[
            ToolResultBlock(tool_call_id="tc_1",
                            blocks=[TextBlock(text='{"status": "stalled_at_hub"}')]),
        ]),
    ]
```

The example above builds blocks by hand; if you'd rather keep conversations in your task data, store them in the canonical `Message` wire shape and let pydantic parse them — all block types, placement validation included:

```python theme={null}
def get_messages(self) -> list[Message]:
    return [Message.model_validate(m) for m in self.task_spec["conversation"]]
```

```json theme={null}
{
  "conversation": [
    {"role": "user", "blocks": [{"type": "text", "text": "My order #4411 hasn't arrived."}]},
    {"role": "assistant", "blocks": [
      {"type": "reasoning", "text": "Two weeks late — check the carrier first."},
      {"type": "tool_call", "id": "tc_1", "name": "lookup_order", "input": {"id": "4411"}}
    ]},
    {"role": "user", "blocks": [
      {"type": "tool_result", "tool_call_id": "tc_1",
       "blocks": [{"type": "text", "text": "{\"status\": \"stalled_at_hub\"}"}]}
    ]}
  ]
}
```

`format` conversion handles each provider's shape wherever its schema can represent the block:

| Format                      | Reasoning                                                                                | Tool calls / results                                                                                                                               |
| --------------------------- | ---------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `"anthropic"`               | native `thinking` blocks (`signature` included only when set)                            | native `tool_use` / `tool_result`                                                                                                                  |
| `"openai"` / `"openrouter"` | `reasoning_content` field on the assistant message (the DeepSeek/vLLM/SGLang convention) | native `tool_calls`; each tool result becomes its own `role: "tool"` message. Images inside tool results raise — tool messages are text-only there |
| `"google"`                  | raises — no input representation for unencrypted reasoning                               | native `function_call` / `function_response` parts                                                                                                 |

Where a schema can't represent a block, conversion raises a clear `ValueError` rather than silently dropping content; `format=None` always returns the full canonical messages.

## Compatibility

Both prompt surfaces coexist, and `/prompt` remains the primary one:

* **Existing environments and harnesses change nothing.** A prompt-based environment behaves byte-identically to before.
* **`get_messages()` works against every environment.** Against a prompt-based environment it returns the prompt wrapped as a single user message; against a server older than `0.1.157` (which has no `/messages` route) the client falls back the same way. A harness can adopt `get_messages()` unconditionally.
* **Old harnesses still work against conversation environments.** `/prompt` is served for every environment; a conversation flattens to `[role]`-tagged text blocks (rich blocks render to text: reasoning as `<think>…</think>`, tool calls as `[tool_call name] {args}` lines). A conversation consisting of exactly one plain user message flattens to its blocks verbatim.
* **Rich blocks are versioned.** A server's supported block set is advertised as `message_block_types` in the tool list (absent on servers before `0.1.157`, meaning text/image only). Clients skip block types they don't know when reading `/messages` (lossy but safe), so future block types degrade the same way.

## Next Steps

<Card title="Preference Rewards" icon="scale-balanced" href="/environments/preference-rewards">
  Conversation-shaped tasks pair naturally with preference-based (A ≻ B) rewards
</Card>