> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openreward.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Recording Rollouts
> Record and upload agent rollouts to OpenReward for viewing and analysis
## Goals
* Understand what rollouts are and why you would record them.
* Record rollouts from an agent interacting with an OpenReward environment.
* Use provider-specific logging methods for OpenAI, Anthropic, Google Gemini, and OpenRouter.
* View recorded rollouts in the OpenReward dashboard.
## Prerequisites
* An OpenReward [account](https://openreward.ai/)
* An OpenReward [API key](https://openreward.ai/keys)
* An API key and SDK for your model provider of choice (e.g. OpenAI, Anthropic, Google, OpenRouter)
* Familiarity with the [Your First Environment](/environments/your-first-environment) tutorial
## Setup
Install the OpenReward Python library:
```bash theme={null}
pip install openreward
```
## What are Rollouts?
A **rollout** is a recorded trace of an agent interacting with an environment. It captures every message exchange -- user prompts, assistant responses, reasoning steps, tool calls, and tool results -- along with associated rewards and metadata.
Recording rollouts is useful for:
* **Debugging**: Inspect exactly what your agent did step-by-step.
* **Analysis**: Compare performance across models, prompts, or environment configurations.
* **Training data**: Collect trajectories for supervised finetuning or midtraining.
* **Collaboration**: Your team can view runs on your organisation's OpenReward page.
The OpenReward Python SDK provides provider-specific logging methods that automatically serialize messages from OpenAI, Anthropic, and Google Gemini into a normalized format. For OpenRouter (which uses the OpenAI SDK), you use the OpenAI completions logger.
## Key Concepts
### Runs and Rollouts
A **run** groups related rollouts together under a single name (e.g. `"gpt-5.4-ctf-eval"`). Each **rollout** within a run represents one episode or conversation with the environment.
### Creating a Rollout
```python theme={null}
import openreward
or_client = openreward.OpenReward()
rollout = or_client.rollout.create(
run_name="my-eval-run", # Required: groups rollouts together
rollout_name="episode-1", # Optional: name for this specific rollout
environment="GeneralReasoning/CTF", # Optional: environment identifier
split="train", # Optional: data split
metadata={"model": "gpt-5.4"}, # Optional: custom key-value pairs
task_spec={"id": "task-0"} # Optional: task specification
)
```
### Logging Methods
The SDK provides these logging methods on a `Rollout` object:
| Method | Provider | Input Type |
| ----------------------------------------- | -------------------------------------- | --------------------------------------------------- |
| `rollout.log_openai_response(message)` | OpenAI (Responses API) | Response object or individual output items |
| `rollout.log_openai_completions(message)` | OpenAI (Chat Completions) / OpenRouter | Chat message dicts |
| `rollout.log_anthropic_message(message)` | Anthropic | Message dicts (Anthropic format) |
| `rollout.log_gdm_message(message)` | Google Gemini | `google.genai.types.Content` objects |
| `rollout.log(message)` | Generic | `UserMessage`, `AssistantMessage`, `ToolCall`, etc. |
Each method also accepts these parameters:
* `reward` (`Optional[float]`): Reward signal for this step
* `is_finished` (`Optional[bool]`): Whether this step ends the episode
* `metadata` (`Optional[dict]`): Arbitrary metadata for this step
### Flushing Events
When you are done logging, call `or_client.rollout.close()` to ensure all pending events are flushed and uploaded:
```python theme={null}
or_client.rollout.close()
```
## Recording Rollouts by Provider
In this example we will sample the `GeneralReasoning/CTF` environment and record the full rollout trace to OpenReward.
Make sure you have API keys for [OpenReward](https://openreward.ai/keys) and [OpenAI](https://platform.openai.com/api-keys), and set these as environment variables:
```bash theme={null}
export OPENAI_API_KEY='your-openai-api-key-here'
export OPENREWARD_API_KEY='your-openreward-api-key-here'
```
Save this as `record_rollout.py`:
```python theme={null}
from openai import OpenAI
from openreward import OpenReward
import json
or_client = OpenReward()
oai_client = OpenAI()
MODEL_NAME = "gpt-5.4"
environment = or_client.environments.get(name="GeneralReasoning/CTF")
tasks = environment.list_tasks(split="train")
tools = environment.list_tools(format="openai")
example_task = tasks[0]
# Create a rollout to record this episode
rollout = or_client.rollout.create(
run_name="openai-ctf-rollouts",
rollout_name="episode-0",
environment="GeneralReasoning/CTF",
split="train",
metadata={"model": MODEL_NAME},
task_spec=example_task.task_spec
)
with environment.session(task=example_task) as session:
prompt = session.get_prompt()
input_list = [{"role": "user", "content": prompt[0].text}]
finished = False
# Log the initial user prompt
rollout.log_openai_response(input_list[0])
while not finished:
response = oai_client.responses.create(
model=MODEL_NAME,
tools=tools,
input=input_list
)
# Log the full model response (handles reasoning, text, tool calls)
rollout.log_openai_response(response)
input_list += response.output
for item in response.output:
if item.type == "function_call":
tool_result = session.call_tool(item.name, json.loads(str(item.arguments)))
reward = tool_result.reward
finished = tool_result.finished
tool_output = {
"type": "function_call_output",
"call_id": item.call_id,
"output": json.dumps({
"result": tool_result.blocks[0].text
})
}
input_list.append(tool_output)
# Log the tool result with reward info
rollout.log_openai_response(
tool_output,
reward=reward,
is_finished=finished
)
if finished:
break
# Flush all pending events
or_client.rollout.close()
print("Rollout recorded successfully!")
```
```bash theme={null}
python record_rollout.py
```
Make sure you have API keys for [OpenReward](https://openreward.ai/keys) and [Anthropic](https://platform.claude.com/settings/keys), and set these as environment variables:
```bash theme={null}
export ANTHROPIC_API_KEY='your-anthropic-api-key-here'
export OPENREWARD_API_KEY='your-openreward-api-key-here'
```
Save this as `record_rollout.py`:
```python theme={null}
import anthropic
from openreward import OpenReward
import json
or_client = OpenReward()
ant_client = anthropic.Anthropic()
MODEL_NAME = "claude-sonnet-4-6"
environment = or_client.environments.get(name="GeneralReasoning/CTF")
tasks = environment.list_tasks(split="train")
tools = environment.list_tools(format="anthropic")
example_task = tasks[0]
# Create a rollout to record this episode
rollout = or_client.rollout.create(
run_name="anthropic-ctf-rollouts",
rollout_name="episode-0",
environment="GeneralReasoning/CTF",
split="train",
metadata={"model": MODEL_NAME},
task_spec=example_task.task_spec
)
with environment.session(task=example_task) as session:
prompt = session.get_prompt()
messages = [{"role": "user", "content": prompt[0].text}]
finished = False
# Log the initial user prompt
rollout.log_anthropic_message(messages[0])
while not finished:
response = ant_client.messages.create(
model=MODEL_NAME,
max_tokens=4096,
tools=tools,
messages=messages
)
assistant_msg = {"role": "assistant", "content": response.content}
messages.append(assistant_msg)
# Log the assistant response (handles text, thinking, tool_use blocks)
rollout.log_anthropic_message(assistant_msg)
if response.stop_reason == "tool_use":
tool_use = next(block for block in response.content if block.type == "tool_use")
tool_name = tool_use.name
tool_input = tool_use.input
tool_result = session.call_tool(tool_name, tool_input)
reward = tool_result.reward
finished = tool_result.finished
user_msg = {
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": tool_use.id,
"content": tool_result.blocks[0].text
}
]
}
messages.append(user_msg)
# Log the tool result with reward info
rollout.log_anthropic_message(
user_msg,
reward=reward,
is_finished=finished
)
if finished:
break
else:
break
# Flush all pending events
or_client.rollout.close()
print("Rollout recorded successfully!")
```
```bash theme={null}
python record_rollout.py
```
Make sure you have API keys for [OpenReward](https://openreward.ai/keys) and [Gemini](https://aistudio.google.com/app/apikey), and set these as environment variables:
```bash theme={null}
export GEMINI_API_KEY='your-gemini-api-key-here'
export OPENREWARD_API_KEY='your-openreward-api-key-here'
```
Save this as `record_rollout.py`:
```python theme={null}
from google import genai
from google.genai import types
from openreward import OpenReward
import json
or_client = OpenReward()
gem_client = genai.Client()
MODEL_NAME = "gemini-2.5-flash"
environment = or_client.environments.get(name="GeneralReasoning/CTF")
tasks = environment.list_tasks(split="train")
tools = environment.list_tools(format="google")
genai_tools = [types.Tool(function_declarations=tools)]
genai_config = types.GenerateContentConfig(tools=genai_tools)
example_task = tasks[0]
# Create a rollout to record this episode
rollout = or_client.rollout.create(
run_name="gemini-ctf-rollouts",
rollout_name="episode-0",
environment="GeneralReasoning/CTF",
split="train",
metadata={"model": MODEL_NAME},
task_spec=example_task.task_spec
)
with environment.session(task=example_task) as session:
prompt = session.get_prompt()
contents = [
types.Content(
role="user", parts=[types.Part(text=prompt[0].text)]
)
]
finished = False
# Log the initial user prompt
rollout.log_gdm_message(contents[0])
while not finished:
response = gem_client.models.generate_content(
model=MODEL_NAME,
config=genai_config,
contents=contents
)
model_content = response.candidates[0].content
contents.append(model_content)
# Log the model response (handles text, thinking, function calls)
rollout.log_gdm_message(model_content)
for part in model_content.parts:
if part.function_call:
tool_call = part.function_call
tool_result = session.call_tool(tool_call.name, tool_call.args)
reward = tool_result.reward
finished = tool_result.finished
function_response_part = types.Part.from_function_response(
name=tool_call.name,
response={"result": json.dumps({
"result": tool_result.blocks[0].text
})},
)
user_content = types.Content(role="user", parts=[function_response_part])
contents.append(user_content)
# Log the tool result with reward info
rollout.log_gdm_message(
user_content,
reward=reward,
is_finished=finished
)
if finished:
break
# Flush all pending events
or_client.rollout.close()
print("Rollout recorded successfully!")
```
```bash theme={null}
python record_rollout.py
```
Make sure you have API keys for [OpenReward](https://openreward.ai/keys) and [OpenRouter](https://openrouter.ai/keys), and set these as environment variables:
```bash theme={null}
export OPENROUTER_API_KEY='your-openrouter-api-key-here'
export OPENREWARD_API_KEY='your-openreward-api-key-here'
```
Save this as `record_rollout.py`.
Since OpenRouter uses the OpenAI SDK (Chat Completions format), we use `log_openai_completions` for logging:
```python theme={null}
from openai import OpenAI
from openreward import OpenReward
import json
import os
or_client = OpenReward()
oai_client = OpenAI(
base_url="https://openrouter.ai/api/v1",
api_key=os.environ.get("OPENROUTER_API_KEY")
)
MODEL_NAME = "deepseek/deepseek-v3.2"
environment = or_client.environments.get(name="GeneralReasoning/CTF")
tasks = environment.list_tasks(split="train")
tools = environment.list_tools(format="openrouter")
example_task = tasks[0]
# Create a rollout to record this episode
rollout = or_client.rollout.create(
run_name="openrouter-ctf-rollouts",
rollout_name="episode-0",
environment="GeneralReasoning/CTF",
split="train",
metadata={"model": MODEL_NAME},
task_spec=example_task.task_spec
)
with environment.session(task=example_task) as session:
prompt = session.get_prompt()
input_list = [{"role": "user", "content": prompt[0].text}]
finished = False
# Log the initial user prompt
rollout.log_openai_completions(input_list[0])
while not finished:
response = oai_client.chat.completions.create(
model=MODEL_NAME,
tools=tools,
messages=input_list
)
assistant_msg = response.choices[0].message
input_list.append(assistant_msg)
# Log the assistant response (handles text + tool calls)
rollout.log_openai_completions({
"role": "assistant",
"content": assistant_msg.content,
"tool_calls": assistant_msg.tool_calls
})
tool_calls = assistant_msg.tool_calls
if not tool_calls:
break
for tool_call in tool_calls:
tool_name = tool_call.function.name
tool_args = json.loads(tool_call.function.arguments)
tool_result = session.call_tool(tool_name, tool_args)
reward = tool_result.reward
finished = tool_result.finished
tool_msg = {
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps({
"result": tool_result.blocks[0].text
})
}
input_list.append(tool_msg)
# Log the tool result with reward info
rollout.log_openai_completions(
tool_msg,
reward=reward,
is_finished=finished
)
if finished:
break
# Flush all pending events
or_client.rollout.close()
print("Rollout recorded successfully!")
```
```bash theme={null}
python record_rollout.py
```
If you are using a different provider, a custom model, or want full control over what gets logged, you can use the generic `rollout.log()` method with the built-in message types.
### Generic Message Types
The SDK exports the following types from the `openreward` package:
| Type | Fields | Description |
| ------------------ | -------------------------------------------------------------------------------------- | ----------------------------------- |
| `UserMessage` | `content: str` | A user/human message |
| `AssistantMessage` | `content: str` | An assistant/model response |
| `SystemMessage` | `content: str` | A system prompt |
| `ReasoningItem` | `content: Optional[str]`, `content_reference: Optional[str]`, `summary: Optional[str]` | Hidden reasoning / chain-of-thought |
| `ToolCall` | `name: str`, `content: str`, `call_id: str` | A tool/function invocation |
| `ToolResult` | `content: str`, `call_id: str` | The result of a tool call |
```bash theme={null}
export OPENREWARD_API_KEY='your-openreward-api-key-here'
```
Set any additional API keys required by your model provider.
Save this as `record_rollout.py`:
```python theme={null}
from openreward import (
OpenReward,
UserMessage,
AssistantMessage,
SystemMessage,
ReasoningItem,
ToolCall,
ToolResult,
)
import json
or_client = OpenReward()
environment = or_client.environments.get(name="GeneralReasoning/CTF")
tasks = environment.list_tasks(split="train")
example_task = tasks[0]
# Create a rollout to record this episode
rollout = or_client.rollout.create(
run_name="custom-ctf-rollouts",
rollout_name="episode-0",
environment="GeneralReasoning/CTF",
split="train",
metadata={"model": "my-custom-model"},
task_spec=example_task.task_spec
)
with environment.session(task=example_task) as session:
prompt = session.get_prompt()
finished = False
# Log the user prompt
rollout.log(UserMessage(content=prompt[0].text))
while not finished:
# --- Replace this section with your model's inference ---
model_response = "The answer is 42."
# --------------------------------------------------------
# Log the assistant response
rollout.log(AssistantMessage(content=model_response))
# If your model produces reasoning/thinking, log it:
# rollout.log(ReasoningItem(content="Let me think about this..."))
# If your model makes a tool call, log the call and result:
tool_name = "submit_answer"
tool_args = {"answer": model_response}
call_id = "call_001"
rollout.log(ToolCall(
name=tool_name,
content=json.dumps(tool_args),
call_id=call_id
))
tool_result = session.call_tool(tool_name, tool_args)
rollout.log(
ToolResult(
content=tool_result.blocks[0].text,
call_id=call_id
),
reward=tool_result.reward,
is_finished=tool_result.finished
)
finished = tool_result.finished
# Flush all pending events
or_client.rollout.close()
print("Rollout recorded successfully!")
```
```bash theme={null}
python record_rollout.py
```
## Viewing Rollouts
After running your code, head to [OpenReward](https://openreward.ai) and your profile page. Go to the **Runs** tab to see your recorded runs:
Click on a run to see its rollouts:
Click on a specific rollout to inspect the full message timeline, including reasoning steps, tool calls, and rewards:
## Next Steps
Use recorded rollouts as part of a reinforcement learning training pipeline.
Run systematic evaluations across environments and models.
Create custom environments for your use case.
Record rollouts asynchronously for better performance.