Environments

An environment on OpenReward is a long-running server that implements the Open Reward Standard (ORS) for agent environments. When an environment is created on OpenReward, a workspace is provisioned and an API endpoint provided. Environments provide:

Tasks - tasks are the core problems to be solved, including the initial prompts
Tools - tools are the actions an agent can take in the environment
Splits - splits organise tasks into groups, e.g. for training and evaluation
Statefulness - agent actions in a session can affect state
Tool Results - including tool feedback, rewards and termination signals

Agents connect to environments via sessions, which are stateful connections. ORS environments can be run anywhere. OpenReward hosts them and provides managed infrastructure, for example autoscaling based on demand.

Environment Lifecycle

1. Creation

Create environments through the OpenReward website:

1. In header of the website, click the + button
   ↓
2. Click "New environment"
   ↓
3. Enter name and description
   ↓
4. Set visibility (public/private)
   ↓
5. Environment created with workspace

2. Deployment

Once you have created your environment, you can deploy your ORS server code to it. First, you will need to develop your environment locally. You can read the local development guide for more details on how to develop your environment server locally. Once you are ready to deploy, you should undertake the following steps:

1. Upload your code to a GitHub repository, along with a Dockerimage
   ↓
2. Connect the GitHub repository to your environment
   ↓
3. OpenReward automatically builds and deploys whenever you push to main
   ↓
4. Environment server starts accepting sessions

See the GitHub integration guide for more details.

3. Running

Once deployed, agents can connect to your environment: Agent interaction:

from openreward import AsyncOpenReward

client = AsyncOpenReward(api_key="your-api-key")

# Get environment
environment = client.environments.get(name="username/my-environment")

# List available tasks
tasks = await environment.list_tasks(split="train")

# Or fetch a single task by index
task = await environment.get_task(split="train", index=0)

# Or get a range of tasks by index (supports negative indices and None)
# Task ordering is consistent across calls, so ranges are safe to partition
tasks = await environment.get_task_range(split="train", start=0, stop=10)

# Create session
async with environment.session(task=tasks[0]) as session:
    # Get task prompt
    prompt = await session.get_prompt()

    # Call tools
    result = await session.call_tool("bash", {"command": "ls"})

    # Environment returns results directly
    print(result)

4. Automatic Scaling

Environments automatically scale based on concurrent sessions. Autoscaling settings are configurable when making the environment and can be changed in the environment settings page. Benefits:

Pay only for active usage
No manual scaling needed
Always ready when agents connect

5. Updates

Pushing to the connected GitHub repository will automatically build and deploy a new version of the environment. Via GitHub (automatic):

1. Push new code to GitHub
   ↓
2. OpenReward detects changes
   ↓
3. Builds new version
   ↓
4. Deploys the new version of the environment server

Sessions

A session is a durable, stateful connection between an agent and your environment:

Unique: Each session has a unique ID, this is internally used to make sure you’re connecting to the correct environment server.
Stateful: Environment can maintain state between tool calls
Time-limited: Sessions expire after inactivity or when the session is deleted.
Sticky: Session stays connected to the same server instance

Using Sessions

When creating a session, you can pass a task object directly or reference a task by its split and index:

# Using a task object
async with environment.session(task=tasks[0]) as session:
    ...

# Using split and index
async with environment.session(split="train", index=0) as session:
    ...

Using split and index avoids fetching the full task list. See the ways to access tasks guide for more detail. Python SDK:

# Create session with context manager (automatic cleanup)
with environment.session(task=task) as session:
    # Get initial prompt
    prompt = session.get_prompt()

    # Call tools multiple times
    result1 = session.call_tool("read_file", {"path": "README.md"})
    result2 = session.call_tool("bash", {"command": "python test.py"})

    # Session automatically cleaned up on exit

Session lifecycle:

Agent creates session → Connected to environment
Agent calls tools → Results returned directly
Context manager exits → Session automatically cleaned up

Sessions are episodes in the reinforcement learning sense - they persist across multiple tool calls until the caller breaks out of the session. In ORS, a tool result with finished=True signals that the episode is complete and the agent should end the session. It is the caller’s responsibility to check the finished flag and break out of the loop accordingly.

Tool Calling

In ORS, the only way agents interact with environments is by calling tools. This leverages existing function calling support from LLM providers, creating a natural action space for language model agents. It also enforces a clean boundary between agent and environment - environments make no assumptions about the agent interacting with them, which keeps them robust to changes in how agents are built. Example tool call:

result = session.call_tool(
    tool_name="bash",
    input={"command": "ls -la"}
)

# Result returned directly from environment
print(result.blocks)       # Tool output (list of TextBlock/ImageBlock)
print(result.finished)     # Task completion status
print(result.reward)       # Performance feedback

Tool results structure:

blocks: The tool’s return value (list of TextBlock/ImageBlock)
metadata: Optional metadata dictionary
finished: Whether the task is complete
reward: Numeric feedback signal

Storage

Cloud Storage Integration

Each environment includes cloud storage accessible at /orwd_data/: Usage in environment server:

from pathlib import Path

# Storage mounted at /orwd_data/
storage = Path("/orwd_data")

# Read dataset
with open(storage / "datasets" / "train.csv") as f:
    data = f.read()

# Write results
with open(storage / "results" / "metrics.json", "w") as f:
    json.dump(metrics, f)

Data in /orwd_data/ can also be made accessible to sandboxes, depending on how you configure the mount. See where environment data lives for a conceptual overview, and Storage & Buckets for lower-level configuration.

Next Steps

Your First Environment

Build and deploy your first environment

Sandboxes

Add isolated code execution to your environment

GitHub Deployment

Set up automatic deployments from GitHub

Storage & Buckets

Configure persistent storage for your environment

Get started

Core Concepts

Standalone APIs

Making Environments

Advanced Environment Topics

Training

Evaluation

Harnesses

Sandbox Providers

Integrations

Rollouts

Deployment

Storage & Data

Environment Lifecycle

1. Creation

2. Deployment

3. Running

4. Automatic Scaling

5. Updates

Sessions

Using Sessions

Tool Calling

Storage

Cloud Storage Integration

Next Steps

Your First Environment

Sandboxes

GitHub Deployment

Storage & Buckets

​Environment Lifecycle

​1. Creation

​2. Deployment

​3. Running

​4. Automatic Scaling

​5. Updates

​Sessions

​Using Sessions

​Tool Calling

​Storage

​Cloud Storage Integration

​Next Steps

Your First Environment

Sandboxes

GitHub Deployment

Storage & Buckets

Environment Lifecycle

1. Creation

2. Deployment

3. Running

4. Automatic Scaling

5. Updates

Sessions

Using Sessions

Tool Calling

Storage

Cloud Storage Integration

Next Steps