Using Toolsets

Goals

Understand what toolsets are and why they’re useful
Use pre-built toolsets for common tasks (PDF, Excel, Word, PowerPoint)
Create custom toolsets for your specific needs
Compose multiple toolsets in a single environment

Prerequisites

An OpenReward account
An OpenReward API key
Completion of the Your First Environment tutorial
Completion of the Building Agentic Environments tutorial

Installation

Pre-built toolsets run in sandbox environments, which means your Docker image must include the required dependencies. You need the openreward package with the tools extra dependency:

pip install openreward[tools]

This installs the required libraries for all pre-built toolsets:

pdfplumber, pypdf, reportlab, pdf2image - for PDFToolset
python-docx - for WordToolset
openpyxl - for ExcelToolset
python-pptx - for PowerPointToolset

If you only need specific toolsets, you can install individual dependencies instead:

# For PDFToolset only
pip install pdfplumber pypdf reportlab pdf2image

# For WordToolset only
pip install python-docx

# For ExcelToolset only
pip install openpyxl

# For PowerPointToolset only
pip install python-pptx

Note: Make sure your sandbox Docker image includes these dependencies, or use the recommended generalreasoning/knowledge-worker image which has them pre-installed.

Introduction

When building multiple environments, you’ll often find yourself copying the same tool definitions across different environment classes. For example, if you have three environments that all need to read PDF files, you’d typically copy the same PDF reading tools into each environment. Toolsets solve this problem by letting you define reusable collections of tools that can be declared with a simple toolsets = [PDFToolset, ExcelToolset] statement. Instead of inheritance-based code reuse, toolsets provide a composition-based approach that follows the DRY (Don’t Repeat Yourself) principle. The key benefits:

Reusability: Define tools once, use them in multiple environments
Composition: Mix and match toolsets as needed
Maintainability: Update tools in one place
Clean separation: Keep environment-specific logic separate from general-purpose tools

Using Pre-Built Toolsets

OpenReward provides production-ready toolsets for common tasks. We’ll start with these before showing you how to create custom toolsets.

Available Toolsets

The openreward library includes four pre-built toolsets:

PDFToolset - PDF creation, reading, searching, merging (11 tools)
ExcelToolset - Spreadsheet manipulation, charts, data reading (11 tools)
WordToolset - Document creation, formatting, content manipulation (12 tools)
PowerPointToolset - Presentation creation and editing (11 tools)

All of these toolsets require sandbox access, as they manipulate files within the sandbox environment. For web access there are two toolsets, both exposing web_search / web_fetch and neither needing a sandbox, so they compose with any of the toolsets above:

Web Tools — a WebToolset whose search provider is swappable via OPENREWARD_SEARCH_BACKEND (backdated corpus by default, Tavily and others available).
Backdated Web Tools — a BackSearchToolset pinned to the backdated corpus, for when the cutoff guarantee must not be configurable.

Declare one or the other, not both — they use the same tool names. Important: All pre-built toolsets require sandbox access and the openreward[tools] dependencies. When creating sandboxes for toolset usage, we recommend using the generalreasoning/knowledge-worker image which includes all required dependencies pre-installed. Alternatively, ensure your custom image includes the necessary libraries listed in the Installation section above.

Example: Using PDFToolset

Let’s create an environment that uses PDFToolset to analyze PDF documents. Here’s a complete working example:

from openreward import AsyncOpenReward, SandboxSettings, SandboxBucketConfig
from openreward.environments import Environment, Split, tool, ToolOutput, TextBlock
from openreward.toolsets import PDFToolset
from pydantic import BaseModel

class AnswerParams(BaseModel):
    answer: str

class PDFAnalysisEnv(Environment):
    toolsets = [PDFToolset]  # Declare PDFToolset at class level

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)

        # Setup sandbox - PDFToolset will automatically access this
        self.sandbox_settings = SandboxSettings(
            environment="YourUsername/PDFAnalysisEnv",
            image="generalreasoning/knowledge-worker:latest",
            machine_size="0.5:1",
            block_network=False,
            bucket_config=SandboxBucketConfig(
                mount_path="/tmp/sandbox/",
                read_only=True,
            )
        )

        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(self.sandbox_settings)

    async def setup(self):
        await self.sandbox.start()

    async def teardown(self):
        await self.sandbox.stop()

    def get_prompt(self):
        return [TextBlock(text="Analyze the PDF document and extract key information.")]

    @tool
    async def submit_answer(self, params: AnswerParams) -> ToolOutput:
        """Submit your analysis"""
        return ToolOutput(
            blocks=[TextBlock(text=f"Analysis submitted: {params.answer}")],
            reward=1.0,
            finished=True,
        )

    @classmethod
    def list_tasks(cls, split: str):
        return [{"id": "1", "doc": "report.pdf"}] if split == "train" else []

    @classmethod
    def list_splits(cls):
        return [Split(name="train", type="train"), Split(name="test", type="test")]

With this setup, your environment automatically gains access to all 11 PDF tools. Here are the key tools available:

Tool Name	Description
`pdfs_read_pdf_pages`	Extract text content from PDF pages with optional layout information
`pdfs_search_pdf`	Search for text within PDF pages with context
`pdfs_get_metadata`	Get PDF document metadata and properties
`pdfs_get_document_overview`	Quick overview of PDF structure (page count, text preview, metadata)
`pdfs_merge_pdfs`	Combine multiple PDF files into one
`pdfs_extract_pages`	Extract specific pages from a PDF to a new file
`pdfs_create_pdf`	Create a new PDF file with optional metadata
`pdfs_add_content`	Add text content to an existing PDF page or create a new page
`pdfs_read_image`	Extract embedded images metadata from PDF pages
`pdfs_read_page_as_image`	Convert PDF page to image (PNG/JPEG)
`pdfs_delete_pdf`	Delete a PDF file from the sandbox

An agent interacting with this environment might call pdfs_get_document_overview first to understand the document structure, then pdfs_read_pdf_pages to extract specific content, and finally pdfs_search_pdf to find key terms before submitting an answer.

Example: Using ExcelToolset

Similarly, you can use ExcelToolset for spreadsheet manipulation:

from openreward import AsyncOpenReward, SandboxSettings, SandboxBucketConfig
from openreward.environments import Environment, TextBlock
from openreward.toolsets import ExcelToolset

class SpreadsheetEnv(Environment):
    toolsets = [ExcelToolset]

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)

        # Setup sandbox for ExcelToolset
        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(SandboxSettings(
            environment="YourUsername/SpreadsheetEnv",
            image="generalreasoning/knowledge-worker:latest",
            machine_size="0.5:1",
        ))

    async def setup(self):
        await self.sandbox.start()

    async def teardown(self):
        await self.sandbox.stop()

    def get_prompt(self):
        return [TextBlock(text="Analyze the spreadsheet and calculate the total.")]

ExcelToolset provides these key tools:

Tool Name	Description
`excel_create_spreadsheet`	Create a new Excel workbook with initial worksheet
`excel_list_tabs_in_spreadsheet`	List all worksheet/tab names in the workbook
`excel_add_tab`	Add a new worksheet to existing workbook
`excel_read_tab`	Read data from a specific worksheet
`excel_read_csv`	Read CSV file and convert to Excel format data structure
`excel_edit_spreadsheet`	Modify existing cell value in a worksheet
`excel_add_content_text`	Write data to a range of cells (batch operation)
`excel_delete_content_cell`	Clear cell content (sets to None)
`excel_create_chart`	Create a chart in the worksheet (bar, column, line, pie, scatter, area)
`excel_delete_tab`	Remove a worksheet from workbook
`excel_delete_spreadsheet`	Delete an Excel spreadsheet file from sandbox

Example: Composing Multiple Toolsets

You can use multiple toolsets together in a single environment. All tools from all toolsets become available:

from openreward import AsyncOpenReward, SandboxSettings
from openreward.environments import Environment
from openreward.toolsets import PDFToolset, ExcelToolset, WordToolset

class DocumentProcessorEnv(Environment):
    toolsets = [PDFToolset, ExcelToolset, WordToolset]

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)
        # Setup sandbox (shared by all toolsets)
        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(SandboxSettings(...))

    # ... rest of environment implementation ...

Key points about composition:

All tools from all toolsets are discovered automatically
Tools can be called without knowing which toolset they came from
The framework detects and prevents tool name collisions

WordToolset and PowerPointToolset

WordToolset provides 12 tools for Word document manipulation:

word_create_document - Create new document
word_get_document_overview - Get structure overview
word_read_document_content - Extract text
word_add_content_text - Add paragraphs/headings
word_edit_content_text - Modify paragraphs
word_delete_content_text - Remove paragraphs
word_add_image - Insert images
word_apply_formatting - Bold, italic, fonts, colors
And more…

PowerPointToolset provides tools for presentation manipulation, following similar patterns to the other document toolsets. All pre-built toolsets use the same pattern: import from openreward.toolsets, declare in your environment’s toolsets list, and ensure your environment has a self.sandbox attribute.

Creating Custom Toolsets

While pre-built toolsets cover common scenarios, you can create custom toolsets for your specific needs.

Simple Toolset (No Dependencies)

For tools that don’t need sandbox access or other dependencies, you can create a simple toolset without extending any base class:

from pydantic import BaseModel, Field
from openreward.environments import tool, ToolOutput, TextBlock

class MathParams(BaseModel):
    a: float = Field(..., description="First number")
    b: float = Field(..., description="Second number")

class MathToolset:
    """Simple toolset with no dependencies"""

    @tool
    async def add(self, params: MathParams) -> ToolOutput:
        """Add two numbers together"""
        result = params.a + params.b
        return ToolOutput(
            blocks=[TextBlock(text=f"{params.a} + {params.b} = {result}")],
            reward=0.0,
            finished=False,
        )

    @tool
    async def subtract(self, params: MathParams) -> ToolOutput:
        """Subtract second number from first"""
        result = params.a - params.b
        return ToolOutput(
            blocks=[TextBlock(text=f"{params.a} - {params.b} = {result}")],
            reward=0.0,
            finished=False,
        )

Using it in an environment:

from openreward.environments import Environment

class CalculatorEnv(Environment):
    toolsets = [MathToolset]

    # ... implement required methods ...

Sandbox-Based Toolset

For tools that need sandbox access, extend the Toolset base class:

from openreward.environments import Toolset, tool, ToolOutput, TextBlock
from pydantic import BaseModel, Field

class BashParams(BaseModel):
    command: str = Field(..., description="Bash command to execute")

class FileToolset(Toolset):
    """Toolset that requires sandbox access"""

    @tool
    async def list_files(self, params: BashParams) -> ToolOutput:
        """List files in a directory using bash"""
        # self.sandbox is automatically available from Toolset base class
        output, exit_code = await self.sandbox.run(params.command)

        return ToolOutput(
            blocks=[TextBlock(text=f"Output:\n{output}\nExit code: {exit_code}")],
            reward=0.0,
            finished=False,
        )

    @tool
    async def read_file(self, params: BashParams) -> ToolOutput:
        """Read file content"""
        output, exit_code = await self.sandbox.run(f"cat {params.command}")

        return ToolOutput(
            blocks=[TextBlock(text=output if exit_code == 0 else f"Error: {output}")],
            reward=0.0,
            finished=False,
        )

The Toolset base class:

Receives the environment instance in its constructor
Extracts self.sandbox automatically (via sandbox_attr parameter)
Makes the sandbox available to all tool methods
Supports custom sandbox attribute names if needed

Key Concepts

Collision Detection

The framework prevents duplicate tool names. If your environment defines a tool with the same name as a toolset tool, an error is raised. This ensures clarity about which tool is being called and prevents subtle bugs. Example of a collision:

class MyEnv(Environment):
    toolsets = [PDFToolset]

    @tool
    async def pdfs_read_pdf_pages(self, params):  # Error! Name collision
        ...

This would raise:

ValueError: Tool name collision: 'pdfs_read_pdf_pages' is defined in both the environment and toolset 'PDFToolset'. Please rename one of them to avoid conflicts.

Dependency Injection

When a toolset extends the Toolset base class, it automatically receives the environment instance and extracts dependencies like self.sandbox. You don’t need to manually pass these dependencies - the framework handles this through lazy instantiation.

Best Practices

When to use pre-built toolsets:

You’re working with common file formats (PDF, Excel, Word, PowerPoint)
You need standard operations (read, write, search, merge)
You want battle-tested, production-ready tools

When to create custom toolsets:

You have domain-specific tools used across multiple environments
You’re building tools for file formats not covered by pre-built toolsets
You need specialized operations not available in pre-built toolsets

Naming conventions:

Prefix toolset tools to avoid collisions: pdf_, excel_, math_
Use descriptive toolset class names: PDFToolset, not DocumentToolset
Match parameter model names to tools: MathParams for both add and subtract when they share parameters

When to use toolsets vs environment tools:

Toolsets: Reusable tools used across 3+ environments, logical groups with shared dependencies
Environment tools (@tool): Environment-specific logic, environment-specific state, one-off operations
For tools specific to individual tasks, see Using Task-Specific Tools

Organizing toolset files:

Keep toolsets in a separate toolsets/ directory
One file per toolset: toolsets/math.py, toolsets/file.py
Import and declare: from toolsets.math import MathToolset

Troubleshooting

“AttributeError: ‘MyEnv’ object has no attribute ‘sandbox’” Solution: Ensure you initialize self.sandbox in your environment’s __init__ method before any tools are called. Toolsets extending Toolset look for this attribute.

def __init__(self, task_spec, secrets):
    super().__init__(task_spec, secrets)
    or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
    self.sandbox = or_client.sandbox(SandboxSettings(...))  # Must be set

“Tool name collision detected” Solution: Rename your environment tool or toolset tool to avoid conflicts. Use prefixes to make names unique.

# Bad - collision
@tool
async def read(self, params):  # Conflicts with toolset

# Good - unique name
@tool
async def read_task_config(self, params):  # No collision

“ImportError: cannot import name ‘PDFToolset’” Solution: Update your openreward library to the latest version:

pip install --upgrade openreward

Pre-built toolsets were added in a recent version. Check that you’re using the latest version. Toolset tools not appearing in list_tools() Solution: Ensure toolsets are declared at the class level, not instance level:

# Correct - class level
class MyEnv(Environment):
    toolsets = [MathToolset]

# Incorrect - instance level
class MyEnv(Environment):
    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)
        self.toolsets = [MathToolset]  # Won't work!

Next Steps

Now that you understand toolsets, you can build more sophisticated environments:

Building Agentic Environments

Learn advanced sandbox patterns that work great with toolsets

Using LLM Graders

Combine toolsets with intelligent grading for complex evaluations

Evaluation

Use toolsets in evaluation environments to test model capabilities

Get started

Core Concepts

Making Environments

Advanced Environment Topics

Training

Evaluation

Harnesses

Sandbox Providers

Standalone APIs

Integrations

Rollouts

Deployment

Storage & Data

Goals

Prerequisites

Installation

Introduction

Using Pre-Built Toolsets

Available Toolsets

Example: Using PDFToolset

Example: Using ExcelToolset

Example: Composing Multiple Toolsets

WordToolset and PowerPointToolset

Creating Custom Toolsets

Simple Toolset (No Dependencies)

Sandbox-Based Toolset

Key Concepts

Collision Detection

Dependency Injection

Best Practices

Troubleshooting

Next Steps

Building Agentic Environments

Using LLM Graders

Evaluation

​Goals

​Prerequisites

​Installation

​Introduction

​Using Pre-Built Toolsets

​Available Toolsets

​Example: Using PDFToolset

​Example: Using ExcelToolset

​Example: Composing Multiple Toolsets

​WordToolset and PowerPointToolset

​Creating Custom Toolsets

​Simple Toolset (No Dependencies)

​Sandbox-Based Toolset

​Key Concepts

​Collision Detection

​Dependency Injection

​Best Practices

​Troubleshooting

​Next Steps

Building Agentic Environments

Using LLM Graders

Evaluation

Goals

Prerequisites

Installation

Introduction

Using Pre-Built Toolsets

Available Toolsets

Example: Using PDFToolset

Example: Using ExcelToolset

Example: Composing Multiple Toolsets

WordToolset and PowerPointToolset

Creating Custom Toolsets

Simple Toolset (No Dependencies)

Sandbox-Based Toolset

Key Concepts

Collision Detection

Dependency Injection

Best Practices

Troubleshooting

Next Steps