Skip to main content

​
Goals

  • Understand what toolsets are and why they’re useful
  • Use pre-built toolsets for common tasks (PDF, Excel, Word, PowerPoint)
  • Create custom toolsets for your specific needs
  • Compose multiple toolsets in a single environment

​
Prerequisites

​
Installation

Pre-built toolsets run in sandbox environments, which means your Docker image must include the required dependencies. You need the openreward package with the tools extra dependency: 
pip install openreward[tools]
This installs the required libraries for all pre-built toolsets:
  • pdfplumber, pypdf, reportlab, pdf2image - for PDFToolset
  • python-docx - for WordToolset
  • openpyxl - for ExcelToolset
  • python-pptx - for PowerPointToolset
If you only need specific toolsets, you can install individual dependencies instead: 
# For PDFToolset only
pip install pdfplumber pypdf reportlab pdf2image

# For WordToolset only
pip install python-docx

# For ExcelToolset only
pip install openpyxl

# For PowerPointToolset only
pip install python-pptx
Note: Make sure your sandbox Docker image includes these dependencies, or use the recommended generalreasoning/knowledge-worker image which has them pre-installed.

​
Introduction

When building multiple environments, you’ll often find yourself copying the same tool definitions across different environment classes. For example, if you have three environments that all need to read PDF files, you’d typically copy the same PDF reading tools into each environment. Toolsets solve this problem by letting you define reusable collections of tools that can be declared with a simple toolsets = [PDFToolset, ExcelToolset] statement. Instead of inheritance-based code reuse, toolsets provide a composition-based approach that follows the DRY (Don’t Repeat Yourself) principle. The key benefits:
  • Reusability: Define tools once, use them in multiple environments
  • Composition: Mix and match toolsets as needed
  • Maintainability: Update tools in one place
  • Clean separation: Keep environment-specific logic separate from general-purpose tools

​
Using Pre-Built Toolsets

OpenReward provides production-ready toolsets for common tasks. We’ll start with these before showing you how to create custom toolsets.

​
Available Toolsets

The openreward library includes four pre-built toolsets:
  • PDFToolset - PDF creation, reading, searching, merging (11 tools)
  • ExcelToolset - Spreadsheet manipulation, charts, data reading (10 tools)
  • WordToolset - Document creation, formatting, content manipulation (12 tools)
  • PowerPointToolset - Presentation creation and editing
All of these toolsets require sandbox access, as they manipulate files within the sandbox environment. Important: All pre-built toolsets require sandbox access and the openreward[tools] dependencies. When creating sandboxes for toolset usage, we recommend using the generalreasoning/knowledge-worker image which includes all required dependencies pre-installed. Alternatively, ensure your custom image includes the necessary libraries listed in the Installation section above.

​
Example: Using PDFToolset

Let’s create an environment that uses PDFToolset to analyze PDF documents. Here’s a complete working example: 
from openreward import AsyncOpenReward, SandboxSettings, SandboxBucketConfig
from openreward.environments import Environment, tool, ToolOutput, TextBlock
from openreward.toolsets import PDFToolset
from pydantic import BaseModel

class AnswerParams(BaseModel):
    answer: str

class PDFAnalysisEnv(Environment):
    toolsets = [PDFToolset]  # Declare PDFToolset at class level

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)

        # Setup sandbox - PDFToolset will automatically access this
        self.sandbox_settings = SandboxSettings(
            environment="YourUsername/PDFAnalysisEnv",
            image="generalreasoning/python-ds:3.12-tools",
            machine_size="0.5:1",
            block_network=False,
            bucket_config=SandboxBucketConfig(
                mount_path="/tmp/sandbox/",
                read_only=True,
            )
        )

        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(self.sandbox_settings)

    async def setup(self):
        await self.sandbox.start()

    async def teardown(self):
        await self.sandbox.stop()

    def get_prompt(self):
        return [TextBlock(text="Analyze the PDF document and extract key information.")]

    @tool
    async def submit_answer(self, params: AnswerParams) -> ToolOutput:
        """Submit your analysis"""
        return ToolOutput(
            blocks=[TextBlock(text=f"Analysis submitted: {params.answer}")],
            reward=1.0,
            finished=True,
        )

    @classmethod
    def list_tasks(cls, split: str):
        return [{"id": "1", "doc": "report.pdf"}] if split == "train" else []

    @classmethod
    def list_splits(cls):
        return ["train", "test"]
With this setup, your environment automatically gains access to all 11 PDF tools. Here are the key tools available:
Tool NameDescription
pdfs_read_pdf_pagesExtract text content from PDF pages with optional layout information
pdfs_search_pdfSearch for text within PDF pages with context
pdfs_get_metadataGet PDF document metadata and properties
pdfs_get_document_overviewQuick overview of PDF structure (page count, text preview, metadata)
pdfs_merge_pdfsCombine multiple PDF files into one
pdfs_extract_pagesExtract specific pages from a PDF to a new file
pdfs_create_pdfCreate a new PDF file with optional metadata
pdfs_add_contentAdd text content to an existing PDF page or create a new page
pdfs_read_imageExtract embedded images metadata from PDF pages
pdfs_read_page_as_imageConvert PDF page to image (PNG/JPEG)
pdfs_delete_pdfDelete a PDF file from the sandbox
An agent interacting with this environment might call pdfs_get_document_overview first to understand the document structure, then pdfs_read_pdf_pages to extract specific content, and finally pdfs_search_pdf to find key terms before submitting an answer.

​
Example: Using ExcelToolset

Similarly, you can use ExcelToolset for spreadsheet manipulation: 
from openreward import AsyncOpenReward, SandboxSettings, SandboxBucketConfig
from openreward.environments import Environment, TextBlock
from openreward.toolsets import ExcelToolset

class SpreadsheetEnv(Environment):
    toolsets = [ExcelToolset]

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)

        # Setup sandbox for ExcelToolset
        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(SandboxSettings(
            environment="YourUsername/SpreadsheetEnv",
            image="generalreasoning/python-ds:3.12-tools",
            machine_size="0.5:1",
        ))

    async def setup(self):
        await self.sandbox.start()

    async def teardown(self):
        await self.sandbox.stop()

    def get_prompt(self):
        return [TextBlock(text="Analyze the spreadsheet and calculate the total.")]
ExcelToolset provides these key tools:
Tool NameDescription
excel_create_spreadsheetCreate a new Excel workbook with initial worksheet
excel_list_tabs_in_spreadsheetList all worksheet/tab names in the workbook
excel_add_tabAdd a new worksheet to existing workbook
excel_read_tabRead data from a specific worksheet
excel_read_csvRead CSV file and convert to Excel format data structure
excel_edit_spreadsheetModify existing cell value in a worksheet
excel_add_content_textWrite data to a range of cells (batch operation)
excel_delete_content_cellClear cell content (sets to None)
excel_create_chartCreate a chart in the worksheet (bar, column, line, pie, scatter, area)
excel_delete_tabRemove a worksheet from workbook
excel_delete_spreadsheetDelete an Excel spreadsheet file from sandbox

​
Example: Composing Multiple Toolsets

You can use multiple toolsets together in a single environment. All tools from all toolsets become available: 
from openreward.environments import Environment
from openreward.toolsets import PDFToolset, ExcelToolset, WordToolset

class DocumentProcessorEnv(Environment):
    toolsets = [PDFToolset, ExcelToolset, WordToolset]

    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)
        # Setup sandbox (shared by all toolsets)
        or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
        self.sandbox = or_client.sandbox(SandboxSettings(...))

    # ... rest of environment implementation ...
Key points about composition:
  • All tools from all toolsets are discovered automatically
  • Tools can be called without knowing which toolset they came from
  • The framework detects and prevents tool name collisions

​
WordToolset and PowerPointToolset

WordToolset provides 12 tools for Word document manipulation:
  • word_create_document - Create new document
  • word_get_document_overview - Get structure overview
  • word_read_document_content - Extract text
  • word_add_content_text - Add paragraphs/headings
  • word_edit_content_text - Modify paragraphs
  • word_delete_content_text - Remove paragraphs
  • word_add_image - Insert images
  • word_apply_formatting - Bold, italic, fonts, colors
  • And more…
PowerPointToolset provides tools for presentation manipulation, following similar patterns to the other document toolsets. All pre-built toolsets use the same pattern: import from openreward.toolsets, declare in your environment’s toolsets list, and ensure your environment has a self.sandbox attribute.

​
Creating Custom Toolsets

While pre-built toolsets cover common scenarios, you can create custom toolsets for your specific needs.

​
Simple Toolset (No Dependencies)

For tools that don’t need sandbox access or other dependencies, you can create a simple toolset without extending any base class: 
from pydantic import BaseModel, Field
from openreward.environments import tool, ToolOutput, TextBlock

class MathParams(BaseModel):
    a: float = Field(..., description="First number")
    b: float = Field(..., description="Second number")

class MathToolset:
    """Simple toolset with no dependencies"""

    @tool
    async def add(self, params: MathParams) -> ToolOutput:
        """Add two numbers together"""
        result = params.a + params.b
        return ToolOutput(
            blocks=[TextBlock(text=f"{params.a} + {params.b} = {result}")],
            reward=0.0,
            finished=False,
        )

    @tool
    async def subtract(self, params: MathParams) -> ToolOutput:
        """Subtract second number from first"""
        result = params.a - params.b
        return ToolOutput(
            blocks=[TextBlock(text=f"{params.a} - {params.b} = {result}")],
            reward=0.0,
            finished=False,
        )
Using it in an environment: 
from openreward.environments import Environment

class CalculatorEnv(Environment):
    toolsets = [MathToolset]

    # ... implement required methods ...

​
Sandbox-Based Toolset

For tools that need sandbox access, extend the Toolset base class: 
from openreward.environments import Toolset, tool, ToolOutput, TextBlock
from pydantic import BaseModel, Field

class BashParams(BaseModel):
    command: str = Field(..., description="Bash command to execute")

class FileToolset(Toolset):
    """Toolset that requires sandbox access"""

    @tool
    async def list_files(self, params: BashParams) -> ToolOutput:
        """List files in a directory using bash"""
        # self.sandbox is automatically available from Toolset base class
        output, exit_code = await self.sandbox.run(params.command)

        return ToolOutput(
            blocks=[TextBlock(text=f"Output:\n{output}\nExit code: {exit_code}")],
            reward=0.0,
            finished=False,
        )

    @tool
    async def read_file(self, params: BashParams) -> ToolOutput:
        """Read file content"""
        output, exit_code = await self.sandbox.run(f"cat {params.command}")

        return ToolOutput(
            blocks=[TextBlock(text=output if exit_code == 0 else f"Error: {output}")],
            reward=0.0,
            finished=False,
        )
The Toolset base class:
  • Receives the environment instance in its constructor
  • Extracts self.sandbox automatically (via sandbox_attr parameter)
  • Makes the sandbox available to all tool methods
  • Supports custom sandbox attribute names if needed

​
Key Concepts

​
Collision Detection

The framework prevents duplicate tool names. If your environment defines a tool with the same name as a toolset tool, an error is raised. This ensures clarity about which tool is being called and prevents subtle bugs. Example of a collision: 
class MyEnv(Environment):
    toolsets = [PDFToolset]

    @tool
    async def pdfs_read_pdf_pages(self, params):  # Error! Name collision
        ...
This would raise: ValueError: Tool name collision: 'pdfs_read_pdf_pages' is defined in both the environment and toolset 'PDFToolset'. Please rename one of them to avoid conflicts.

​
Dependency Injection

When a toolset extends the Toolset base class, it automatically receives the environment instance and extracts dependencies like self.sandbox. You don’t need to manually pass these dependencies - the framework handles this through lazy instantiation.

​
Best Practices

When to use pre-built toolsets:
  • You’re working with common file formats (PDF, Excel, Word, PowerPoint)
  • You need standard operations (read, write, search, merge)
  • You want battle-tested, production-ready tools
When to create custom toolsets:
  • You have domain-specific tools used across multiple environments
  • You’re building tools for file formats not covered by pre-built toolsets
  • You need specialized operations not available in pre-built toolsets
Naming conventions:
  • Prefix toolset tools to avoid collisions: pdf_, excel_, math_
  • Use descriptive toolset class names: PDFToolset, not DocumentToolset
  • Match parameter model names to tools: MathParams for both add and subtract when they share parameters
When to use toolsets vs environment tools:
  • Toolsets: Reusable tools used across 3+ environments, logical groups with shared dependencies
  • Environment tools: Task-specific logic, environment-specific state, one-off operations
Organizing toolset files:
  • Keep toolsets in a separate toolsets/ directory
  • One file per toolset: toolsets/math.py, toolsets/file.py
  • Import and declare: from toolsets.math import MathToolset

​
Troubleshooting

“AttributeError: ‘MyEnv’ object has no attribute ‘sandbox’” Solution: Ensure you initialize self.sandbox in your environment’s __init__ method before any tools are called. Toolsets extending Toolset look for this attribute. 
def __init__(self, task_spec, secrets):
    super().__init__(task_spec, secrets)
    or_client = AsyncOpenReward(api_key=secrets.get("api_key"))
    self.sandbox = or_client.sandbox(SandboxSettings(...))  # Must be set
“Tool name collision detected” Solution: Rename your environment tool or toolset tool to avoid conflicts. Use prefixes to make names unique. 
# Bad - collision
@tool
async def read(self, params):  # Conflicts with toolset

# Good - unique name
@tool
async def read_task_config(self, params):  # No collision
“ImportError: cannot import name ‘PDFToolset’” Solution: Update your openreward library to the latest version: 
pip install --upgrade openreward
Pre-built toolsets were added in a recent version. Check that you’re using version 0.5.0 or later. Toolset tools not appearing in list_tools() Solution: Ensure toolsets are declared at the class level, not instance level: 
# Correct - class level
class MyEnv(Environment):
    toolsets = [MathToolset]

# Incorrect - instance level
class MyEnv(Environment):
    def __init__(self, task_spec, secrets):
        super().__init__(task_spec, secrets)
        self.toolsets = [MathToolset]  # Won't work!

​
Next Steps

Now that you understand toolsets, you can build more sophisticated environments:

Building Agentic Environments

Learn advanced sandbox patterns that work great with toolsets

Using LLM Graders

Combine toolsets with intelligent grading for complex evaluations

Evaluation

Use toolsets in evaluation environments to test model capabilities