Skip to main content

Goals

  • Learn the different ways to access tasks from an environment
  • Understand when to use listing vs index-based access
  • Learn how to implement task methods in your environment server

Prerequisites

Introduction

Tasks are JSON objects that describe a specific problem or challenge for an agent to solve. Each environment organises tasks into splits (e.g. "train", "test"). The SDK provides several ways to access tasks depending on your needs — from listing all tasks in a split, to fetching individual tasks by index, to retrieving a range of tasks for partitioned workloads.

Listing Tasks

You can list all tasks for a split:
This returns the full list of task objects for the given split. It’s the simplest approach and works well when your task set is small enough to load in one call.

Index-Based Access

For environments with large task sets, you can query the number of tasks and fetch individual tasks by index without loading the entire list:

Getting a Range of Tasks

You can also fetch a contiguous range of tasks by index. This is useful for partitioning tasks across multiple workers:
Task ordering is consistent across calls, so ranges are safe to use for partitioning workloads.

Implementing Tasks in Your Environment

When building an environment server, implement list_tasks, list_splits, and optionally override get_task and num_tasks:
The base class provides default implementations of num_tasks and get_task that call list_tasks internally. If your task set is large or expensive to load, you can override these for better performance:

Next Steps

Using Task-Specific Tools

Define tools that vary per task for environments with multiple tool interfaces