> ## Documentation Index
> Fetch the complete documentation index at: https://docs.openreward.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Web Tools
> One web search and fetch surface for your environment, with a swappable backend
## Introduction
`WebToolset` gives an agent `web_search` and `web_fetch` tools **without tying
your environment to a particular search provider**. Which provider actually
answers is process configuration, not code:
```bash theme={null}
# default — GR's backdated corpus
(nothing to set)
# or route the same tools to Tavily
export OPENREWARD_SEARCH_BACKEND=tavily
export TAVILY_API_KEY="tvly-xxxxxxxx"
```
The environment stays identical either way. Input validation, the citation
envelope the model sees, the fetch cache and the error contract all live in the
SDK and are shared by every backend — so a task written against one backend
behaves the same against another. Only the sources differ.
Requires `openreward >= 0.1.148`. The Tavily backend also needs
`pip install 'openreward[search]'`; the default `backsearch` backend needs
nothing beyond the base install.
### Which toolset should I use?
| | `WebToolset` | [`BackSearchToolset`](/environments/backdated-web-tools) |
| ----------------------- | ---------------------------------------------------------------- | ------------------------------------------------------------- |
| Tools | `web_search`, `web_fetch` | `web_search`, `web_fetch` |
| Backend | Swappable via `OPENREWARD_SEARCH_BACKEND` | Always the backdated corpus |
| Point-in-time guarantee | Only on backdated backends | Always |
| Use when | You want to change search providers without editing environments | The `as_of` cutoff is load-bearing and must not be switchable |
`BackSearchToolset` deliberately ignores `OPENREWARD_SEARCH_BACKEND`. A toolset
whose whole value is the leakage-free guarantee should not be silently
redirected to the live web by an environment variable.
## Configuration
| Variable | Required | Purpose |
| --------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------- |
| `OPENREWARD_SEARCH_BACKEND` | no | `backsearch` (default) or `tavily`. An unrecognised value logs a warning and falls back to `backsearch` |
| `OPENREWARD_API_KEY` | for `backsearch` | Your OpenReward API key |
| `TAVILY_API_KEY` | for `tavily` | Your Tavily API key |
| `TAVILY_SEARCH_DEPTH` | no | `basic` (default) or `advanced` |
| `TAVILY_MAX_RESULTS` | no | Results per search (default `8`, capped at Tavily's limit of 20) |
| `TAVILY_EXTRACT_DEPTH` | no | `basic` (default) or `advanced` |
| `TAVILY_API_BASE_URL` | no | Override the Tavily host (egress proxy, self-hosted, or a test stub) |
| `OPENREWARD_WEB_AS_OF` | no | Default cutoff date (ISO `YYYY-MM-DD`) — honoured by backdated backends only |
A backend that is missing its key is **not** a startup failure — the session
still opens, and the problem surfaces on the first tool call. What happens then
depends on whether the agent could plausibly recover; see below.
## Fatal vs recoverable errors
`WebToolset` splits tool failures in two, and the distinction matters for
training data:
* **Recoverable** — a blocked domain, an unparseable URL, an empty result set.
These come back to the model as ordinary tool output with the code in
`metadata["error"]`, because the agent can act on them by searching
differently or picking another source. An empty result set is a real answer
to the query, not a failure.
* **Fatal** — `not-configured` (no API key) or `quota-exceeded`. These raise
`SearchBackendUnavailable`.
The reason for raising: handed back as text, the agent re-issues a dead call
until it hits the turn cap, never reaches the terminal tool, and the rollout
scores **0.0 — indistinguishable from a model that answered wrongly**. Raising
lets the platform retry the tool call and, if it still fails, end the rollout
with a blank reward, which is excluded from training rather than poisoning it.
Pass `raise_on_fatal=False` to get the everything-is-soft behaviour.
`BackSearchToolset` is unchanged and always soft-errors.
## Backends
| Backend | Sources | `as_of` cutoff | `allowed_domains` / `blocked_domains` | Domain policy preflight |
| ---------------------- | ------------------------------------------------ | -------------------------------------------- | ------------------------------------------------------------- | ----------------------- |
| `backsearch` (default) | GR's backdated corpus — news, SEC filings, arXiv | **Honoured** — results bounded to the cutoff | Yes | Yes |
| `tavily` | The live web | **Ignored** — see the warning below | Best-effort (mapped to `include_domains` / `exclude_domains`) | No |
**Tavily searches the live web, so `as_of` is ignored.** Tavily does offer
publish-date filters, but they filter a live index by *claimed* publish date
rather than serving a frozen crawl, so they cannot give the leakage-free
guarantee an evaluation or prediction task needs. Rather than imply a guarantee
we cannot keep, the cutoff is dropped and a warning is logged the first time one
is requested. If post-cutoff leakage would invalidate your task, stay on
`backsearch`.
**`allowed_domains` is not a hard guarantee on Tavily.** It maps to Tavily's
`include_domains`, which filters correctly for indexed domains — but when the
requested domain is absent from Tavily's index, Tavily silently returns
*unfiltered* results rather than an empty set. Asking for `reuters.com`, for
example, comes back with results from other sites entirely. If your task depends
on sources being restricted to a domain, verify the hostnames yourself, or use
`backsearch`, where the filter is applied by the index.
The tool descriptions the model sees follow the configured backend: on a
live-web backend the "results are limited to documents published on or before
the cutoff date" wording is removed, so the model is never told about a bound
that will not hold.
## Using the toolset in an environment
Declare `WebToolset` at class level. Like `BackSearchToolset` it is **not**
sandbox-backed — it talks to a search provider over HTTP — so your environment
does not need a `self.sandbox`.
```python theme={null}
from openreward.environments import Environment, Split, TextBlock
from openreward.toolsets import WebToolset
class ResearchEnv(Environment):
toolsets = [WebToolset]
@classmethod
def list_splits(cls):
return [Split(name="test", type="test")]
@classmethod
def list_tasks(cls, split):
return [{"id": "1", "question": "What happened with X in late 2025?"}]
def get_prompt(self):
return [TextBlock(text="Research the question using web_search and web_fetch, then answer with sources.")]
```
The agent then sees two tools:
| Tool | Input | Output |
| ------------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
| `web_search` | `query` (≥ 2 chars), optional `allowed_domains` **or** `blocked_domains` (never both) | A `Links:` list of `{title, url}` sources plus a reminder to cite them |
| `web_fetch` | `url`, `prompt` | Extracted page text (truncated to 100 KB) |
### Selecting it by name in a session
`WebToolset` is a registered built-in, so a client can attach it to a session by
name:
```python theme={null}
from openreward import OpenReward
or_client = OpenReward()
env = or_client.environments.get(name="MyOrg/ResearchEnv")
tasks = env.list_tasks(split="test")
with env.session(task=tasks[0], toolset="web") as session:
result = session.call_tool("web_search", {"query": "US interest rate decision December 2025"})
print(result.blocks[0].text)
```
### Passing credentials as session secrets
Environments conventionally receive API keys through the session `secrets`
mapping rather than the server's process environment. Expose them to the toolset
with a `search_secrets` hook and the configured backend picks out the key it
needs — `tavily_api_key` for Tavily, `api_key` for backsearch:
```python theme={null}
class ResearchEnv(Environment):
toolsets = [WebToolset]
def __init__(self, task_spec, secrets={}):
super().__init__(task_spec, secrets)
self.search_secrets = secrets
```
```python theme={null}
with env.session(
task=task,
secrets={"tavily_api_key": "tvly-xxxxxxxx"},
) as session:
...
```
Without the hook the backend falls back to the process environment. See
[Keeping Secrets Secret](/environments/keeping-secrets-secret) for how secret
values reach a hosted environment.
### Choosing the backend per environment
`OPENREWARD_SEARCH_BACKEND` is the usual control, but an environment can pin or
override it with a `search_backend` hook — read live on every tool call, like
`web_as_of`:
```python theme={null}
class ResearchEnv(Environment):
toolsets = [WebToolset]
search_backend = "tavily" # attribute, @property, or zero-arg callable
```
Resolution order for each call: an explicit `backend=` passed to the toolset
constructor, then `env.search_backend`, then `OPENREWARD_SEARCH_BACKEND`, then
`backsearch`.
### Composing with other toolsets
Because it needs no sandbox, `WebToolset` composes cleanly with sandbox-backed
toolsets:
```python theme={null}
from openreward.toolsets import WebToolset, CLIToolset
class ResearchAndCodeEnv(Environment):
toolsets = [WebToolset, CLIToolset]
```
Do **not** declare `WebToolset` and `BackSearchToolset` together — they expose
the same tool names and the framework rejects duplicates.
## Standalone usage
The same search and fetch are available without an environment:
```python theme={null}
import asyncio
from openreward.tools import Search, Fetch
async def main():
s = Search() # OPENREWARD_SEARCH_BACKEND picks the provider
result = await s.run("US interest rate decision December 2025")
if result.ok:
print(result.output) # citation-ready text
print(result.data["mode"]) # which backend answered
for hit in result.data["hits"]:
print(hit["title"], "->", hit["url"])
f = Fetch(backend="tavily") # or pin one explicitly
page = await f.run("https://example.com/article", "Summarise the key points")
if page.ok:
print(page.data["content"])
asyncio.run(main())
```
Both `run(...)` calls return a `WebToolResult`:
| Field | Meaning |
| -------------------------------- | ----------------------------------------------------------------------------------- |
| `.ok` | `True` on success, `False` on a recoverable error |
| `.output` | The string to show a model (the result, or an error message) |
| `.data` | Structured payload — search: `{query, hits, mode}`; fetch: `{url, prompt, content}` |
| `.error_code` / `.error_message` | Set when `.ok` is `False` |
Error codes are the same on every backend, so calling code can branch on them
without knowing which provider is configured:
| Code | Meaning |
| -------------------------------------------------------------------------------- | -------------------------------------------------------------------- |
| `not-configured` | No key for the selected backend, or `tavily-python` is not installed |
| `missing-query` | Query shorter than 2 characters |
| `domain-lists-conflict` | Both `allowed_domains` and `blocked_domains` were passed |
| `invalid-url` / `hostname-too-shallow` / `userinfo-not-allowed` / `url-too-long` | Rejected fetch URL |
| `search-failed` / `fetch-error` | The provider failed or returned nothing usable |
| `quota-exceeded` | The provider reported a usage limit |
## Adding your own backend
Backends live in a registry, so a new provider does not require changes to the
engine or to any environment:
```python theme={null}
from openreward.search_backends import SearchBackend, register_backend
from openreward.tools.web import WebToolResult, format_search_output, validate_search_input
class ExaBackend(SearchBackend):
name = "exa"
supports_as_of = False
def __init__(self, *, config=None, secrets=None):
...
async def search(self, *, query, k=None, as_of=None,
allowed_domains=None, blocked_domains=None) -> WebToolResult:
query, allowed, blocked, error = validate_search_input(query, allowed_domains, blocked_domains)
if error is not None:
return error
hits = [...] # {title, url, snippet, score}
return WebToolResult.success(
format_search_output(query, hits), # shared renderer — do not hand-roll
{"query": query, "hits": hits, "mode": "exa"},
)
async def fetch(self, *, url, prompt, as_of=None) -> WebToolResult:
...
register_backend("exa", ExaBackend)
```
Two rules make backends interchangeable:
* **Render through the shared helpers** (`format_search_output`,
`render_fetch_text`) and **validate through the shared validators**
(`validate_search_input`, `validate_fetch_url`). Models are trained on the
exact output shape, and identical inputs must produce identical error codes.
* **Never raise for a recoverable problem.** Return
`WebToolResult.error(code, message)` so the agent can read it and try
something else.
## Notes
* **Citations.** Search output is formatted so the model is reminded to cite the
returned URLs as markdown links in its final answer.
* **Caching.** Fetches are cached in-process for 15 minutes. Cache keys are
namespaced per backend (and, for backdated backends, per cutoff), so a live
page is never served out of a backdated fetch's entry or vice versa.
* **Soft errors.** Bad inputs come back as a normal result with `.ok == False`
and a clear message rather than as an exception.
## Next Steps
The point-in-time corpus, and the toolset pinned to it
Compose document toolsets and build custom ones
Call the search and fetch endpoints directly over HTTP
How API keys and secrets reach your environment