Environment Engine

1 Overview

The Environment Engine (env_engine/) is the execution backend for agentic rollouts — a self-contained service the training pipeline reaches only over a small HTTP API, sending a request and reading back a result rather than touching container logic or the scheduler directly.

It is built on ROCK (Reinforcement Open Construction Kit), a client–server framework that owns the container lifecycle. On the reference 8-GPU node (see Getting Started), the ROCK admin server runs co-located on 127.0.0.1:8080 and orchestrates one SWE-Bench container per task.

2 Layered architecture

A request travels down four layers; the result returns up:

Every layer maps to a concrete module under env_engine/rock/:

3 The sandbox REST API

All sandbox verbs are registered on sandbox_router (rock/admin/entrypoints/sandbox_api.py) and mounted with a version prefix in rock/admin/main.py:

# rock/admin/main.py
app.include_router(sandbox_router, prefix="/apis/envs/sandbox/v1", tags=["sandbox"])
app.include_router(warmup_router,  prefix="/apis/envs/sandbox/v1", tags=["warmup"])
app.include_router(gem_router,     prefix="/apis/v1/envs/gem",   tags=["gem"])

Every handler is wrapped by @handle_exceptions(…) and returns a RockResponse[T] envelope whose status field is "Success" or "Failed". The core lifecycle verbs:

Two sibling routers share the same prefix: warmup_api.py pre-pulls task images so the first task avoids full cold-start, and sandbox_proxy_api.py routes each request to the node owning that sandbox. A separate GEM router (rock/admin/gem/api.py, /apis/v1/envs/gem) exposes the Gym-style make / reset / step protocol for turn-based environments like the Sokoban demo.

4 Python SDK

Callers use the async SDK in rock/sdk/sandbox/: Sandbox (in client.py) wraps every REST verb as a coroutine, and SandboxConfig (in config.py) declares the container spec. The canonical smoke test from env_engine/README_ROCK.md:

import asyncio
from rock.actions import CreateBashSessionRequest
from rock.sdk.sandbox.client import Sandbox
from rock.sdk.sandbox.config import SandboxConfig

async def run_sandbox():
    cfg = SandboxConfig(image="python:3.11", memory="8g", cpus=2.0)
    sb  = Sandbox(cfg)
    await sb.start()                                            # POST /start
    await sb.create_session(CreateBashSessionRequest(session="bash-1"))
    r = await sb.arun(cmd="echo Hello Rock", session="bash-1")  # /run_in_session
    await sb.stop()                                             # POST /stop

asyncio.run(run_sandbox())

Sandbox also exposes execute, run_nohup_and_wait, write_file/read_file, upload, commit, and close; SandboxGroup starts a pool of identical sandboxes with bounded start concurrency and retry. The config surface:

5 How training consumes it

The in-house SWE agent skips the high-level SDK: rl_engine/rollout/sandbox/rock.py defines a thin RockSandbox wrapper that POSTs to the same /apis/envs/sandbox/v1 endpoints with aiohttp — deliberately dependency-free ("ROCK API is NOT mandatory"), so the rollout never imports ROCK:

The SWE-Bench per-step flow

Task configuration lives in rl_engine/examples/swe_agent/environment/swe_sandbox.py (SWE_TOOL_CONFIGS). For each task the agent:

1. resolves the per-instance image with get_swe_bench_image(instance_id) — e.g. swebench/sweb.eval.x86_64.django_1776_django-16911:latest;
2. starts a container (cpus=4, memory="16g") with the editor entrypoint bind-mounted read-only (SWE_ENTRYPOINT_LOCAL; mount details in Sandbox Management §5);
3. dispatches each tool call into a bash session (through run_in_session) under conda activate testbed — file-edit actions run python3 /home/swe_execute_sandbox.py --action … --params …; bash actions run the command directly.

One env.step round trip against an already-started sandbox; container start is a separate call (timing below).

Startup timing & image supply

6 Deployments & configuration

The bottom layer is a family of deployment backends under rock/deployments/, selected by DeploymentManager (manager.py) and sharing the AbstractDeployment contract (abstract.py):

Podman is not a separate module — DockerDeployment shells out through a container provider (rock/utils/providers/) that abstracts over both runtimes, so the same task graph runs unchanged on either.

Configuration files

Runtime config lives in env_engine/rock-conf/: rock-local.yml drives single-node development, rock-test.yml targets the test suite:

# env_engine/rock-conf/rock-local.yml (excerpt)
ray:
    runtime_env:
        working_dir: ./
        pip: ["apscheduler", "opentelemetry-api", "gem-llm", ...]
    namespace: "rock-sandbox-local"
warmup:
    images: ["python:3.11"]
runtime:
    enable_auto_clear: true

The rock CLI

The rock command (rock/cli/main.py, subcommands under rock/cli/command/) manages the admin process via AdminCommand, with two actions:

# start the admin server (default 127.0.0.1:8080)
rock admin start

# graceful SIGTERM to running admin processes
rock admin stop

Install independently of the trainer: build the ROCK venv per env_engine/README_ROCK.md (uv-managed Python recommended), then rock admin start; a source/pip install also needs ROCK_WORKER_ENV_TYPE=pip. Launch scripts health-check this server before every run and abort if it is down (AGENTS.md).

7 Directory tour

8 Operations reference

Ports & health

Key environment variables

The full registry is rock/env_vars.py; the knobs that matter most in training:

EnvHub

EnvHub (rock/envhub/server.py) is a standalone, SQLite-backed registry for reproducible environment definitions. The SWE lane bypasses it (images come from Docker Hub by instance id); Gym-style environments resolve their images through it.

Where to look next