Getting Started

1 Prerequisites

STACX targets a Linux host with Docker and CUDA-capable GPUs. A single 8-GPU node (A100 / GH200 class) runs the full teacher + student + sandbox stack; smaller setups are fine for evaluation and CPU-only dev installs.

echo <your-docker-hub-token> | docker login -u <your-docker-hub-user> --password-stdin

2 Install the sandbox engine

Clone the repo, then set up the sandbox engine (ROCK). It runs in its own small uv environment and must be started before any eval or training that executes verifiers. In a separate terminal:

# clone
git clone <this-repo> && cd stacx

# sandbox engine — its own env, its own terminal
cd env_engine
uv venv --python 3.11 --python-preference only-managed
uv sync --all-extras
source .venv/bin/activate

rock admin start                   # listens on http://127.0.0.1:8080

Verify that Docker and the admin server are both reachable:

docker --version
curl http://127.0.0.1:8080/        # admin server root; 200 = up

The admin server is the single entry point for sandbox lifecycle (/apis/envs/sandbox/v1/…): a Ray-based resource scheduler tracks GPUs/CPUs and starts, reuses, and tears down per-task Docker (or Podman) containers. Full detail lives in env_engine/README.md and env_engine/README_ROCK.md, plus the Sandbox Management page.

Smoke-test a sandbox

Before committing to a long run, confirm ROCK can start, exec into, and stop a real SWE-Bench container. Save this as test_rock.py and run it inside the ROCK venv:

import asyncio, time
from rock.actions import CreateBashSessionRequest
from rock.sdk.sandbox.client import Sandbox
from rock.sdk.sandbox.config import SandboxConfig

async def main():
    t0 = time.time()
    cfg = SandboxConfig(
        image="swebench/sweb.eval.x86_64.django_1776_django-10097:latest",
        memory="4g", cpus=1.0)
    sb = Sandbox(cfg)
    await sb.start();                          print(f"[{time.time()-t0:.1f}s] started")
    await sb.create_session(CreateBashSessionRequest(session="bash-1"))
    r = await sb.arun(cmd="cat /etc/os-release | head -1", session="bash-1")
    print(f"[{time.time()-t0:.1f}s] exec OK:", r.output.strip())
    await sb.stop();                           print(f"[{time.time()-t0:.1f}s] stopped")

asyncio.run(main())

3 Pull the train/eval runtime

There is nothing to install for the RL engine itself — train and eval drivers run inside a pinned Docker image, entered through the thin wrappers shared/docker_run_train.sh / shared/docker_run_eval.sh (the repo is mounted into the container). Pull it once:

docker pull lichangh20/slime-rl:stable

4 Run your first evaluation

Evaluating an installed-scaffold agent is four commands: download the Harbor task tree, convert it into a tasks.jsonl, build the per-task Docker images, then run the shared eval script. The eval driver connects to a running SGLang server rather than launching one — two one-time pieces of setup come first.

One-time: SGLang serving env

Installed-agent eval talks to an external SGLang server, run from a small pinned uv env at the repo root (an SGLang new enough for Qwen3.5 — the training image's is too old). Only installed-agent eval uses this env: training in both lanes and in-house agent eval serve SGLang in-process inside the runtime image.

uv venv .venv-sglang --python 3.11
source .venv-sglang/bin/activate
uv pip install --prerelease=allow "sglang[all]==0.5.10rc0"

One-time: site profile

Once per server, set the host IP and the sandbox GPU budget (gpus, minus the serving GPUs) in the site profile — it is the single source of truth for serving GPUs, TP size, and address, so the server and the eval agree by construction:

sed -i "s/REPLACE_WITH_SERVER_IP/$(hostname -I | awk '{print $1}')/" \
  rl_engine/examples/shared/profiles/cluster_layout_default.json

Detail: rl_engine/examples/shared/profiles/README.md.

Serve the model

# blocks in the foreground; run it in its own terminal
source .venv-sglang/bin/activate
bash rl_engine/examples/shared/serve_sglang.sh

curl http://<node-ip>:9001/v1/models    # 200 = ready

serve_sglang.sh reads the site profile, so the serving GPUs, TP size, and address match the eval automatically. Defaults: MODEL=Qwen/Qwen3.5-35B-A3B-FP8, PORT=9001; per-run knobs (MODEL, PORT, SGLANG_SERVE_GPUS, SGLANG_TP_SIZE) are in the script header. Wait for "The server is fired up and ready to roll!".

Download tasks, build images, run

With the sandbox engine running (§2) and the server up — SWE-bench Verified, Django-100 subset. Harbor task trees stage in the repo-local, gitignored external/harbor-datasets/, downloaded straight from the public Harbor registry — no Harbor install needed:

# 1. download the Harbor task tree (Django-100 subset; without
#    --task-names-file the FULL ~500-task verified set downloads and
#    step 3 then builds ~500 images)
python rl_engine/examples/shared/download_tasks.py swebench-verified@1.0 \
  -o external/harbor-datasets/swebench-verified_1.0 \
  --task-names-file rl_engine/examples/swebench/config/django100_task_names.yaml

# 2. Harbor task dirs → tasks.jsonl
python rl_engine/examples/shared/prepare_jsonl.py \
  --tasks-dir external/harbor-datasets/swebench-verified_1.0 \
  --task-names-yaml rl_engine/examples/swebench/config/django100_task_names.yaml \
  --output rl_engine/examples/swebench/config/tasks_django100.jsonl \
  --registry swebench

# 3. build the per-task images
bash rl_engine/examples/shared/build_images.sh \
  rl_engine/examples/swebench/config/tasks_django100.jsonl

# 4. run eval (OpenHands scaffold; SCAFFOLD=terminus2 for Terminus-2)
BENCH=swebench SCAFFOLD=openhands SGLANG_ROUTER_PORT=9001 \
  bash rl_engine/examples/shared/docker_run_eval.sh

Outputs land under external/results/eval/: the per-task results JSONL (reward, termination reason, verifier data), its .summary.json, the driver .log, and per-task sandbox logs under container_logs/<run>/. The wrapper detaches and prints a tail -f command to follow the run. Other benchmarks swap BENCH=<name> (terminal_bench, algotune, kernel_bench, seta, skyrl); each example's README is the self-contained walkthrough.

5 Launch your first training run

The launch scripts under scripts/train/swe/ encode the model, data, Docker, Ray, SGLang, sandbox-engine, checkpoint, and W&B settings for each recipe — start from one of these rather than calling rl_engine.train directly. We use the DAgger recipe (mixed-loss DAgger-OPD, 4B student) as the first run. It co-locates the whole stack on one node:

Stage external/

external/ is .gitignored — you stage models, data, and output roots locally (~60 GB). For the 4B DAgger run you need the SFT-iter0 student (both the HF dir and the _torch_dist dir), the 30B teacher, and the four data JSONLs — all public, no HuggingFace login needed (uvx runs the HF CLI with no host env to install):

# pre-create the bind-mount roots as yourself (see the note below)
mkdir -p external/{models,data,results,ckpts,logs}

# 4B student (SFT-iter0): HF dir (tokenizer/chat template) + torch_dist (slime --load)
uvx --from 'huggingface_hub[cli]' hf download lichangh20/qwen3-4b-instruct-sft-swegym-iter0 \
    --local-dir external/models/qwen3-4b-instruct-sft-iter0
uvx --from 'huggingface_hub[cli]' hf download lichangh20/qwen3-4b-instruct-sft-swegym-iter0-distcp \
    --local-dir external/models/qwen3-4b-instruct-sft-iter0_torch_dist

# teacher (Qwen3-Coder-30B) + the 4 SWE data jsonls
uvx --from 'huggingface_hub[cli]' hf download Qwen/Qwen3-Coder-30B-A3B-Instruct \
    --local-dir external/models/qwen3-coder-30b-a3b-instruct
uvx --from 'huggingface_hub[cli]' hf download lichangh20/stacx-swe-online-dagger-data \
    --repo-type dataset --local-dir external/data/swe

Launch

With ROCK running (§2) and the runtime image pulled (§3), one command drives the whole run. This lane is self-contained — teacher and student SGLang are served in-process on the node's own GPUs, so the external server and site profile from §4 are not used. WANDB_KEY is the only required env var (this exact name — the scripts do not read WANDB_API_KEY):

export WANDB_KEY=<your-wandb-key>

bash scripts/train/swe/online_dagger/dagger_4b.sh        # dagger, 4B
# aggrevate_4b.sh · dagger_8b.sh · aggrevate_8b.sh — same pattern

End to end, with no manual steps in between, the script:

1. Health-checks ROCK at 127.0.0.1:8080 (aborts if it is down).
2. Launches the teacher SGLang (Qwen3-Coder-30B, TP=4) on GPUs 0–3 at 127.0.0.1:30055 and waits for ready (~3–5 min warmup; it reuses an already-healthy teacher).
3. Launches the student training container on GPUs 4–7.
4. Runs 5 rollouts × 512 tasks (≈1 epoch); per iter: rollout → finish-only filter → teacher force-decode (K3 logprobs) → 3 train passes → SGLang weight sync → eval on 3 datasets → save checkpoint.

Checkpoints land under external/ckpts/online_dagger_mixed_opd/<RUN_TAG>/, per-task eval JSONL + failure-class summaries under external/results/eval/swe_agent_online_dagger_mixed_opd/<RUN_TAG>/, and training logs under external/logs/. Re-running the same command auto-resumes from resume_state.json — it skips completed rollouts, reloads weights + optimizer state, and re-opens the same W&B run. For the full walk-through and knob catalog, read the DAgger-OPD quickstart and the Algorithm Recipes page.

6 Where to go next

The rest of the docs go deep on each engine. Start with the recipe you plan to run, then the component pages behind it.