STACX

What is STACX

STACX (STand Alone CompleX) is a modular framework for agentic training, evaluation, and reinforcement learning. Three engines under one Ray job: roll out an agent, score it in a real sandbox, and train on the scored traces.

Every run — training or eval — follows the same path:

The rollout loop — tasks fan out to the scheduler, an agent acts inside a sandbox, a verifier turns the trace into a reward, and the trainer syncs refreshed weights back for the next iteration.

Those stages map onto five long-lived components spread across the GPU arrays:

STACX overall framework — the Trainer owns pretrain / SFT / RL; the Rollout manager drives generation and evaluation against the Environment engine, with weights synced back over the loop.

The three engines

In practice you set up two things: the sandbox engine (a ROCK service that owns Docker/Podman containers and exposes the rock admin CLI) and the train/eval runtime (the pinned lichangh20/slime-rl:stable Docker image the train and eval drivers run in). Together they carry three responsibilities.

Train an agent

The launch scripts under scripts/ encode the model, data, Docker, Ray, SGLang, sandbox, checkpoint, and W&B settings for each recipe — start from one of these rather than calling rl_engine.train directly. Every script auto-detects the repo root and auto-resumes from resume_state.json when re-run with the same config.

In-house SWE agent — five recipes × 4B/8B

The in-house SWE agent (rl_engine/examples/swe_agent/) trains a small student against a frozen Qwen3-Coder-30B teacher on SWE-Gym tasks — training sees every token natively. Five recipes share the same student and eval; each has a _4b and a _8b variant.

export WANDB_KEY=<your-key>
bash scripts/train/swe/grpo/grpo_4b.sh          # swap in dagger / aggrevate / sft / opd; _8b for 8B

Deep dive: scripts/train/swe/QUICKSTART_OPD.md and the repo AGENTS.md runbook; every knob is catalogued in scripts/train/swe/ENV_REFERENCE.md.

Installed-scaffold training

A second lane trains black-box installed scaffolds (OpenHands, Terminus-2) inside Harbor task containers, capturing tokens through a recording proxy. One Ray job rolls out on the train bench, applies an SFT/GRPO update, syncs weights, and periodically evals on the eval bench.

# SFT on SkyRL, eval on SWE-bench (the defaults)
BENCH=skyrl EVAL_BENCH=swebench SCAFFOLD=openhands ALGO=sft \
  bash rl_engine/examples/shared/docker_run_train.sh
# axes: SCAFFOLD=openhands|terminus2   ALGO=sft|grpo

Prerequisite: both benches need their prepare_jsonl + build_images artifacts first (see Evaluate below).

Other training lanes

Evaluate a benchmark

Evaluation is four commands: download the Harbor task tree, convert it to a tasks.jsonl, build the per-task Docker images, then run the shared eval script. The eval driver runs inside the runtime image via shared/docker_run_eval.sh and connects to a running SGLang server rather than launching one.

# 1. download the Harbor task tree (public registry; --task-names-file for a subset)
python rl_engine/examples/shared/download_tasks.py swebench-verified@1.0 \
  -o external/harbor-datasets/swebench-verified_1.0
# 2. Harbor task dirs -> tasks.jsonl
python rl_engine/examples/shared/prepare_jsonl.py \
  --tasks-dir external/harbor-datasets/swebench-verified_1.0 \
  --output rl_engine/examples/swebench/config/tasks.jsonl --registry swebench
# 3. build the per-task images
bash rl_engine/examples/shared/build_images.sh rl_engine/examples/swebench/config/tasks.jsonl
# 4. run eval (SCAFFOLD=openhands|terminus2)
BENCH=swebench SCAFFOLD=openhands SGLANG_ROUTER_PORT=9001 \
  bash rl_engine/examples/shared/docker_run_eval.sh

Six benchmarks ship with STACX and run through shared/docker_run_eval.sh with BENCH=<name>:

The in-house agents evaluate through the eval-only path of rl_engine.train instead, each with a standalone per-example scripts/eval.sh: the in-house SWE agent (SWE-bench Verified), MLE-Dojo (Kaggle competitions, eval-only), and ReTool (AIME) — rl_engine/examples/{swe_agent,mle_dojo,retool}/. These runs serve the model in-process, so no external SGLang server is needed. See the Rollout Engine and Environment Engine pages for how rollout and sandboxes cooperate during eval.

Integrate a new benchmark

Two steps take a benchmark from raw source to a runnable STACX example:

1. Convert to Harbor task format — one directory per task, each with instruction.md, task.toml, environment/Dockerfile, and tests/test.sh. The verifier writes a bare float reward to /logs/verifier/reward.txt.
2. Add the STACX example — create rl_engine/examples/<bench>/ (an evaluator_config.py + config/ + README.md) so the benchmark runs through shared/docker_run_eval.sh. The layout and profile knobs are documented in rl_engine/examples/SKILL.md.

Key invariant: BenchmarkProfile.evaluator_id must equal task_type in config/eval_datasets.yaml. For the sandbox and container plumbing this touches, see Sandbox Management.

Documentation map

Where to go next — grouped the way the sidebar is laid out on every other page.