PyTRIO setup and your first GRPO training run

Your local machine handles data, tools and the experiment loop. PyTRIO executes model sampling and training remotely. You need Python 3.13 or later, uv, and a PyTRIO account with access to training resources.

For a map of the experiments, read the PyTRIO tutorials. Readers familiar with Tinker can start with the training workflow comparison.

Get the code and install dependencies

git clone https://github.com/KMnO4-zx/agentic-rl-lab.git
cd agentic-rl-lab
uv sync

pyproject.toml and uv.lock record the experiment dependencies. Dataset, external API and environment requirements vary by chapter; follow the relevant quick start.

Log in to the services

uv run trio login
uv run swanlab login

PyTRIO handles training and sampling; SwanLab tracks training metrics. Available models and account access depend on the platforms’ current configuration.

Run GRPO

Start with the GRPO quick start. It translates the repository’s 01-grpo/start.md, including the current entry point, sampling settings and metric descriptions.

Understand the loop

Read the GRPO walkthrough alongside the code: prepare prompts → sample a group → compute rewards and advantages → update the policy. Training consumes remote service resources; begin with the small configuration in the quick start.

Add interactive environments

ALFWorld, TEMPO and AgentOPSD also need the environment dependencies:

uv sync --extra alfworld

Then prepare data and environments for ALFWorld, TEMPO or AgentOPSD. Search-R1 retrieval backends, the ReTool local sandbox and Spec-o3 multimodal data have their own configuration requirements.

Keep an experiment reproducible

Record the model name, checkpoint, data split, random seed, sampling budget and evaluation commands. Align these conditions before comparing methods.

Choose your next direction in the experiment handbook.