Run all commands from the repository root. The project requires Python >=3.13.

uv sync
trio login
swanlab login

1. Download the data

uv run python 03-search-r1/prepare_data.py

For DeepSeek Search, log in once:

uv run deepseek-search login

Wikipedia search does not require an API key. The client uses the English Wikimedia Action API with a default concurrency of 3, following its published limits.

For Zhihu search, set ZHIHU_SEARCH_KEYS in 03-search-r1/.env. The template is 03-search-r1/.env.example.

2. Start training

DeepSeek Search backend:

uv run python 03-search-r1/train.py \
    --max-steps 20 \
    --questions-per-batch 8 \
    --group-size 8 \
    --save-every 5 \
    --base-model Qwen/Qwen3.5-4B \
    --search-backend deepseek \
    --run-name search-r1-qwen35-4b-deepseek \
    --swanlab-mode online

Wikipedia backend:

uv run python 03-search-r1/train.py \
    --max-steps 20 \
    --questions-per-batch 8 \
    --group-size 8 \
    --save-every 5 \
    --base-model Qwen/Qwen3.5-4B \
    --search-backend wikipedia \
    --run-name search-r1-qwen35-4b-wikipedia \
    --swanlab-mode online

Zhihu search backend:

uv run python 03-search-r1/train.py \
    --max-steps 20 \
    --questions-per-batch 8 \
    --group-size 8 \
    --save-every 5 \
    --base-model Qwen/Qwen3.5-4B \
    --search-backend zhihu \
    --run-name search-r1-qwen35-4b-zhihu \
    --swanlab-mode online

For the full training run, set --max-steps to 100 and --save-every to 50.

3. Run evaluation

The examples below use DeepSeek Search. First evaluate the base model:

uv run python 03-search-r1/eval.py \
    --batch-size 16 \
    --base-model Qwen/Qwen3.5-4B \
    --search-backend deepseek \
    --search-model deepseek-v4-flash \
    --search-concurrency 16 \
    --search-timeout 60 \
    --output 03-search-r1/eval_result/eval_results_base_deepseek_search.jsonl

Then evaluate the trained sampler weights:

uv run python 03-search-r1/eval.py \
    --batch-size 16 \
    --model-path 'trio://RUN_ID/sampler_weights/STEP_20_WEIGHTS_NAME' \
    --search-backend deepseek \
    --search-model deepseek-v4-flash \
    --search-concurrency 16 \
    --search-timeout 60 \
    --output 03-search-r1/eval_result/eval_results_rl_step_20_deepseek_search.jsonl

For Wikipedia, use --search-backend wikipedia --search-concurrency 3 --search-timeout 15 in both commands. For Zhihu, use --search-backend zhihu. The base model and checkpoint must use identical backends and search settings.