ProgramLab: Train single and multi agent to rebuild programs with RL

October 2026 · Research

Train single and multi agent to rebuild programs with RL

Can an agent learn to rebuild a whole program just by watching it run? We introduce ProgramLab: 1,262 RL environments for agents learning to rebuild programs through trial-and-errors. Training Qwen3.6-35B-A3B on ProgramLab gains +10.3 points as a single agent and +7.9 points as a five-agent team on ProgramBench! We open-source the environments and recipes so everyone can try it!

1,262RL environments from real Go, Rust and C/C++ projects

+10.3single agent on ProgramBench
13.7% → 24.0%

+7.9five-agent team on ProgramBench
17.2% → 25.0%

Train one agent, or a whole team

Using Qwen3.6-35B-A3B, we run RL on ProgramLab and evaluate on ProgramBench:

Single agent: 13.7% → 24.0% in Claude Code under a two-hour budget.
A 5-Agent team: 17.2% → 25.0% in under a 30-minute budget.

Long-horizon agentic RL improves across harnesses. Trained in GitHub Copilot CLI, the policy improves ProgramBench in all harnesses we test, including three unseen during training: Claude Code (+10.3), Codex CLI (+8.3), and mini-SWE-agent (+4.1). Under an extended six-hour budget, Claude Code goes from 19.0 to 27.6.

ProgramBench pass rate before and after RL across four harnesses — **Figure 1.** ProgramBench pass rate of Qwen3.6-35B-A3B before and after RL on ProgramLab, with a two-hour budget. RL runs only in GitHub Copilot CLI, yet the gains carry over to all four harnesses.

Agent learns to write compilable programs during RL. During RL training with 1024 rollouts per step(64 task * 16 rollout), rollouts grow from about 43 to 72 turns while mean training reward rises with 8× more rollouts ending with a compiling program (3.4% → 27.9%), under 2-hour rollout and 128k context budget.

Training reward and turns per rollout during single-agent RL — **Figure 2.** Single-agent RL training on ProgramLab. Left: mean training reward at each step. Right: mean turns per rollout, which grow from 42.8 in the first ten steps to 72.0 in the last ten.

Multi-agent RL also works. RL on ProgramLab raises a five-agent Qwen3.6-35B-A3B team from 17.2% to 25.0% (+7.9 points) on ProgramBench under a 30-minute budget.

Five-agent team pass rate during multi-agent RL — **Figure 3.** ProgramBench pass rate of the five-agent team during multi-agent RL, on all 200 tasks with a 30-minute budget.

Data recipe

How we build ProgramLab. We collect and filter open programmable projects from package registries (Homebrew, crates.io, pkg.go.dev) and GitHub which reliably produce a standalone executable. For each task, we use Codex CLI with GPT-5.5 to construct the environment following three steps below, each checked by an independent validator that does not trust the agent's own report:

Build the reference. Compile the project's main executable and record the build script.
Synthesize behavioral tests. Probe the executable and write tests that assert on its observable effects. A test is kept only if it passes on the reference, passes on repeated runs, and fails on a dummy binary that accepts any input. We design a coverage-guided iteration to produce high-quality and compact test suites that are suitable for RL.
Coverage-guided iteration. After each round, we measure line coverage of the reference on an instrumented build nd feed the coverage report back to the agent, which then writes tests for the behavior it has not yet exercised. The loop repeats until coverage stops improving, so each new test has to reach code that earlier tests missed.
Package the cleanroom. Ship an execute-only binary and its documentation; remove source, build caches, and version-control history.

What's in ProgramLab? ProgramLab contains 1,262 tasks from 1,262 distinct public repositories (566 Go, 416 Rust, 280 C/C++), with 187,208 hidden behavioral tests in total. The reference programs are substantial: the median project has an estimated 11.5K lines of source code, and about 150 exceed 100K lines. Each task contains:

A reference executable, built from a pinned commit and shipped execute-only.
Documentation: usage text, help output, man pages, and any fixtures the agent cannot create itself.
A hidden test suite: a median of 127 tests that run the agent's program and compare its observable behavior with the reference.
- The result is a small but thorough suite: a median of 127 tests (ProgramBench has 770) covers 76.6% of the reference's first-party lines, achieving over 90% for projects under 5K lines.
Provenance: the source repository and commit the executable was built from.

Tasks ship in the ProgramBench format, so the official ProgramBench evaluator scores them without modification. Following ProgramBench's difficulty score, 205 tasks are easy, 889 medium, and 168 hard.

Task difficulty distribution — **Figure 5.** Task difficulty in ProgramLab under ProgramBench’s difficulty score: 205 easy, 889 medium, and 168 hard tasks.

Line coverage against project size — **Figure 6.** Line coverage of each hidden test suite against project size, falling from over 90% under 5K lines to 42.0% above 50K lines.

Why ProgramLab is suitable for RL

Graded. The reward is the fraction of tests passed, so a partially working program earns partial credit. In our single-agent run, 95% of training tasks gave a nonzero reward at least once, so nearly every task provides learning signal from the start.
Validated. We drop tests that the reference fails in the cleanroom (4,047), that a no-op binary passes (1,738), or that are flaky (550). Every released task scores 1 with the reference and 0 with an empty submission.
Prevent hacking
Fast to score. Because the coverage-guided loop adds a test only when it reaches new code, suites stay small and every rollout is cheap to score.

Learning to rebuild programs with RL

In each task, the agent works in a cleanroom container with a build toolchain and no network access. It can run the reference executable and read its documentation (usage text, help output, man pages), but it cannot read the source. It writes code in any language, plus a compile.sh that builds a candidate executable. ProgramBench (Yang et al., 2026) makes this challenge concrete, and even state-of-the-art agents still struggle on it. We wanted to explore whether open models can learn this skill through RL on ProgramLab, as a single agent or as a whole team.

Overview of ProgramLab task construction and RL — **Figure 4.** Overview of ProgramLab. (a) Agents turn public Go, Rust, and C/C++ projects into packaged tasks with hidden tests. (b) During RL, the coding agent probes the locked executable in a cleanroom, writes a new program, and is rewarded by the fraction of hidden tests it passes.

For scoring, the candidate is rebuilt in a fresh container and run against a hidden test suite. The reward is the fraction of tests passed, r = |T_pass| / |T| ∈ [0, 1].

Trajectory collection for black-box harnesses. We run off-the-shelf harnesses as the policy's interface.

Recording proxy. The cleanroom allows only loopback networking, so every model call, from every agent, goes through a proxy that logs prompt tokens, sampled tokens, and log-probabilities. Calls are merged into training sequences by prefix matching, and only sampled tokens are trained.
Context compaction. When the harness summarizes its history, a new training segment starts. The summarization call is trained too.
Asynchronous training. Generation and training overlap, with a cap on how stale samples can be.

Multi-agent trajectory collection. Because every worker's model calls pass through the same recording proxy, the team's rollout is collected like a single-agent one and scored once on the final program. All five agents share one policy, and every agent's segments carry the team's reward.

Run configurations
	Single agent	Multi agents
Model	Qwen3.6-35B-A3B	Qwen3.6-35B-A3B
Training harness	GitHub Copilot CLI	Agensh with 5 agents running GitHub Copilot CLI
Rollouts per task	16	4
Training budget per rollout	2 hours, 128K context	30 minutes, 256K context

Building programs from scratch: ProgramBench, Commit0, NL2Repo-Bench, RepoZero, Zero2Repo.
Training environments for coding agents: SWE-Gym, R2E-Gym, SWE-smith, MindForge, CUA-Gym.
Agentic RL: SkyRL-Agent, Golubev et al., LLM-in-Sandbox, AReaL, ReSum, SUPO, Agent Lightning.
Multi-agent harnesses and RL: Agensh. [TODO: add multi-agent RL papers, ask Zhihao]

What's next

Open-sourcing the data and training pipeline. The data synthesis pipeline will follow, so others can extend ProgramLab, along with our RL training infrastructure, rl-bridge [TBD: link]. We expect to release both in November.

BibTeX

@misc{programlab2026,
  title={ProgramLab: Train single and multi agents to rebuild programs},
  author={Yao Tang and Daixuan Cheng and Li Dong and Zhihao Zhan and Shaohan Huang and Jiatao Gu and Furu Wei},
  howpublished={Blog Post},
  url={https://programlab.blog/},
  year={2026}
}