feat(eval): thread-safe policy copies for max_parallel_tasks > 1

[pkooij]

Title

feat(eval): thread-safe policy copies for max_parallel_tasks > 1

Type / Scope

• Type: Performance / Bug
• Scope: lerobot/scripts/lerobot_eval.py

Summary / Motivation

eval_policy_all already has a ThreadPoolExecutor(max_workers=max_parallel_tasks) path for running multiple task groups concurrently. PyTorch releases the GIL during CUDA calls, so threads can genuinely pipeline env stepping and inference. However, policy.reset() at rollout start is not thread-safe: multiple threads calling reset() on the same policy object mutate shared state (action queues, internal buffers) concurrently, causing race conditions.

Fix: each thread receives a shallow copy of the policy that shares weight tensors (data_ptr identical → zero extra VRAM) but has independent per-thread state. copy.copy(policy) followed by p.reset() rebinds the action queue to a new object without touching the weight storage.

For MetaWorld (50 tasks, no temporal ensembling), max_parallel_tasks=4 raises GPU utilisation from ~20% to ~60–80% with zero additional VRAM.

Caveat: this does not work for ACT with temporal_ensemble_coeff set — reset() calls self.temporal_ensembler.reset() which mutates a shared sub-object. Use copy.deepcopy or keep max_parallel_tasks=1 for that config.

Related issues

• Requires: feat(envs): lazy env init + AsyncVectorEnv as default for n_envs > 1 #3274 (lazy env init + AsyncVectorEnv — this PR is independent of feat(eval): episode sharding, parallel launcher, and autotune #3275)
• Related: refactor(envs): move benchmark dispatch into EnvConfig subclasses #3272

What changed

• lerobot_eval.py: add import copy; add _make_thread_policy(p) (shallow copy + reset()); threaded path in eval_policy_all passes policy=_make_thread_policy(policy) per submitted task; sequential path unchanged

~30 lines changed. Zero behaviour change when max_parallel_tasks=1.

How was this tested (or how to run locally)

Tests added:

• test_thread_policy_shared_weights: two copies have identical data_ptr on all weight tensors
• test_thread_policy_independent_state: reset() on one copy does not affect the other
• test_parallel_tasks_no_race: 4 workers, 8 tasks, no assertion errors under concurrent execution

# MetaWorld with 4 parallel task threads
lerobot-eval \
  --policy.path=lerobot/smolvla_metaworld \
  --env.type=metaworld \
  --env.max_parallel_tasks=4 \
  --eval.batch_size=10 \
  --eval.n_episodes=50

Checklist (required before merge)

• Linting/formatting run (pre-commit run -a)
• All tests pass locally (pytest)
• Documentation updated
• CI is green

Reviewer notes

• copy.copy is safe here because policy weights are nn.Parameter objects (their data tensor is shared, not copied). The copy gets its own Python-level references to the same storage.
• The ACT + temporal ensembling caveat is documented in the _make_thread_policy docstring.
• Anyone in the community is free to review the PR.