ParaVT — Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

Video LMMs · Agentic RL · Long Video Understanding

ParaVT Taming the Tool Prior Paradox
for Parallel Tool Use in Agentic Video Reinforcement Learning

Zuhao Yang2,6 Kaichen Zhang3,6 Sudong Wang4 Keming Wu5,6 Zhongyu Yang2 Bo Li6 Xiaojuan Qi3 Shijian Lu2,✉ Xingxuan Li1,✉ Lidong Bing1

1MiroMind 2Nanyang Technological University 3The University of Hong Kong 4Hong Kong University of Science and Technology (Guangzhou) 5Tsinghua University 6LMMs-Lab

✉Corresponding authors

Can a large multimodal model learn to natively invoke tool calls in parallel under agentic RL — when the very prior that enables tool use also destabilizes it?

Paper Code Model Data Daily Paper

69.4

VideoMME w/ sub — open-source 7-8B SOTA

+7.9

average gain over Qwen3-VL-8B base (%)

0.64

peak parallel tool-call format compliance

benchmarks topped at 7-8B

SFT cold-start samples

4.4K

PARA-GRPO RL samples

Motivation

Long-video understanding is increasingly framed as agentic video reasoning: a large multimodal model (LMM) post-trained with reinforcement learning to invoke video-processing tools. Prior native-RL methods, including our earlier LongVT (CVPR 2026), dispatch these tool calls sequentially, one per turn, which is brittle to single mis-localizations, prone to multi-turn context drift, and linear in inference cost.

Failure modes

Two failure modes of vanilla GRPO on a tool-native LMM. Format Fragility (left): under temperature sampling, the policy still generates reasoning content but its SFT-learned <think>/<tool_call>/<answer> boundary closures decay, leaving rollouts that look almost-parseable but mis-tagged. Tool Necessity Gap (right): with 64 overview frames many prompts can be answered without tools, so the GRPO advantage between calling and skipping is near-zero, and the parallel tool-call rate collapses within a few RL steps while accuracy oscillates flatly.

Tool Prior Paradox

The shared cause — pretrained tool priors. A weaker-prior LMM stays format-stable but RL elicits zero tool calls; a stronger-prior LMM explores tools but loses format. Prior strength drives both failure modes, so a workable recipe must keep format and reward tool-use credit at the same time. The contrast pairs Qwen2.5-VL-7B (weak prior: fτ=0.85 but 0% tool calls over 520 RL steps) with Qwen3-VL-8B (strong prior: 14.4% tool usage but fτ=0.13 collapses fast), and PARA-GRPO recovers fτ to 0.41 on the latter, isolating the prior as the shared driver of both failures.

We introduce ParaVT, the first multi-agent end-to-end RL-trained framework for Parallel Video Tool calling: a main agent emits multiple temporal-window crops in a single turn, dispatches them to weight-sharing sub-agents, and aggregates the parallel evidence into a final answer. Applying standard GRPO to ParaVT surfaces two coupled failures driven by the same pretrained tool prior — Format Fragility (the SFT-learned structural tags collapse under temperature sampling) and the Tool Necessity Gap (the skip-tool reward shortcut). We name this trade-off the Tool Prior Paradox and tame it with PARA-GRPO (Parseability-Anchored and Ratio-gAted GRPO): a targeted format reward applied only at the structural-token positions most prone to collapse, paired with a per-prompt frame-budget randomization that lets calling the tool earn measurable RL credit. Across seven evaluation splits spanning six long-video benchmarks plus a temporal-grounding split, ParaVT sets a new open-source 7-8B SOTA on six of the seven, improving over the Qwen3-VL-8B base by +7.9% on average, with PARA-GRPO lifting training-time format compliance from 0.13 to 0.64.

01 / Method

Parallel hierarchical agent + PARA-GRPO

A main agent emits one or more <tool_call> blocks in a single turn; each call is handled by an independent sub-agent that shares weights with the main agent and returns a short summary; the main agent aggregates the summaries into a final answer. PARA-GRPO targets the structural-format collapse and the skip-tool reward shortcut that vanilla GRPO surfaces on a tool-native LMM.

Architecture

Sequential (left) vs. Parallel (right) tool dispatch. ParaVT replaces the one-tool-per-turn pattern with single-turn parallel dispatch: the main agent emits all required crop_video calls in one turn, each is processed by an independent sub-agent, and a gather-and-reason pass produces the final answer. Visual-token density inside one turn stays bounded as the number of crops grows.

Rfmt(o) = Σt∈struct(o) log πθ(ot | o<t)

Exploration Anchoring. Two cooperating mechanisms repair the structural-boundary collapse without restricting reasoning content. (i) Constrained Generation pins a Think Prefix (<think>) on every response and credits the presence of a final <answer> block via an Answer Suffix term, ruling out blind direct answers and tool-only rollouts. (ii) Selective Anchoring applies the targeted format reward only at the structural-token positions struct(o) most prone to collapse, leaving tool-call content tokens unconstrained.

K ∼ Uniform({4, 8, 16, 32, 64})

nFrames Gating. Randomize the per-prompt overview frame budget K so that, for a controllable fraction of GRPO groups, calling the tool earns measurable credit over skipping on prompts where the overview alone is insufficient. The mixture of budgets in a batch keeps the cross-rollout advantage of tool use non-degenerate, creating the gradient signal that the Tool Necessity Gap otherwise eliminates. Exploration Anchoring must take effect first so the gating gradient is credited to parseable tool-using rollouts.

02 / Results

SOTA results across six long-video benchmarks

All open-source rows were re-evaluated under a unified protocol (image_url channel, 64 frames, per-baseline native prompt). Best result in bold; underlined marks ParaVT's value when it is not the best column-wise. * withheld due to benchmark–training-data overlap. † native tool-call schema not reconcilable with Charades-STA grounding output.

Model VideoMME
w/o sub VideoMME
w/ sub LongVideo-
Bench LVBench MLVU MMVU Charades-STA
test (mIoU)

Proprietary LMMs — best-setting numbers from official reports
GPT-4o	71.9	77.2	66.7	34.7	64.6	66.7	—
Gemini 1.5 Pro	75.0	81.3	64.4	33.1	74.3	65.8	—
Open Instruct LMMs — direct answer
Qwen2.5-VL-7B	55.7	64.5	46.4	32.2	47.8	65.4	31.6
Open Reasoning LMMs — <think> → <answer>
Video-R1-7B	57.6	66.0	57.4	36.9	61.6	61.3	25.4
VideoChat-R1-7B	50.4	58.2	49.2	23.8	58.7	65.0	31.5
VideoRFT-7B	58.5	65.6	55.1	38.0	44.9	42.7	18.7
Time-R1-7B	58.9	66.2	56.0	38.2	60.5	63.4	34.7
ReWatch-R1-7B	58.8	65.0	53.6	38.5	60.1	59.8	20.2
Video-Thinker-7B	61.9	65.3	56.0	*	65.2	64.5	29.0
Open Agentic LMMs — <think> → <tool_call> → <answer>
Qwen3-VL-8B	59.9	68.4	52.2	33.1	58.3	68.0	49.3
Conan-7B	55.5	62.8	54.5	38.2	59.2	64.0	25.4
LongVT-RFT-7B	59.5	66.0	54.7	37.9	59.4	63.4	23.4
SAGE-7B	44.1	52.4	37.4	31.8	49.7	55.7	28.9
VideoZoomer-7B	45.3	48.3	39.6	22.9	46.2	61.6	†
ParaVT-8B (Ours)	62.1	69.4	60.4	39.8	65.0	68.6	50.1

Higher is better. ParaVT tops 6 of 7 columns and is within 0.2 pt of the best on MLVU. Avg over the seven columns: 59.3 for ParaVT vs. 55.6 for the Qwen3-VL-8B base (+7.9% relative).

03 / Ablation

PARA-GRPO restores format compliance while incentivizing tool exploration

Each row reports mean training-time format reward fτ at sampling temperature τ=0.7 and mean training-time tool-call rate per rollout κ. Curves on the right plot the same four runs step-by-step. Best result in bold; rows shaded grey mark the full PARA-GRPO recipe.

Setting fτκ VideoMME
w/o sub VideoMME
w/ sub LV-
Bench MLVU

(A) Training Stage
Qwen3-VL-8B	0.03	0.45	59.9	68.4	33.1	58.3
+ SFT Cold-Start	0.13	2.50	60.7	69.0	39.1	63.7
+ SFT + GRPO	0.13	0.02	62.0	68.6	39.3	64.5
+ SFT + PARA-GRPO	0.41	0.21	62.1	69.4	39.8	65.0
(B) Component Effectiveness
SFT + GRPO	0.13	0.02	62.0	68.6	39.3	64.5
+ Exploration Anchoring	0.35	0.19	61.7	68.7	39.3	64.1
+ nFrames Gating	0.10	1.36	61.3	68.7	39.1	63.6
Full PARA-GRPO	0.41	0.21	62.1	69.4	39.8	65.0
− Tool Reward Rtool	0.33	0.04	61.9	68.5	38.7	64.3
− Penalty Term γ	0.36	0.27	61.6	69.0	38.8	64.5
(C) Dispatch Mode
Sequential Tool Calling	—	—	61.4	68.8	37.5	64.1
Parallel Tool Calling	—	—	62.1	69.4	39.8	65.0

Training dynamics

Vanilla GRPO vs. PARA-GRPO across RL steps. Vanilla GRPO's format reward collapses within 20 steps; PARA-GRPO lifts it above 0.6 and holds it through the run while keeping the tool-call rate non-degenerate. Reasoning-content tokens stay free to evolve under RL.

Reading the table. Block (A) shows that SFT cold-start mostly transfers tool-call format (κ: 0.45 → 2.50) but not format reward (fτ=0.13), and that GRPO on top of cold-start collapses κ to 0.02. PARA-GRPO recovers fτ to 0.41 and κ to 0.21, and posts the best score on every benchmark column. Block (B) confirms neither component alone suffices: Exploration Anchoring on its own holds format up but suppresses tool calls; nFrames Gating on its own keeps tools active but tanks format. Only their composition wins on both axes. Block (C) shows the same trained checkpoint dispatched in parallel beats sequential calling by +0.7 to +2.3 across benchmarks, with no extra training.

Citation

If you find this project helpful, please consider citing our paper

@misc{yang2026paravt, title={{ParaVT}: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning}, author={Zuhao Yang and Kaichen Zhang and Sudong Wang and Keming Wu and Zhongyu Yang and Bo Li and Xiaojuan Qi and Shijian Lu and Xingxuan Li and Lidong Bing}, year={2026}, eprint={2605.20342}, archivePrefix={arXiv}, primaryClass={cs.CV} }