Ant Group’s Robbyant has released LingBot-VLA 2.0, a Vision-Language-Action (VLA) foundation model for robots. The release includes a technical report, an Apache-2.0 codebase, and a 6B checkpoint. The research team targets a well-known gap: VLA models often work in labs but stumble in deployment. LingBot-VLA 2.0 advances the prior version along three practical axes. These are generalization, an expanded action space, and predictive dynamics modeling.
What is LingBot-VLA 2.0?
LingBot-VLA 2.0 is a generalist robot policy built on a vision-language backbone. It converts camera images and a language instruction into robot actions. The public model is lingbot-vla-v2-6b, a 6B ‘native depth’ checkpoint. It uses Qwen3-VL-4B-Instruct as the VLM backbone. Two teacher models, LingBot-Depth and DINO-Video, supervise training through distillation.
One inference call takes about 130 ms on an NVIDIA GeForce RTX 4090D. That measurement uses 10 denoising steps. The action expert uses a Mixture-of-Experts (MoE) design for scaling.
Data pipeline: 60,000 hours across 20 configurations
Generalization starts with data. The research team curates roughly 60,000 hours of pre-training data. This covers 50,000 hours of robot trajectories and 10,000 hours of egocentric human videos. The robot data spans 20 robot configurations, from single-arm rigs to full humanoids. The raw pool is larger: about 90,000 robot hours and 20,000 egocentric hours. A redesigned pipeline filters noisy samples down to the high-quality set.
Filtering is explicit and measurable. The research team computes third-order jerk along with velocity and acceleration Z-scores per embodiment. Episodes with abnormal smoothness or over 95% static signals are dropped. Videos are checked against replayed states using each robot’s URDF. Annotators remove blur, occlusion, dropped frames, and multi-view misalignment. Egocentric clips pass a VLM filter, then egocentric SLAM and MANO hand-pose reconstruction.
Annotation is automated with a vision-language model. Qwen3.6-27B segments each video into temporally contiguous subtasks. Each subtask gets an atomic action from a closed vocabulary of 18 categories. That vocabulary holds 15 primitive actions plus transit, idle, and other. Across the corpus, move and transit dominate by frequency.
Unified action representation
Different robots expose different joints, so LingBot-VLA 2.0 unifies them. It uses a 55-dimensional canonical vector for both states and actions. The layout is fixed across every embodiment in the dataset.
Each arm end-effector pose uses XYZ coordinates plus a rotation quaternion, giving 7 dimensions per arm. Robots that lack a body part simply pad the corresponding dimensions. This lets one model control arms, hands, grippers, waists, heads, and mobile bases.
MoE action expert
The action expert replaces its feed-forward network with sparse MoE layers. Each MoE layer keeps one shared expert along with several routed experts. Only the top-K routed experts activate per token, so active compute stays bounded. Each expert is a SwiGLU MLP with a smaller intermediate width.
Routing follows a sigmoid-based, auxiliary-loss-free strategy inspired by DeepSeek-V3. A per-expert bias corrects load imbalance without adding a load-balancing loss. Routing confidence still comes from the model’s original, unbiased affinity scores. Under matched active parameters, the MoE model reaches lower training loss than a dense baseline. It also reaches lower validation action error on GM-100 tasks.
Dual-query distillation for predictive dynamics
Real execution needs anticipation, not just reaction to the current frame. LingBot-VLA 2.0 appends two learnable queries to the visual and text tokens. Qt targets the current observation, and Qt+T targets a future observation. The horizon T equals the action chunk size.
Two teachers supervise these queries. LingBot-Depth supplies explicit geometric cues through depth prediction. DINO-Video supplies temporally grounded semantic priors. DINO-Video is built on the DINOv3 backbone with block-wise causal temporal attention and 3D-RoPE. It is trained on 5M video clips spanning internet, egocentric, and robotic data. On the LARYBench evaluation, DINO-Video leads on three of four metrics.
Benchmark results
Robbyant evaluates the model in a generalist setting on the GM-100 (Great March 100) bimanual benchmark. A single policy is jointly trained on nine tasks per embodiment. Results are reported as progress score / success rate.
On long-horizon mobile manipulation, the model is tested under two settings. In-domain (ID) uses the training distribution, while OOD perturbs pose and objects.
Gains are largest on tasks needing accurate object grounding. On Agilex Retrieve keychain, success moves from 60.0 to 100.0 versus version 1.0. Some tasks still show a gap between progress and success. That gap points to failures at the final precise placement or release step.
Getting started
The repository ships install, download, and deployment scripts. The example below downloads the released weights.
python3 scripts/download_hf_model.py –repo_id robbyant/lingbot-vla-v2-6b –local_dir lingbot-vla
Real-robot deployment runs the policy server with compiled inference.
python -m deploy.lingbot_vla_v2_policy \
–model_path path_to_posttraining_ckpt \
–use_compile \
–use_length 25 \
–port port
Post-training uses LeRobot v2.1 or v3.0 datasets. The provided example fine-tunes on RoboTwin 2.0 across 50 tasks. Routing can use sequence-wise auxiliary loss with z-loss, or a loss-free setup. The config also exposes the Muon optimizer, with AdamW as the default.
Use cases with examples
The expanded action space maps to concrete deployment scenarios.
- Kitchen mobile manipulation: Astribot S1 sorts fruits and drinks into a refrigerator. This needs base movement, door opening, and object placement together.
- Surface cleaning: Cobot Magic-ARX X5 wipes foam off a stove with a sponge. This chains grasping, wiping, and tool repositioning.
- Bimanual packing and sorting: GM-100 tasks include egg packing, tool packing, and block sorting.
- Dexterous-hand control: Unitree G1, Fourier GR-2, and AgiBot A2 use 12-DoF hands, not grippers.
Interactive Dynamic Explainer
Key Takeaways
- LingBot-VLA 2.0 is pretrained on ~60,000 hours of data — 50,000 hours of robot trajectories across 20 configurations plus 10,000 hours of egocentric human video.
- A single 55-dimensional canonical action space unifies arms, end-effectors, grippers, dexterous hands, waists, heads, and mobile bases, so one model covers whole-body control.
- The action expert uses a token-level auxiliary-loss-free MoE, which reaches lower training loss and validation action error than a dense model at matched active parameters.
- Dual-query distillation adds current/future queries supervised by LingBot-Depth (geometry) and DINO-Video (temporal dynamics) for future-aware manipulation.
- On the GM-100 generalist setting it scores 66.2/34.4 on AgileX Cobot Magic and 34.6/15.6 on Galaxea R1Pro, ahead of π0.5 and LingBot-VLA-1.0, and it beats π0.5 on both long-horizon mobile tasks in-domain and OOD.
Check out the Paper and Model Weights. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Note:Thanks to the Ant Research team for the thought leadership/ Resources for this article. Ant Research team has supported this content/article for promotion.
