An automated framework for generating diverse, physically plausible manipulation tasks to enhance Vision-Language-Action model pre-training
The pursuit of general-purpose robotic manipulation is hindered by the scarcity of diverse, real-world interaction data. Unlike data collection from the web in vision or language, robotic data collection is an active process incurring prohibitive physical costs.
Existing manual methods are unscalable and biased toward common tasks. To address this, we introduce RoboGene, an agentic framework designed to automate the generation of diverse, physically plausible manipulation tasks across single-arm, dual-arm, and mobile robots.
RoboGene integrates three core components: diversity-driven sampling, self-reflection mechanisms, and human-in-the-loop refinement. We conduct datasets of 18k trajectories and introduce novel metrics to assess task quality and diversity.
Implements LFU strategy to explore task space comprehensively, ensuring coverage of edge cases often overlooked by manual curation.
a self-reflection mechanism that improves task-generation quality from three complementary perspectives: constraint adherence, task novelty, and embodied plausibility.
Continuous refinement through human feedback, enabling the system to adapt to specific robot embodiments.
Evaluated across multiple robot platforms including single-arm, dual-arm manipulators, and mobile manipulators.
Automated task generation across multiple embodiments including mobile manipulators, single-arm, and dual-arm systems.
Water Plant
Organize Desk
Assembling Crucible
Replenish Shelves
Sort Buttons
Pour Water
Handover Tape
Open Medical Box
Rinse Lettuce
Quantitative assessment of pre-training efficacy and real-world execution demonstrations.
Grill Skewers
Weigh beakers
Build Blocks
Lubricate Gears
Novel Objects
Illumination Changes
Static Distractors
Background Variations
Instruction Change:
“Hang the cups on the holder” → “Insert the cups”
In-Distribution
