Authors: Yue Su*, Xinyu Zhan*, Hongjie Fang, Han Xue, Haoshu Fang, Yong-Lu Li, Cewu Lu, Lixin Yang†
This is the 3D version of Dense Policy, you can also refer 2D Dense Policy Code here.
You can also refer DSPv2, an effective and generalizable dense policy, can deploy for whole-body mobile manipulation.
Please following the installation guide to install the dsp conda environments and the dependencies, as well as the real robot environments. Also, remember to adjust the constant parameters in dataset/constants.py and utils/constants.py according to your own environment.
Please calibrate the camera(s) with the robot before data collection and evaluation to ensure correct spatial transformations between camera(s) and the robot. Please refer to calibration guide for more details.
You can view the sampled data (cut task) from this link, which contains task data (one trajectory for instance). You can ignore other files since they are for MBA We follow the data managemnet pattern as RH20T.
Task_name
`-- train/
|-- [episode identifier 1]
| |-- metadata.json # metadata
| |-- timestamp.txt # calib timestamp
| |-- cam_[serial_number 1]/
| | |-- color # RGB
| | | |-- [timestamp 1].png
| | | |-- [timestamp 2].png
| | | |-- ...
| | | `-- [timestamp T].png
| | |-- depth # depth
| | | |-- [timestamp 1].png
| | | |-- [timestamp 2].png
| | | |-- ...
| | | `-- [timestamp T].png
| | |-- tcp # tcp
| | | |-- [timestamp 1].npy
| | | |-- [timestamp 2].npy
| | | |-- ...
| | | `-- [timestamp T].npy
| | `-- gripper_command # gripper command
| | |-- [timestamp 1].npy
| | |-- [timestamp 2].npy
| | |-- ...
| | `-- [timestamp T].npy
| `-- cam_[serial_number 2]/ # similar camera structure
`-- [episode identifier 2] # similar episode structure
conda activate dsp
bash train.shPlease follow the deployment guide to modify the evaluation script.
Modify the arguments in eval.sh, then
conda activate dsp
bash eval.sh@InProceedings{Su_2025_ICCV,
author = {Su, Yue and Zhan, Xinyu and Fang, Hongjie and Xue, Han and Fang, Hao-Shu and Li, Yong-Lu and Lu, Cewu and Yang, Lixin},
title = {Dense Policy: Bidirectional Autoregressive Learning of Actions},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2025},
pages = {14486-14495}
}DSP is licensed under CC BY-NC-SA 4.0
