Zerui Chen1 Rolandos Alexandros Potamias2 Shizhe Chen1 Cordelia Schmid1
1WILLOW, INRIA Paris, France
2Imperial College London, UK
This is the implementation of HORT, an state-of-the-art hand-held object reconstruction algorithm:
git clone https://github.com/zerchen/hort.git
cd hort
The code has been tested with PyTorch 2.4.1 and CUDA 12.1. It is suggested to use an anaconda encironment to install the the required dependencies:
conda create --name hort python=3.12
conda activate hort
conda install pytorch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 pytorch-cuda=12.1 -c pytorch -c nvidia
# Install requirements
pip install -r requirements.txt
pip install -U git+https://github.com/luca-medeiros/lang-segment-anything.git
conda install pytorch3d-0.7.8-py312_cu121_pyt241.tar.bz2 # https://anaconda.org/pytorch3d/pytorch3d/files?page=2
cd /home/zerchen/workspace/code/hort_init/hort/models/tgs/models/snowflake/pointnet2_ops_lib && python setup.py installDownload the pretrained models using:
wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/detector.pt -P ./pretrained_models/
wget https://huggingface.co/spaces/rolpotamias/WiLoR/resolve/main/pretrained_models/wilor_final.ckpt -P ./pretrained_models/
wget https://huggingface.co/zerchen/hort_models/resolve/main/hort_final.pth.tar -P ./pretrained_models/It is also required to download MANO model from MANO website.
Create an account by clicking Sign Up and download the models (mano_v*_*.zip). Unzip and place the right hand model MANO_RIGHT.pkl under the mano_data/mano/ folder.
Note that MANO model falls under the MANO license.
python demo.py --img_folder demo_img
python vis_ho.py -e out_demo/test1.json # visualize the result in open3dYou can start a local demo for inference by running:
python gradio_demo.pyParts of the code are based on WiLoR, SnowflakeNet and Lang-SAM.
HORT is licensed under MIT License. This repository also depends on WiLoR, Ultralytics library and MANO Model, which are fall under their own licenses.
If you find HORT useful for your research, please consider citing our paper:
@article{chen2025hort,
title={{HORT}: Monocular Hand-held Objects Reconstruction with Transformers},
author={Chen, Zerui and Potamias, Rolandos Alexandros and Chen, Shizhe and Schmid, Cordelia},
journal={arXiv preprint arXiv:2503.21313},
year={2025}
}