Incremental Learning for Robot Shared Autonomy
Yiran Tao, Guixiu Qiao, Ding Dan, Zackory Erickson
in Submission
ILSA is an Incrementally Learned Shared Autonomy framework that continually improves its assistive control policy through repeated user interactions. ILSA leverages synthetic kinematic trajectories for initial pretraining, reducing the need for expert demonstrations, and then incrementally finetunes its policy after each manipulation interaction, with mechanisms to balance new knowledge acquisition with existing knowledge retention during incremental learning.
- Setup
- Generate Synthetic Kinematic Trajectories
- Pretrain the Action Generation Model
- Run ILSA
- Citation
Clone this git repo.
git clone https://github.com/yirantao1000/ilsa.git
We recommend working with a conda environment.
conda env create -f environment.yaml
conda activate ilsa
If installing from this yaml file doesn't work, manual installation of missing packages should also work.
Part of the code is based on Robomimic. Install robomimic following instructions.
Object localization of the code is based on SAM with text prompt implemented by Luca Medeiros. Set up following instructions.
Fill in the codes the robot initial position, x and y ranges of the robot workspace, and position ranges of the experiment objects. Then run:
python generate_traj_cereal.py #cereal puring task
python generate_traj_pill.py #pill bottle storage task
python normalize.py --folder data/pill/raw --target_folder data/pill/normalized
Paste the printed min_proprios, max_proprios, min_actions, and max_actions values in the find_min_maxfunction in robomimic/ilsa.py, run_ILSA.py, and generate_corrected_traj.py.
python to_hdf5.py --data_name #cereal or pill
This part is partially based on Robomimic. Move the scripts below to the corresponding directory within your robomimic installation path:
mv robomimic/train_ilsa.py /path/to/robomimic/directory/robomimic/scripts
mv robomimic/ilsa.py /path/to/robomimic/directory/robomimic/algo
mv robomimic/ilsa_config.py /path/to/robomimic/directory/robomimic/config
mv robomimic/obs_nets.py /path/to/robomimic/directory/robomimic/models
mv robomimic/policy_nets.py /path/to/robomimic/directory/robomimic/models
Enter your robomimic installation path:
cd /path/to/robomimic/directory/
Run
python robomimic/scripts/train_ilsa.py --name cereal --config /path/to/ilsa/directory/configs/cereal.json --dataset /path/to/ilsa/directory/data/cereal/kinematics.hdf5 --output_dir /path/to/ilsa/directory/exp_results
python robomimic/scripts/train_ilsa.py --name pill --config /path/to/ilsa/directory/configs/pill.json --dataset /path/to/ilsa/directory/data/pill/kinematics.hdf5 --output_dir /path/to/ilsa/directory/exp_results
Paste the ckpt_path into run_ILSA.py, and previous_ckpt_path in incre_pill.json or incre_milk.json.
Repeat the steps below:
python run_ILSA.py
python generate_corrected_traj.py
cd /path/to/robomimic/directory/
python robomimic/scripts/train_ILSA.py --name cereal_incre --config /path/to/ilsa/directory/configs/incre_cereal.json --dataset /path/to/ilsa/directory/data/cereal/incremental/0/modified_0-0+50init.hdf5 --output_dir /path/to/ilsa/directory/exp_results
After finetuning, modify previous_ckpt_path in incre_pill.json or incre_milk.json to prepare for the next finetuning.
- Part of the codes is based on Robomimic
- Object localization of the code is based on SAM with text prompt implemented by Luca Medeiros.
If you find this codebase/paper useful for your research, please consider citing:
@misc{tao2024incrementallearningrobotshared,
title={Incremental Learning for Robot Shared Autonomy},
author={Yiran Tao and Guixiu Qiao and Dan Ding and Zackory Erickson},
year={2024},
eprint={2410.06315},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2410.06315},
}