AllocNet: Learning Time Allocation for Trajectory Optimization

About

AllocNet is a lightweight learning-based trajectory optimization framework.

Authors: Yuwei Wu, Xiatao Sun, Igor Spasojevic, and Vijay Kumar from the Kumar Lab.

Video Links: Youtube

Related Paper: Y. Wu, X. Sun, I. Spasojevic and V. Kumar, "Deep Learning for Optimization of Trajectories for Quadrotors," in IEEE Robotics and Automation Letters, vol. 9, no. 3, pp. 2479-2486, March 2024 arxiv Preprint

If this repo helps your research, please cite our paper at:

@ARTICLE{10412114,
  author={Wu, Yuwei and Sun, Xiatao and Spasojevic, Igor and Kumar, Vijay},
  journal={IEEE Robotics and Automation Letters}, 
  title={Deep Learning for Optimization of Trajectories for Quadrotors}, 
  year={2024},
  volume={9},
  number={3},
  pages={2479-2486}}

Acknowledgements

• Dataset: The raw point cloud dataset from M3ED
• Front-end Path Planning: We use OMPL planning library
• Planning Modules and Visualization: We use the module in GCOPTER

Run our pre-trained Model in Simulation

The repo has been tested on 20.04 with ros-desktop-full installation.

1. Prerequisites

1.1 ROS and OMPL

Follow the guidance to install ROS and install OMPL:

sudo apt install libompl-dev

1.2 libtorch

Download the libtorch and put it into the "AllocNet/src/planner/libtorch/" folder: GPU version, or CPU version

1.3 QP solver

We use osqp to solve quadratic programming, install by:

git clone -b release-0.6.3 https://github.com/osqp/osqp.git
cd osqp
git submodule init
git submodule update
mkdir build & cd build
cmake ..
sudo make install

cd ../..
git clone https://github.com/robotology/osqp-eigen.git
cd osqp-eigen
mkdir build & cd build
cmake ..
sudo make install

2. Build on ROS

2.1 Build

git clone git@github.com:yuwei-wu/AllocNet.git && cd AllocNet/src
wstool init && wstool merge utils.rosinstall && wstool update
catkin build

2.2 Swith towards GPU and CPU

The default mode is set to the GPU version.

To switch to the CPU,

1. navigate to line 29 in the 'learning_planning.hpp' file and replace 'device(torch::kGPU)' with 'device(torch::kCPU)'. After making this change, recompile the code for the updates to take effect.
2. In "AllocNet/src/planner/launch/learning_planning.launch, line 63, change the model

   <param name="ModelPath" value="$(find planner)/models/seq5_tokenthresh0_35.pt"/>

to

   <param name="ModelPath" value="$(find planner)/models/seq5_tokenthresh0_35_cpu.pt"/>

You can also check: - Installing C++ Distributions of PyTorch

3. Run

source devel/setup.bash
roslaunch planner learning_planning.launch

click 2D Nav Goal to trigger planning:

Train new models

0. Folder Structure

network/
│
├── config/                - Configuration files for training and testing.
│
│
├── utils/                 - Utility functions and classes.
│   └── learning/          - Contains network classes and layers
│
└── train_minsnap_<...>.py - Scripts for training
└── test_minsnap_<...>.py  - Scripts for testing
└── ts_conversion_<...>.py - Scripts for converting to TorchScript

1. Pre-requisites

• Ubuntu 20.04 / Windows 11
  • If using WSL2 with simulation running in windows, please add export WSL_HOST_IP=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2}') to your .bashrc file to allow communication between Windows and the subsystem.
• Python 3.8
• CUDA 11.7

2. Setup

2.1 Install Dependencies

• Install Ubuntu packages
  • sudo apt-get install python3-dev python3-venv
• Create a virtual environment
  • python3 -m venv venv
• Activate the virtual environment
  • source venv/bin/activate
• Install the requirements
  • pip install wheel
  • pip install numpy==1.24.2
  • pip install -r requirements.txt

2.2 Setup Iris

Follow the instructions to install, and you may need to change the CMakeLists.txt in iris-distro/CMakeLists.txt

iris: https://github.com/rdeits/iris-distro

For AMD CPU, if you encounter a core dump, please refer to instructions in this link:

https://github.com/rdeits/iris-distro/issues/81

pip install -U kaleido

3. Run

• For training, please run python train_minsnap_<model_configuration>.py
• For testing, please run python test_minsnap_<model_configuration>.py
• For converting the learned model to TorchScript, please run python ts_conversion_<model_configuration>.py

Maintaince

For any technical issues, please contact Yuwei Wu (yuweiwu@seas.upenn.edu, yuweiwu20001@outlook.com).