You must have a CUDA-enabled NVIDIA video card. The code is tested on Ubuntu 20.04 and CUDA 11.8. You may need at least 40 GB of GPU memory and 64 GB of RAM to run the code.
The repository is highly dependent on nerfstudio and gsplat.
# clone the repository
git clone https://github.com/OpenDriveLab/MTGS.git
cd MTGS/
# create a conda environment
conda create --name mtgs -y python=3.9
conda activate mtgs
# tiny-cuda-nn requires >gcc 9.0
conda install -c conda-forge gxx=9.5.0
# if you do not have cuda 11.8 locally, install it with the following command
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit
# install torch and torchvision
pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118
# install the requirements
pip install -r requirements.txtIf you want to process the nuplan data by yourself, you need to first install the nuplan-devkit with version 1.2.0 and colmap with version 3.11.1.
Then, you need to install the customized version of kiss-icp and UniDepth in this repository.
# Install the requirements for nuplan data processing
pip install -r requirements_data.txt
# Install kiss-icp.
# this is a customized version with some changes, based on kiss-icp 1.0.0
# tested on cmake 3.29.6 and gcc 9.5.0
cd thirdparty/kiss-icp
make editable
# Install UniDepth.
# The requirements are aligned with MTGS.
pip install -e thirdparty/UniDepth
# Download the pretrained models for UniDepth and Mask2Former
mkdir -p ckpts/huggingface
huggingface-cli download --local-dir ckpts/huggingface/facebook/mask2former-swin-large-cityscapes-semantic facebook/mask2former-swin-large-cityscapes-semantic
huggingface-cli download --local-dir ckpts/huggingface/lpiccinelli/unidepth-v2-vitl14 lpiccinelli/unidepth-v2-vitl14 --revision 1d0d3c52f60b5164629d279bb9a7546458e6dcc4