TRAM尝鲜

https://github.com/yufu-wang/tram
因为大多数云计算平台都是用docker开容器的，docker内不能再装docker（安全权限受限），所以锁定在一个平台上做就行了。然后打包自己的镜像。

制作images

autodl算力云，租了个最便宜的2080Ti，0.88元/小时，先把环境配好（一般要5小时左右，我首次踩坑用了10小时）。
基础镜像选pytorch2.0_py3.8_cuda11.8，然后再创建一个py3.10的conda环境。

pip仓库速度：清华>阿里（也可以在AutoPanel > 实用工具里调镜像）
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

预执行

conda install -c conda-forge mamba
mamba create -n tram python=3.10 -y
mamba activate tram

source /etc/network_turbo
cd /
git clone --recursive https://github.com/yufu-wang/tram

install.sh

修改如下

set -x
# mamba remove -n tram --all -y
# mamba create -n tram python=3.10 -y
# mamba activate tram

# mamba install nvidia/label/cuda-11.8.0::cuda-toolkit -y # you can disable this if you already have cuda-11.8
mamba install pytorch==2.0.0 torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia -y
pip install 'git+https://github.com/facebookresearch/detectron2.git@a59f05630a8f205756064244bf5beb8661f96180'
pip install "git+https://github.com/facebookresearch/pytorch3d.git@stable"

mamba install -c pyg pytorch-scatter=2.1.2=py310_torch_2.0.0_cu118
mamba install -c conda-forge suitesparse

pip install pulp
pip install supervision

pip install open3d
pip install opencv-python
pip install loguru
pip install chumpy
pip install einops
pip install plyfile
pip install pyrender
pip install segment_anything
pip install scikit-image
pip install smplx
pip install timm==0.6.7
pip install evo
pip install pytorch-minimize
pip install imageio[ffmpeg]
pip install numpy==1.23
pip install gdown

编译 DROID-SLAM

cd thirdparty/DROID-SLAM
python setup.py install

手动下载gdown

由于云主机不方便设置proxy，可以用在本机上下好，再ssh上传。

pip install gdown
gdown --fuzzy -O ./droid.pth https://drive.google.com/file/d/1PpqVt1H4maBa_GbPJp4NwxRsd9jk-elh/view?usp=sharing
gdown --fuzzy -O ./vimo_checkpoint.pth.tar https://drive.google.com/file/d/1fdeUxn_hK4ERGFwuksFpV_-_PHZJuoiW/view?usp=share_link
gdown --fuzzy -O ./example_video.mov https://drive.google.com/file/d/1H6gyykajrk2JsBBxBIdt9Z49oKgYAuYJ/view?usp=share_link

测试

#!/bin/bash
trap "set +x && echo ❗ dont forgot: mamba activate tram" EXIT

vd="$(readlink -f $1)"
out_dir="$(basename $1 | sed 's/\.[^.]*$//')"
mkdir -p $out_dir
vd_dir="$(readlink -f $out_dir)"

cd $(dirname $0)
cmd1="ls $out_dir/tracks.npy"
cmd2="ls $out_dir/masked_droid_slam.npz"
cmd3="ls $out_dir/hps"
ln -s $vd_dir $out_dir || exit 3  #soft link

set -x
# 1. Run detection, segmentation and multi-person tracking
$cmd1 2>null || python scripts/detect_track_video.py --video "$vd" --visualization && \ 

# 2. Run Masked DROID-SLAM, estimate a focal length
[[ -n "$2" ]] && img_focal="--img_focal $2" && \
$cmd2 2>null || python scripts/slam_video.py --video "$vd" $img_focal && \

# 3. Run 4D human capture with VIMO.
$cmd3 2>null || python scripts/vimo_video.py --video "$vd" && \

# 4. Put everything together. Render the output video.
python scripts/tram_video.py --video "$vd" || (
  set +x
  echo
  echo "💡 Tip of known error"
  echo "❌ ValueError: bin_size too small, number of faces per bin must be less than 22;
Solution: set bin_size=-1 in /tram/scripts/tram_video.py
https://github.com/yufu-wang/tram/issues/5"
  echo
  echo "❌ ValueError: not enough values to unpack (expected 2, got 0)
Solution: https://github.com/yufu-wang/tram/issues/6"
)