llama-cpp-python web server cuda 编译安装简单说明

llama-cpp-python 推荐的玩法是自己编译，以下是关于cuda 支持编译的简单说明

参考构建命令

• 命令

export CUDACXX=/usr/local/cuda-12.5/bin/nvcc # 此处核心是指定了nvcc 编译器路径，同时安装过cuda-drivers , 还需要配置环境变量

export PATH=$PATH:/usr/local/cuda-12.5/bin/

CUDACXX=/usr/local/cuda-12.5/bin/nvcc CMAKE_ARGS="-DLLAMA_CUDA=on -DLLAMA_CUBLAS=on  -DLLAVA_BUILD=OFF -DCUDA_DOCKER_ARCH=compute_6"  make  GGML_CUDA=1

可能的问题

• 比如cuda 编译的DCUDA_DOCKER_ARCH变量
  核心就是配置

Makefile:950: *** I ERROR: For CUDA versions < 11.7 a target CUDA architecture must be explicitly provided via environment variable CUDA_DOCKER_ARCH, e.g. by running "export CUDA_DOCKER_ARCH=compute_XX" on Unix-like systems, where XX is the minimum compute capability that the code needs to run on. A list with compute capabilities can be found here: https://developer.nvidia.com/cuda-gpus .  Sto

说明

以上核心是关于cuda 编译参数的配置，如果有其他问题，可以参考以下链接

参考资料

https://github.com/ggerganov/llama.cpp/blob/master/docs/build.md
https://llmops-handbook.distantmagic.com/deployments/llama.cpp/aws-ec2-cuda.html
https://github.com/jetsonhacks/buildLibrealsense2TX/issues/13
https://stackoverflow.com/questions/72278881/no-cmake-cuda-compiler-could-be-found-when-installing-pytorch