ceph 16.2.15(Pacific)编译
以下流程在ubuntu22.04 和 openEuler20.03 都实际操作过
获取ceph源码
从https://docs.ceph.com/en/latest/releases/找到pacific版本对应的tag为16.2.15
git clone https://github.com/ceph/ceph.git
# 查看所有tag
git ls-remote --tags origin |grep 16.2.
618f440892089921c3e944a991122ddc44e60516 refs/tags/v16.2.15^{}
# 根据git log的commit找到pacifc的release tag为v16.2.15
commit 618f440892089921c3e944a991122ddc44e60516 (HEAD -> ceph-pacific-learning, tag: v16.2.15, tag: ls, origin/ceph-pacific-learning)
Author: Ceph Release Team <ceph-maintainers@ceph.io>
Date: Mon Feb 26 19:21:09 2024 +0000
16.2.15
Signed-off-by: Ceph Release Team <ceph-maintainers@ceph.io>
# 或者 git show v16.2.15,可以确认即可
# 切换到pacific的release tag
git checkout v16.2.15
# 这个tag是一个快照,不可修改,所以创建我们自己的分支
git checkout -b ceph-pacific-learning
编译
拉取submodule网络问题
执行命令git submodule update --init --recursive --progress
更新子模块
如果子模块拉取失败:
- 方法一:这个地址会记录最新国内可用的github镜像地址,https://fcp7.com/github-mirror-daily-updates.html,修改ceph顶层目录下的.gitmodule,例如对于
[submodule "src/erasure-code/jerasure/gf-complete"]
,将url = https://github.com/ceph/gf-complete.git
部分替换为镜像的地址如下:
[submodule "ceph-object-corpus"]
path = ceph-object-corpus
url = https://gitclone.com/ceph/ceph-object-corpus.git
[submodule "src/civetweb"]
path = src/civetweb
url = https://gitclone.com/ceph/civetweb
[submodule "src/erasure-code/jerasure/jerasure"]
path = src/erasure-code/jerasure/jerasure
url = https://gitclone.com/ceph/jerasure.git
branch = v2-ceph
[submodule "src/erasure-code/jerasure/gf-complete"]
path = src/erasure-code/jerasure/gf-complete
url = https://gitclone.com/ceph/gf-complete.git
branch = v3-ceph
[submodule "src/rocksdb"]
path = src/rocksdb
url = https://gitclone.com/ceph/rocksdb
ignore = dirty
[submodule "ceph-erasure-code-corpus"]
path = ceph-erasure-code-corpus
url = https://gitclone.com/ceph/ceph-erasure-code-corpus.git
执行git submodule sync --recursive
更新git拉取的地址,再次执行git submodule update --init --recursive --progress
- 方法二:如果方法一中的镜像源没有地址,根据拉取失败的提示
fatal: clone of 'https://github.com/spdk/dpdk.git' into submodule path '/data/ceph-learning/src/spdk/dpdk' failed
,手动多拉取几次git clone https://github.com/spdk/dpdk.git /data/ceph-learning/src/spdk/dpdk --progress
,或者将https换成git执行git clone git://github.com/spdk/dpdk.git /data/ceph-learning/src/spdk/dpdk --progress
- 方法三:那种方式上网,解决99%的编译问题
安装依赖
./install-deps.sh
如果是openEuler系统
index 7e137d687f1..b22d45f409d 100755
--- a/install-deps.sh
+++ b/install-deps.sh
@@ -410,7 +410,7 @@ else
$SUDO env DEBIAN_FRONTEND=noninteractive apt-get -y remove ceph-build-deps
if [ "$control" != "debian/control" ] ; then rm $control; fi
;;
- centos|fedora|rhel|ol|virtuozzo)
+ centos|fedora|rhel|ol|virtuozzo|openEuler) # 修改这里
builddepcmd="dnf -y builddep --allowerasing"
echo "Using dnf to install dependencies"
case "$ID" in
do_cmake.sh
cmake -LH
可以看到cmake可用参数和描述信息,例如
// ceph-mgr is enabled
WITH_MGR:BOOL=ON
// Build the mgr/dashboard frontend using `npm`
WITH_MGR_DASHBOARD_FRONTEND:BOOL=ON
// build OCF-compliant cluster resource agent
WITH_OCF:BOOL=OFF
dashboard依赖node、ng的一堆包,系统版本不同依赖问题也不同,编译会很麻烦,如果用不到它可以关掉:
./do_cmake.sh -DWITH_MGR_DASHBOARD_FRONTEND=OFF
如果需要dashboard,直接执行./do_cmake.sh
后,到build目录下,按照下面的编译dashboard配置后,再回来执行make
编译
执行./do_cmake.sh
后会在ceph源码顶层目录下生成一个build目录
cd build
make -j16
Tips:make过程中也会下载一些软件包,没有那种方式上网可能会导致超时失败,失败后可以继续
make -j16
增量编译
vstart启动
要使用vstart启动ceph,还需要编译相关环境,在build目录下
make vstart
MON=1 OSD=2 MGR=1 RGW=1 NFS=1 ../src/vstart.sh -d -n -x
根据输出提示在shell中EXPORT
export PYTHONPATH=/data/ceph-learning/src/pybind:/data/ceph-learning/build/lib/cython_modules/lib.3:/data/ceph-learning/src/python-common:$PYTHONPATH
export LD_LIBRARY_PATH=/data/ceph-learning/build/lib:$LD_LIBRARY_PATH
export PATH=/data/ceph-learning/build/bin:$PATH
alias cephfs-shell=/data/ceph-learning/src/tools/cephfs/shell/cephfs-shell
CEPH_DEV=1
root@yielde:/data/ceph-learning/build# ceph -s
2025-02-09T18:23:21.560+0800 ffff73eae120 -1 WARNING: all dangerous and experimental features are enabled.
2025-02-09T18:23:21.568+0800 ffff73eae120 -1 WARNING: all dangerous and experimental features are enabled.
cluster:
id: 22b8a5ef-b731-43d6-9e5a-e3c3e77cad68
health: HEALTH_OK
services:
mon: 1 daemons, quorum a (age 3m)
mgr: x(active, since 2m)
mds: 1/1 daemons up, 2 standby
osd: 2 osds: 2 up (since 3m), 2 in (since 3m)
rgw: 1 daemon active (1 hosts, 1 zones)
data:
volumes: 1/1 healthy
pools: 8 pools, 321 pgs
objects: 229 objects, 20 KiB
usage: 2.0 GiB used, 200 GiB / 202 GiB avail
pgs: 321 active+clean
io:
client: 85 B/s rd, 0 op/s rd, 0 op/s wr
问题
- 无法访问dashboard,
ceph -s
输出Module 'dashboard' has failed: No module named 'routes'
,缺python包pip3 install routes
即可 - 启动cephfs可能会失败缺包,手动安装
Requirements
------------
The following packages are required to enable CephFS and RGW exports with nfs-ganesha:
- ``nfs-ganesha``, ``nfs-ganesha-ceph``, ``nfs-ganesha-rados-grace`` and
``nfs-ganesha-rados-urls`` packages (version 3.3 and above)
Ganesha Configuration Hierarchy
编译dashboard
如果使用-DWITH_MGR_DASHBOARD_FRONTEND=OFF关了dashboard的话可以忽略这个
安装nodejs
方法一 下载编译好的源码包
wget https://nodejs.org/dist/v16.14.2/node-v16.14.2-linux-x64.tar.xz
xz -d node-v16.14.2-linux-x64.tar.xz
tar -xvf node-v16.14.2-linux-x64.tar
mkdir /usr/local/soft
mv node-v16.14.2-linux-x64 /usr/local/soft
ln -s /usr/local/soft/node-v16.14.2-linux-x64/bin/node /usr/local/bin/node
ln -s /usr/local/soft/node-v16.14.2-linux-x64/bin/npm /usr/local/bin/npm
npm config set registry https://registry.npmmirror.com
npm install -g @angular/cli@16.2.0
ln -s /usr/local/soft/node-v16.14.2-linux-x64/lib/node_modules/@angular/cli/bin/ng.js /usr/local/bin/ng
方法二 nvm安装node(推荐)
# 复制脚本到主机
https://github.com/nvm-sh/nvm/blob/master/install.sh
bash install.sh
# 重新打开终端
nvm ls-remote --lts
nvm install v16.14.2
npm -g install @angular/cli@16.2.10
编译nodeenv
旧版本的ceph编译dashboard会有nodejs和angular/cli版本冲突的问题
# /data/ceph-learning换成自己的ceph源码顶层目录
cd /data/ceph-learning/src/pybind/mgr/dashboard && vim CMakeLists.txt
# 修改 COMMAND ${mgr-dashboard-nodeenv-dir}/bin/nodeenv --verbose ${node_mirror_opt} -p --node=12.18.2
# 为 COMMAND ${mgr-dashboard-nodeenv-dir}/bin/nodeenv --verbose ${node_mirror_opt} -p --node=16.14.2
cd /data/ceph-learning/build
root@yielde:/data/ceph-learning/build# make mgr-dashboard-nodeenv
Installing collected packages: nodeenv
Successfully installed nodeenv-1.9.1
* Install prebuilt node (16.14.2) ..... done.
* Appending data to /data/ceph-learning/build/src/pybind/mgr/dashboard/node-env/bin/activate
* Appending data to /data/ceph-learning/build/src/pybind/mgr/dashboard/node-env/bin/activate.fish
Built target mgr-dashboard-nodeenv
其他
boost下载慢
生成build后,可以先下载好 https://download.ceph.com/qa/boost_1_73_0.tar.bz2,放到/data/ceph-learning/build/boost/src目录下
root@yielde:/data/ceph-learning/build/boost/src# ls -lh
total 105M
drwxr-xr-x 2 root root 4.0K Feb 9 14:38 Boost
-rw-r--r-- 1 root root 105M Feb 9 15:27 boost_1_73_0.tar.bz2
drwxr-xr-x 2 root root 4.0K Feb 9 14:55 Boost-stamp
总结
其实棘手的问题就两个,一是网络方面众所周知的问题访问git和其他一些仓库会超时失败,二是前端相关的nodejs组件等版本问题导致编译失败。要耐心查看报错,修改CmakeList.txt或Makefile、手动下载相关软件包到目录,一步一步解决。
附一个大佬写的vscode+clangd的调试环境配置,我还没尝试:
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】凌霞软件回馈社区,博客园 & 1Panel & Halo 联合会员上线
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】博客园社区专享云产品让利特惠,阿里云新客6.5折上折
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步