edgedb 内部pg 数据存储的探索 (四) 源码编译

 

edgedb 基于python开发，同时源码重包含了好多子项目,以下进行简单的源码编译

clone 代码

需要递归处理，加上recursive，比较慢稍等

git clone --recursive https://github.com/edgedb/edgedb.git

效果

正克隆到 'edgedb'...
remote: Enumerating objects: 165, done.
remote: Counting objects: 100% (165/165), done.
remote: Compressing objects: 100% (129/129), done.
remote: Total 48698 (delta 68), reused 66 (delta 36), pack-reused 48533
接收对象中: 100% (48698/48698), 13.53 MiB | 767.00 KiB/s, done.
处理 delta 中: 100% (33738/33738), done.
子模组 'edb/server/pgproto' (https://github.com/MagicStack/py-pgproto.git) 已为路径 'edb/server/pgproto' 注册
子模组 'postgres' (https://github.com/edgedb/postgres.git) 已为路径 'postgres' 注册
正克隆到 'edb/server/pgproto'...
remote: Enumerating objects: 32, done.
remote: Counting objects: 100% (32/32), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 207 (delta 14), reused 22 (delta 9), pack-reused 175
接收对象中: 100% (207/207), 67.21 KiB | 0 bytes/s, done.
处理 delta 中: 100% (133/133), done.
子模组路径 'edb/server/pgproto'：检出 'aca615c71d7060555b629323a1aa6bb1af7b50f2'
正克隆到 'postgres'...
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
接收对象中: 4% (31265/714724), 11.86 MiB | 100.00 KiB/s
接收对象中: 4% (31588/714724), 11.96 MiB | 94.00 KiB/s

构建

• 环境准备

  使用venv 进行python3 环境的准备

python3 -m venv edgedb
source   edgedb/bin/active

• 构建

  比较慢，稍等

cd edgedb
make

• 效果

find edb -name '*.pyx' | xargs touch
python setup.py build_ext --inplace
warning: no files found matching '*.py' under directory 'examples'
warning: no files found matching '*.pem' under directory 'tests'
edgedb/pgproto/pgproto.c: 在函数‘__pyx_f_6edgedb_7pgproto_7pgproto_i64_to_hex’中:
edgedb/pgproto/pgproto.c:10446:28: 警告：‘char’转换自‘int’时可能改变值 [-Wconversion]
     __pyx_v_i = (__pyx_v_i - 1);
                            ^
edgedb/pgproto/pgproto.c: 在函数‘__pyx_f_6edgedb_7pgproto_7pgproto_pg_uuid_bytes_from_str’中:
edgedb/pgproto/pgproto.c:10744:55: 警告：‘uint8_t’转换自‘int’时可能改变值 [-Wconversion]
   for (__pyx_t_9 = 0; __pyx_t_9 < __pyx_t_8; __pyx_t_9+=1) {
                                                       ^
edgedb/pgproto/pgproto.c:10955:30: 警告：‘uint8_t’转换自‘int’时可能改变值 [-Wconversion]
       __pyx_v_j = (__pyx_v_j + 1);
                              ^

........

• install local

  稍等，有点慢，因为server 依赖pg 数据库。。。

pip install .
Looking in indexes: http://mirrors.cloud.aliyuncs.com/pypi/simple/
Processing /opt/edgedb
Collecting asyncpg~=0.18.2 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/25/eb/9674bb30b80403ee3110dcbd54d71080dfc71d0774760955353aaa5c1627/asyncpg-0.18.3-cp36-cp36m-manylinux1_x86_64.whl (2.4MB)
    100% |████████████████████████████████| 2.4MB 91.0MB/s 
Collecting click~=6.7 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl (71kB)
    100% |████████████████████████████████| 71kB 63.2MB/s 
Collecting httptools>=0.0.13 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz (104kB)
    100% |████████████████████████████████| 112kB 84.5MB/s 
Collecting immutables>=0.9 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/e3/91/bc4b34993ef77aabfd1546a657563576bdd437205fa24d4acaf232707452/immutables-0.9-cp36-cp36m-manylinux1_x86_64.whl (91kB)
    100% |████████████████████████████████| 92kB 1.3MB/s 
Collecting parsing~=1.6.1 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/fc/19/05dff0134f244265c7cee60f5075ce9236c69b28c8a9767fa5354d15d246/parsing-1.6.1-py3-none-any.whl
Collecting prompt_toolkit>=2.0.0 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/f7/a7/9b1dd14ef45345f186ef69d175bdd2491c40ab1dfa4b2b3e4352df719ed7/prompt_toolkit-2.0.9-py3-none-any.whl (337kB)
    100% |████████████████████████████████| 337kB 90.4MB/s 
Collecting psutil~=5.6.1 (from edgedb-server==1.0a2.dev29+g946bbdf)
  Downloading http://mirrors.cloud.aliyuncs.com/pypi/packages/2f/b8/11ec5006d2ec2998cb68349b8d1317c24c284cf918ecd6729739388e4c56/psutil-5.6.1.tar.gz (427kB)
    100% |████████████████████████████████| 

Installing collected packages: asyncpg, click, httptools, immutables, six, parsing, wcwidth, prompt-toolkit, psutil, Pygments, setproctitle, uvloop, promise, rx, graphql-core, edgedb, edgedb-server
  Running setup.py install for httptools ... done
  Running setup.py install for psutil ... done
  Running setup.py install for setproctitle ... done
  Running setup.py install for promise ... done
  Running setup.py install for edgedb ... done
  Running setup.py install for edgedb-server ... /

• 查看安装

pip list

效果

Package Version             
-------------- --------------------
asyncpg 0.18.3              
click 6.7                 
edgedb 0.6.1               
edgedb-server 1.0a2.dev29+g946bbdf
graphql-core 2.1                 
httptools 0.0.13              
immutables 0.9                 
parsing 1.6.1               
pip 10.0.1              
promise 2.2.1               
prompt-toolkit 2.0.9               
psutil 5.6.1               
Pygments 2.3.1               
Rx 1.6.1               
setproctitle 1.1.10              
setuptools 39.0.1              
setuptools-scm 3.2.0               
six 1.12.0              
uvloop 0.12.2              
wcwidth 0.1.7 

• 生成的console 命令

ls  -sial  bin

效果

140985 4 drwxr-xr-x 2 root root 4096 4月 23 17:13 .
134239 4 drwxr-xr-x 15 root root 4096 4月 23 17:06 ..
141201 4 -rw-r--r-- 1 root root 2191 4月 23 16:58 activate
141513 4 -rw-r--r-- 1 root root 1247 4月 23 16:58 activate.csh
141512 4 -rw-r--r-- 1 root root 2411 4月 23 16:58 activate.fish
141194 4 -rwxr-xr-x 1 root root 241 4月 23 16:58 easy_install
141195 4 -rwxr-xr-x 1 root root 241 4月 23 16:58 easy_install-3.6
142182 4 -rwxr-xr-x 1 root root 450 4月 23 17:13 edgedb
142183 4 -rwxr-xr-x 1 root root 464 4月 23 17:13 edgedb-server
141507 4 -rwxr-xr-x 1 root root 223 4月 23 16:58 pip
141508 4 -rwxr-xr-x 1 root root 223 4月 23 16:58 pip3
141509 4 -rwxr-xr-x 1 root root 223 4月 23 16:58 pip3.6
142018 4 -rwxr-xr-x 1 root root 226 4月 23 17:04 pygmentize
140988 0 lrwxrwxrwx 1 root root 7 4月 23 16:58 python -> python3
140987 0 lrwxrwxrwx 1 root root 16 4月 23 16:58 python3 -> /usr/bin/python3

• 运行

  直接是不可以运行的，需要配置参数,配置可以参考如下

[Unit]
Description=Next generation object-relational database
Documentation=https://edgedb.com/
After=syslog.target
After=network.target

[Service]
Type=notify

User=edgedb
Group=edgedb

Environment=EDGEDATA=/var/lib/edgedb/1-alpha1/data/
RuntimeDirectory=edgedb

ExecStart=/usr/lib64/edgedb-1-alpha1/bin/edgedb-server -D ${EDGEDATA} --runstate-dir=%t/edgedb
ExecReload=/bin/kill -HUP ${MAINPID}
KillMode=mixed
KillSignal=SIGINT
TimeoutSec=0

[Install]
WantedBy=multi-user.target

几个错误

• setuptools_scm 问题

distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('setuptools_scm~=3.2.0')

解决方法：

pip  install setuptools_scm~=3.2.0

说明

通过学习源码的编译我们可以进一步了解edgedb 的工作原理,对于启动可以进一步学习源码，了解启动的处理逻辑，目前直接运行edgedb-server
是有问题的，实际上因为后端是使用的pg 需要配置类似pg 的相关参数，实际我们可以解决systemd 的管理文件

参考资料

https://github.com/edgedb/edgedb