mesos是什么呢?
一个分布式调度框架,让你编写代码时面对整个集群像面对一台机器那么简单。所有的运行,资源调度都可以由它来帮你搞掂。
1.mesos安装有两种方式:
1)参考官网的getstart,下载源码然后进行编译
http://mesos.apache.org/gettingstarted/
这个就不多说,下载源码,configue、make、make install就好了。
2)使用mesosphere编译好的包,直接安装
https://mesosphere.com/downloads/ 点mesos的get statred
mesosphere造福人类,ubuntu安装如下:
# Setup deb source sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]') CODENAME=$(lsb_release -cs) # Add the repository echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | \ sudo tee /etc/apt/sources.list.d/mesosphere.list sudo apt-get -y update #install sudo apt-get -y install mesos
so easy!
2.部署运行
1)单机运行模式
#如果是一键安装 #通过下面命令,就可以开启本地模式 mesos-local #代码安装则在build目录下面脚本启动 ./bin/mesos-local.sh
2) 集群运行模式
#源码编译模式,进入build
#必须加上 --ip ,否则都是127.0.0.1 ,这样在集群里面交换信息的时候,就会出现不能识别(都以为是本机的端口)
# Start mesos master.
$ ./bin/mesos-master.sh --ip=127.0.0.1 --work_dir=/var/lib/mesos
# Start mesos slave.
$ ./bin/mesos-slave.sh --master=127.0.0.1:5050
#命令行更多的参数参考官方文档 http://mesos.apache.org/documentation/latest/configuration/
#可以使用自带的mesos-daemon.sh启动,但是运行信息全部都会写入null中去
#例如:
mesos-daemon.sh mesos-master --work_dir=/var/lib/mesos --ip=10.230.136.197
mesos-daemon.sh mesos-slave --master=10.230.136.197:5050 --ip=10.29.23.28
#也可以自己编写脚本,把运行信息写入到指定文件,便于查看
nohup mesos-slave --master=10.230.136.197:5050 --ip=10.29.23.28 > mesos.info 2>&1
$./bin/mesos-master.sh --work_dir=/var/lib/mesos --ip=10.230.136.197
I1105 04:57:33.349346 7152 main.cpp:229] Build: 2015-09-19 00:50:33 by qingpingzhang
I1105 04:57:33.349531 7152 main.cpp:231] Version: 0.24.0
I1105 04:57:33.349786 7152 main.cpp:252] Using 'HierarchicalDRF' allocator
I1105 04:57:33.359891 7152 leveldb.cpp:176] Opened db in 9.846621ms
I1105 04:57:33.368098 7152 leveldb.cpp:183] Compacted db in 8.143958ms
I1105 04:57:33.368237 7152 leveldb.cpp:198] Created db iterator in 66033ns
I1105 04:57:33.368304 7152 leveldb.cpp:204] Seeked to beginning of db in 23717ns I1105 04:57:33.368751 7152 leveldb.cpp:273] Iterated through 3 keys in the db in 411999ns I1105 04:57:33.368988 7152 replica.cpp:744] Replica recovered with log positions 8001 -> 8002 with 0 holes and 0 unlearned I1105 04:57:33.371697 7173 recover.cpp:449] Starting replica recovery I1105 04:57:33.372709 7152 main.cpp:465] Starting Mesos master I1105 04:57:33.373004 7173 recover.cpp:475] Replica is in VOTING status I1105 04:57:33.373239 7173 recover.cpp:464] Recover process terminated I1105 04:57:33.377918 7169 master.cpp:378] Master 20151105-045733-3314083338-5050-7152 (ip-10-230-136-197.ec2.internal) started on 10.230.136.197:5050 I1105 04:57:33.377959 7169 master.cpp:380] Flags at startup: --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate="false" --authenticate_slaves="false" --authenticators="crammd5" --authorizers="local" --framework_sorter="drf" --help="false" --initia lize_driver_logging="true" --ip="10.230.136.197" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_slave_ping_timeouts="5" --port="5050" --quiet="false" --recovery_slave_removal_limit="100%" --registry="replicated_log" --registry_fetch_timeout="1m ins" --registry_store_timeout="5secs" --registry_strict="false" --root_submissions="true" --slave_ping_timeout="15secs" --slave_reregister_timeout="10mins" --user_sorter="drf" --version="false" --webui_dir="/home/qingpingzhang/dev/mesos-0.24.0/build/../src/webui" --work_d ir="/var/lib/mesos" --zk_session_timeout="10secs" I1105 04:57:33.378235 7169 master.cpp:427] Master allowing unauthenticated frameworks to register I1105 04:57:33.378258 7169 master.cpp:432] Master allowing unauthenticated slaves to register I1105 04:57:33.378309 7169 master.cpp:469] Using default 'crammd5' authenticator W1105 04:57:33.378353 7169 authenticator.cpp:505] No credentials provided, authentication requests will be refused. I1105 04:57:33.378376 7169 authenticator.cpp:512] Initializing server SASL I1105 04:57:33.390660 7172 master.cpp:1525] The newly elected leader is master@10.230.136.197:5050 with id 20151105-045733-3314083338-5050-7152 I1105 04:57:33.390744 7172 master.cpp:1538] Elected as the leading master! I1105 04:57:33.390810 7172 master.cpp:1308] Recovering from registrar I1105 04:57:33.391114 7174 registrar.cpp:311] Recovering registrar I1105 04:57:33.392326 7173 log.cpp:661] Attempting to start the writer I1105 04:57:33.394917 7169 replica.cpp:477] Replica received implicit promise request with proposal 6 I1105 04:57:33.397905 7169 leveldb.cpp:306] Persisting metadata (8 bytes) to leveldb took 2.948589ms I1105 04:57:33.397948 7169 replica.cpp:345] Persisted promised to 6 I1105 04:57:33.398849 7174 coordinator.cpp:231] Coordinator attemping to fill missing position I1105 04:57:33.399190 7173 log.cpp:677] Writer started with ending position 8002 I1105 04:57:33.401041 7170 leveldb.cpp:438] Reading position from leveldb took 76286ns I1105 04:57:33.401178 7170 leveldb.cpp:438] Reading position from leveldb took 28917ns I1105 04:57:33.403611 7172 registrar.cpp:344] Successfully fetched the registry (485B) in 12.319232ms I1105 04:57:33.404095 7172 registrar.cpp:443] Applied 1 operations in 136495ns; attempting to update the 'registry' I1105 04:57:33.406857 7168 log.cpp:685] Attempting to append 557 bytes to the log I1105 04:57:33.407032 7174 coordinator.cpp:341] Coordinator attempting to write APPEND action at position 8003 I1105 04:57:33.408102 7167 replica.cpp:511] Replica received write request for position 8003 I1105 04:57:33.410161 7167 leveldb.cpp:343] Persisting action (577 bytes) to leveldb took 1.985457ms I1105 04:57:33.410202 7167 replica.cpp:679] Persisted action at 8003 I1105 04:57:33.410912 7170 replica.cpp:658] Replica received learned notice for position 8003 I1105 04:57:33.412888 7170 leveldb.cpp:343] Persisting action (579 bytes) to leveldb took 1.931473ms I1105 04:57:33.412925 7170 replica.cpp:679] Persisted action at 8003 I1105 04:57:33.412955 7170 replica.cpp:664] Replica learned APPEND action at position 8003 I1105 04:57:33.414744 7167 registrar.cpp:488] Successfully updated the 'registry' in 10.496768ms I1105 04:57:33.415020 7167 registrar.cpp:374] Successfully recovered registrar I1105 04:57:33.415087 7171 log.cpp:704] Attempting to truncate the log to 8003 I1105 04:57:33.415216 7172 coordinator.cpp:341] Coordinator attempting to write TRUNCATE action at position 8004 I1105 04:57:33.415881 7168 replica.cpp:511] Replica received write request for position 8004 I1105 04:57:33.416004 7171 master.cpp:1335] Recovered 2 slaves from the Registry (518B) ; allowing 10mins for slaves to re-register I1105 04:57:33.417834 7168 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 1.903245ms I1105 04:57:33.417870 7168 replica.cpp:679] Persisted action at 8004 ./bin/mesos-master.sh --work_dir=/var/lib/mesos --ip=10.230.136.197 I1105 04:57:33.349346 7152 main.cpp:229] Build: 2015-09-19 00:50:33 by qingpingzhang I1105 04:57:33.349531 7152 main.cpp:231] Version: 0.24.0 I1105 04:57:33.349786 7152 main.cpp:252] Using 'HierarchicalDRF' allocator I1105 04:57:33.359891 7152 leveldb.cpp:176] Opened db in 9.846621ms I1105 04:57:33.368098 7152 leveldb.cpp:183] Compacted db in 8.143958ms I1105 04:57:33.368237 7152 leveldb.cpp:198] Created db iterator in 66033ns I1105 04:57:33.418318 7169 replica.cpp:658] Replica received learned notice for position 8004 I1105 04:57:33.419096 7169 leveldb.cpp:343] Persisting action (20 bytes) to leveldb took 608618ns I1105 04:57:33.419170 7169 leveldb.cpp:401] Deleting ~2 keys from leveldb took 41579ns I1105 04:57:33.419198 7169 replica.cpp:679] Persisted action at 8004 I1105 04:57:33.419229 7169 replica.cpp:664] Replica learned TRUNCATE action at position 8004 I1105 04:57:41.857591 7173 master.cpp:3636] Registering slave at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with id 20151105-045733-3314083338-5050-7152-S0 I1105 04:57:41.858408 7171 registrar.cpp:443] Applied 1 operations in 111553ns; attempting to update the 'registry' I1105 04:57:41.860332 7167 log.cpp:685] Attempting to append 737 bytes to the log I1105 04:57:41.860429 7174 coordinator.cpp:341] Coordinator attempting to write APPEND action at position 8005 I1105 04:57:41.861102 7171 replica.cpp:511] Replica received write request for position 8005 I1105 04:57:41.861977 7171 leveldb.cpp:343] Persisting action (757 bytes) to leveldb took 772697ns I1105 04:57:41.862018 7171 replica.cpp:679] Persisted action at 8005 I1105 04:57:41.862601 7168 replica.cpp:658] Replica received learned notice for position 8005 I1105 04:57:41.864603 7168 leveldb.cpp:343] Persisting action (759 bytes) to leveldb took 1.946092ms I1105 04:57:41.864646 7168 replica.cpp:679] Persisted action at 8005 I1105 04:57:41.864678 7168 replica.cpp:664] Replica learned APPEND action at position 8005 I1105 04:57:41.866068 7171 registrar.cpp:488] Successfully updated the 'registry' in 7.5648ms I1105 04:57:41.866261 7168 log.cpp:704] Attempting to truncate the log to 8005 I1105 04:57:41.866374 7173 coordinator.cpp:341] Coordinator attempting to write TRUNCATE action at position 8006 I1105 04:57:41.866920 7169 replica.cpp:511] Replica received write request for position 8006 I1105 04:57:41.868661 7174 master.cpp:3699] Registered slave 20151105-045733-3314083338-5050-7152-S0 at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000] I1105 04:57:41.868976 7169 leveldb.cpp:343] Persisting action (18 bytes) to leveldb took 1.996354ms I1105 04:57:41.869005 7169 replica.cpp:679] Persisted action at 8006 I1105 04:57:41.869000 7170 hierarchical.hpp:540] Added slave 20151105-045733-3314083338-5050-7152-S0 (ip-10-29-23-28.ec2.internal) with cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000] (allocated: ) I1105 04:57:41.869566 7174 replica.cpp:658] Replica received learned notice for position 8006 I1105 04:57:41.870249 7174 leveldb.cpp:343] Persisting action (20 bytes) to leveldb took 633721ns I1105 04:57:41.870314 7174 leveldb.cpp:401] Deleting ~2 keys from leveldb took 32741ns I1105 04:57:41.870342 7174 replica.cpp:679] Persisted action at 8006 I1105 04:57:41.870370 7174 replica.cpp:664] Replica learned TRUNCATE action at position 8006 I1105 04:57:41.870602 7167 master.cpp:3998] Received update of slave 20151105-045733-3314083338-5050-7152-S0 at slave(1)@10.29.23.28:5051 (ip-10-29-23-28.ec2.internal) with total oversubscribed resources I1105 04:57:41.871006 7167 hierarchical.hpp:600] Slave 20151105-045733-3314083338-5050-7152-S0 (ip-10-29-23-28.ec2.internal) updated with oversubscribed resources (total: cpus(*):8; mem(*):5986; disk(*):196338; ports(*):[31000-32000], allocated: )
可以看到slave注册的时候(红色部分)把机器的全部资源都注册上了,可以通过选项来限制资源:
--resources="mem:8096;cpus:2"
框架跑起来比较容易,关健是如何使用起来。