代码改变世界

gearman管理

2014-12-04 14:36  youxin  阅读(1651)  评论(0)  编辑  收藏  举报

通常，Gearman被用来分发任务，以便实现异步操作。下面捋捋如何管理Gearman。

说明：请自行安装好Gearman和PHP PECL Gearman。

（我之前安装的gearman php的c语言扩展，gearman-monitor和gearman-manager都需要

Net_Gearman(也就是 

A pure PHP API that can be found as Net_Gearman on PEAR.

）

准备

我们先来创建一个Worker，实现一个简单的显示功能：

<?php
$worker= new GearmanWorker();
$worker->addServer('127.0.0.1', '4730');
$worker->addFunction('echo', 'my_echo_function');
while ($worker->work());
function my_echo_function($job) {
    return $job->workload();
}
?>

然后我们运行它：

shell> php /path/to/worker/file

可能你已经注意到，代码里有一个死循环，是不是需要Sleep一下？让我们监测看看：

shell> strace -r php /path/to/worker/file
0.000099 poll([{fd=3, events=POLLIN}], 1, 10000) = 0 (Timeout)
10.006522 write(3, "\0REQ\0\0\0\t\0\0\0\0", 12) = 12

可见PHP PECL Gearman内部已经做了休息十秒的设置，我们就不用杞人忧天了。

接下来我们以Shell为Client来调用一下：

shell> gearman -f echo "hello, world."

到这里，准备工作基本就齐活儿了，相信大家已经对Gearman有了一个初步的认识。

管理

出于效率的考虑，我们往往会启动很多个Worker，但具体应该启动多少个呢？十个还是一百个？少了不够，多了浪费，到底应该如何度量呢？

其实Gearman本身已经提供了相应的命令供我们查看状态：

shell> (echo status; sleep 0.1) | nc 127.0.0.1 4730

命令的结果会分为四列，它们的含义从左到右依次是：

  1. Function name: A string denoting the name of the function of the job
  2. Number in queue: A positive integer indicating the total number of jobs for this function in the queue. This includes currently running ones as well (next column)
  3. Number of jobs running: A positive integer showing how many jobs of this function are currently running
  4. Number of capable workers: A positive integer denoting the maximum possible count of workers that could be doing this job. Though they may not all be working on it due to other tasks holding them busy.

从这些信息可以推断出：如果系统比较繁忙的话，Number of jobs running的数值会接近Number of capable workers；Number in queue可能会大于Number of capable workers。此时我们应该增加Worker的数量，反之则应该考虑减少Worker的数量。

另外，推荐大家结合使用watch命令来监控Gearman的状态，这样更直观一些：

shell> watch -n 1 "(echo status; sleep 0.1) | nc 127.0.0.1 4730"

实际应用中，还有很多特殊情况需要考虑，比如说：程序代码更新后，如何避免手动重启Worker？还需要注意的是Worker长时间运行，一旦意外中断或者内存泄漏怎么办？通常这类进程控制问题用Supervisor都可以轻松搞定，有兴趣的读者自己看看吧。此外网络上还有一些不错的工具可以玩玩，比如：GearmanManager，Gearman-Monitor。

需要小心的一件事情是数据的共享。Gearman 不进行所交换数据的任何转换或操作。对于这里使用的简单字符串和整数没有问题，但是不能共享 PHP 中的数组值并期望能在 Java 语言中被理解。对于这种类型的交互，可以使用很多结构化数据标准中的一种，比如 JavaScript Object Notation (JSON) 或 XML。另外，如果您在处理来自数据库的信息，只要共享 ID 或者找到需要处理的数据时要用到的信息即可，或者使用 memcached 这样的透明方法（尽管可能仍然需要 JSON 或等价物）。

 转自：http://huoding.com/2012/10/30/196

http://blog.chinaunix.net/uid-20357359-id-1963682.html

 

GearmanMonitor 和 GearmanManager

GearmanMonitor 是用来查看 Gearman 服务状态的工具，包括 运行中/过的队列 Queue，运行中的所有 workers，及服务器 servers。GearmanMonitor 需要有 Net_Gearman 支持。安装 pear 后，直接执行：

我们首先安装NET_geraman（也就是php写的gearman库）。

https://github.com/brianlmoon/net_gearman

Installation

   1. Install PEAR if it is not already installed on your system.
   2. Use git to pull this repo to your servers. Be sure where you install it is in the autoload or include path before the rest of PEAR.

 Linux上安装php的pear 

在搭建centreon的过程中，需要pear模块支持。

什么是pear

pear是PHP扩展与应用库（the PHP Extension and Application Repository）的缩写。它是一个PHP扩展及应用的一个代码仓库，简单地说，pear就是PHP的cpan。

在官网上有说明详细的安装信息，这里作简单说明。

http://pear.php.net/manual/en/about-pear.php

我的PHP目录为/usr/local/php5

在Linux下安装PHP的PEAR:
1)下载
#curl -o go-pear.php  http://pear.php.net/go-pear

如果提示：

PHP Warning:  PHP Startup: Invalid library (maybe not a PHP library) ‘json.so’  in Unknown on line 0
Sorry!  Your PHP version is too new (5.2.9) for this go-pear.
Instead use http://pear.php.net/go-pear.phar for a more stable and current
version of go-pear, more suited to your PHP version.

那么要从http://pear.php.net/go-pear.phar获取。

#curl -o go-pear.php http://pear.php.net/go-pear.phar

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100 3594k  100 3594k    0     0   186k      0  0:00:19  0:00:19 –:–:–  196k

会在当前目录下载go-pear.php 页面。
2)运行go-pear.php
# /usr/local/php5/bin/php go-pear.php

3)这里按回车继续安装，CTRL+C放弃安装。

Below is a suggested file layout for your new PEAR installation.  To
change individual locations, type the number in front of the
directory.  Type ‘all’ to change all of them or simply press Enter to
accept these locations.

1. Installation base ($prefix)                   : /usr/local/php5
2. Temporary directory for processing            : /tmp/pear/install
3. Temporary directory for downloads             : /tmp/pear/install
4. Binaries directory                            : /usr/local/php5/bin
5. PHP code directory ($php_dir)                 : /usr/local/php5/lib/php
6. Documentation directory                       : /usr/local/php5/docs
7. Data directory                                : /usr/local/php5/data
8. User-modifiable configuration files directory : /usr/local/php5/cfg
9. Public Web Files directory                    : /usr/local/php5/www
10. Tests directory                               : /usr/local/php5/tests
11. Name of configuration file                    : /usr/local/php5/etc/pear.conf

1-11, ‘all’ or Enter to continue:
Beginning install…
Configuration written to /usr/local/php5/etc/pear.conf…
Initialized registry…
Preparing to install…
installing phar://go-pear.phar/PEAR/go-pear-tarballs/Archive_Tar-1.3.7.tar…
installing phar://go-pear.phar/PEAR/go-pear-tarballs/Console_Getopt-1.3.0.tar…
installing phar://go-pear.phar/PEAR/go-pear-tarballs/PEAR-1.9.4.tar…
installing phar://go-pear.phar/PEAR/go-pear-tarballs/Structures_Graph-1.0.4.tar…
installing phar://go-pear.phar/PEAR/go-pear-tarballs/XML_Util-1.2.1.tar…
install ok: channel://pear.php.net/Archive_Tar-1.3.7
install ok: channel://pear.php.net/Console_Getopt-1.3.0
install ok: channel://pear.php.net/Structures_Graph-1.0.4
install ok: channel://pear.php.net/XML_Util-1.2.1
install ok: channel://pear.php.net/PEAR-1.9.4
PEAR: Optional feature webinstaller available (PEAR’s web-based installer)
PEAR: Optional feature gtkinstaller available (PEAR’s PHP-GTK-based installer)
PEAR: Optional feature gtk2installer available (PEAR’s PHP-GTK2-based installer)
PEAR: To install optional features use “pear install pear/PEAR#featurename”

** WARNING! Old version found at /usr/local/php5/bin, please remove it or be sure to use the new /usr/local/php5/bin/pear command

The ‘pear’ command is now at your service at /usr/local/php5/bin/pear

** The ‘pear’ command is not currently in your PATH, so you need to
** use ‘/usr/local/php5/bin/pear’ until you have added
** ‘/usr/local/php5/bin’ to your PATH environment variable.

Run it without parameters to see the available actions, try ‘pear list’
to see what packages are installed, or ‘pear help’ for help.

For more information about PEAR, see:

http://pear.php.net/faq.php

http://pear.php.net/manual/

安装完毕

 

安装 pear 后，直接执行：

1
2
3
4
5
[root@www bin]# pear install Net_Gearman
# 如果提示
# No releases available for package "pear.php.net/Net_Gearman"
# 则执行
[root@www bin]# pear install  channel://pear.php.net/Net_Gearman-0.2.3
 

Net_gearman安装完后就可以用gearmanMonitor了

GearmanMonitor 

下载后放到web目录下就可以访问了。

修改GearmanMonitor目录下的_config.php配置文件

<?php
/**
 * Gearman Monitor configuration file
 *
 * The following server fields are available:
 *  - address: Gearman server address, hostname and port
 *  - name: Gearman server name to display in Gearman Monitor interface
 *
 * Example:
 * $cfgServers[$i]['address'] = '192.168.0.10:4730';
 * $cfgServers[$i]['name'] = 'Gearman server 1';
 * ++ $i;
 *
 * $cfgServers[$i]['address'] = '192.168.1.1:7003';
 * $cfgServers[$i]['name'] = 'Gearman server 2';
 * ++ $i;
 */
$i = 0;
$cfgServers = array();
$cfgServers[$i]['address'] = '';
$cfgServers[$i]['name'] = '';
++ $i;

address改成127.0.0.1:4730

GearmanManager

GearmanManager 用来统一管理用 PHP 编写的 Gearman workers。需要 PHP 启用 pcntl。用 install/install.sh 安装完成后，根据需要和设置，修改 /etc/init.d/gearman-manager:

##PATH##
DAEMON=/usr/local/bin/gearman-manager
PIDDIR=/tmp
PIDFILE=${PIDDIR}/manager.pid
LOGFILE=/tmp/gearman-manager.log
CONFIGDIR=/data/gearman-manager
GEARMANUSER="gearmand"
PARAMS="-c ${CONFIGDIR}/config.ini"

GearmanManager 安装时，选择的是 PECL library，启动时可能会遇见如下的问题：

1
2
3
4
5
[root@www gearman-manager]# /etc/init.d/gearman-manager start
Starting gearman-manager: [  OK  ]
[root@www  gearman-manager]# php: libgearman/universal.cc:553: bool gearman_request_option(gearman_universal_st&, gearman_string_t&): Assertion `con->recv_state == GEARMAN_CON_RECV_UNIVERSAL_NONE' failed.
php: libgearman/universal.cc:553: bool gearman_request_option(gearman_universal_st&, gearman_string_t&): Assertion `con->recv_state == GEARMAN_CON_RECV_UNIVERSAL_NONE' failed.
.....

解决办法如 Bug #60764 Enabling Non-Blocking Mode causes asseration 这里所提， 修改 pecl-manager.php ：

1
2
//注释下面这句
//$thisWorker->addOptions(GEARMAN_WORKER_NON_BLOCKING);

 

参考：http://www.livingelsewhere.net/tag/gearmanmanager/

http://my.oschina.net/vincentwy/blog/137862