jmxtrans监控kafka

我们知道jmx可以将程序内部的信息暴露出来，但是要想监控这些信息的话，就还需要自己写java程序调用jmx接口去获取数据，并按照某种格式发送到其他地方（如监控程序zabbix,ganglia）。这时jmxtrans就派上用场了，jmxtrans的作用是自动去jvm中获取所有jmx格式数据，并按照某种格式（json文件配置格式）输出到其他应用程序（常用的有ganglia）

安装

主页：https://github.com/jmxtrans/jmxtrans（这里面也有一个下载地址，貌似版本更高）
下载地址：https://github.com/jmxtrans/jmxtrans/downloads
sudo yum install jmxtrans-20121016.145842.6a28c97fbb-0.noarch.rpm
jmxtrans安装目录：/usr/share/jmxtrans
jmxtrans配置文件 ：/etc/sysconfig/jmxtrans
json文件默认目录：/var/lib/jmxtrans/
日志路径：/var/log/jmxtrans/jmxtrans.log

启动

sudo /usr/share/jmxtrans/jmxtrans.sh start
sudo /usr/share/jmxtrans/jmxtrans.sh start /var/lib/jmxtrans/kafka.json # 也可以指定json文件路径

停止

sudo /usr/share/jmxtrans/jmxtrans.sh stop

监控jvm

jvm需要加入以下参数

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=9999
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

监控kafka示例

配置kafka参数

第一步: 在kafka集群的所有机器上安装JMXTrans第二步: 编辑 “kafka-run-class.sh”
　　　　KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false "第三步: 编辑 “kafka-server-start.sh” 
　　　　export JMX_PORT=${JMX_PORT:-9999}    

 

监控kafka信息发送到ganglia,/var/lib/jmxtrans/kafka.json

{
    "servers": [
        {
            "port": "9999",  # jmx 端口
            "host": "127.0.0.1",   # jmx 的主机
            "queries": [
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "jvmheapmemory",
                                "port": 8649,
                                "host": "127.0.0.1"
                            }
                        }
                    ],
                    "obj": "java.lang:type=Memory",
                    "resultAlias": "heap",
                    "attr": [
                        "HeapMemoryUsage",
                        "NonHeapMemoryUsage"
                    ]
                },
        {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "kafka topic stats",
                                "port": 8649,   # ganglia的gmond端口
                                "host": "127.0.0.1",  # ganglia 主机
                "typeNames":[
                    "name"
                ]
                            }
                        }
                    ],
                    "obj": "kafka.server:type=BrokerTopicMetrics,name=*",
                    "resultAlias": "Kafka",
                    "attr": [
                        "Count",
                        "OneMinuteRate"
                    ]
                },
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "kafka server request",
                                "port": 8649,
                                "host": "127.0.0.1"
                            }
                        }
                    ],
                    "obj":"kafka.server:type=OffsetManager,name=NumGroups",
                    "resultAlias": "OffsetManager NumGroups",
                    "attr": [
                        "Value"
                    ]
                },
                {
                    "outputWriters": [
                        {
                            "@class": "com.googlecode.jmxtrans.model.output.GangliaWriter",
                            "settings": {
                                "groupName": "jvmGC",
                                "port": 8649,
                                "host": "127.0.0.1",
                                "typeNames": [
                                    "name"
                                ]
                            }
                        }
                    ],
                    "obj": "java.lang:type=GarbageCollector,name=*",
                    "resultAlias": "GC",
                    "attr": [
                        "CollectionCount",
                        "CollectionTime"
                    ]
                }
            ],
            "numQueryThreads": 2
        }
    ]
}