kafka on CentOS-7

准备安装环境，下载软件包

groupadd kafka
useradd  -M -s /bin/false -g kafka kafka

mkdir -p /kafka/{src,conf,data-logs}
chown kafka:kafka -R  /kafka
chmod 750 -R /kafka

wget  -P /kafka/src  http://mirrors.tuna.tsinghua.edu.cn/apache/kafka/2.1.0/kafka_2.11-2.1.0.tgz

tar zxf /kafka/src/kafka_2.11-2.1.0.tgz 

ln -s /kafka/src/kafka_2.11-2.1.0  /opt/kafka

在/kafka/conf目录下创建kaka配置文件server.properties，修改相应参数：

cp /opt/kafka/config/server.properties  /kafka/conf/

• Standalone模式：

log.dirs=/kafka/data-logs
delete.topic.enable=true

• Clustered模式：

broker.id=0
listeners=PLAINTEXT://kafka-1:9092
log.dirs=/kafka/data-logs
delete.topic.enable=true
zookeeper.connect=zk1:2181,zk2:2181,zk3:2181
zookeeper.connection.timeout.ms=6000

编写systemd.service服务脚本/etc/systemd/system/kafka.service：

[Unit]
Description=Apache Kafka server (broker)
After=network.target
After=syslog.target
#kafka和zookeeper装在一台机器上时需要这个配置
After=zookeeper.target 

[Service]
Type=simple
User=root
Group=root
ExecStart=/opt/kafka/bin/kafka-server-start.sh /kafka/conf/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh

ExecReload=/bin/kill -HUP $MAINPID
KillMode=none

Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

重新加载配置信息：systemctl daemon-reload
启动kafka：systemctl start kafka.service
关掉kafka：systemctl stop kafka.service
查看进程状态及日志：systemctl status kafka.service
开机自启动：systemctl enable kafka.service
关闭自启动：systemctl disable kafka.service

kafka connect：

/opt/kafka/bin/connect-standalone.sh /opt/kafka/config/connect-standalone.properties /kafka/conf/connector1.properties

# tasks to create:
name=test-postgresql-jdbc-connect
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1

connection.url=jdbc:postgresql://127.0.0.1:5432/demo?user=postgres&password=&currentSchema=base
mode=incrementing
incrementing.column.name=seq_id
topic.prefix=test-