ELK安装(ubuntu)

一.安装jdk8

经过我测试logstash5.x不支持java10和11，所以安装java8

 

加入LinuxUprising Java PPA

sudo add-apt-repository ppa:webupd8team/java

sudo apt-get update

 

安装java8(Long Term Support)

sudo apt-get install oracle-java8-installer

查看java是否安装成功

java -version

 

设置java11为默认

sudo apt-get install oracle-java8-set-default

 

二.新建用户

因为安全问题elasticsearch 不让用root用户直接运行，创建新用户

adduser elk 输入两次密码

passwd elk 输入两次密码

root给elk赋权限，chown -R elk /opt/elasticsearch-5.6.11

 

三.安装elastcisearch

卸载elastcisearch

sudo systemctl stop elasticsearch

sudo apt-get --purge autoremove elasticsearch

rm -rf /var/lib/elasticsearch

rm -rf /etc/elasticsearch

 

安装

由于目前IK Analysis插件5.x的最高版本为5.6.11，所以elk我们也安装5.6.11

wget -P /opt/downloads https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.11.tar.gz

sudo tar zxvf /opt/downloads/elasticsearch-5.6.11.tar.gz -C /opt

 

进入配置文件

sudo vim /opt/elasticsearch-5.6.11/config/elasticsearch.yml

修改network.host的值

network.host: 192.168.x.x 自身ip，限制为内网访问

network.host: 0.0.0.0 也可以不限制ip

http.port: 9200

 

删除lock

rm -rf /opt/elasticsearch-5.6.11/data/nodes/0/node.lock

切换到elk用户，守护进程启动es

su - elk -c "/opt/elasticsearch-5.6.11/bin/elasticsearch -d"

 

若启动不成功，日志文件报max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

sudo sysctl -w vm.max_map_count=262144

上述方法只是临时修改，若要永久修改

在/etc/sysctl.conf文件最后添加一行

vm.max_map_count=262144

查看是否修改成功

more /proc/sys/vm/max_map_count

再启动es即可

 

测试es是否正常运行

curl -X GET http://127.0.0.1:9200

 

关闭代码为

jps|grep Elasticsearch

显示 xxxx Elasticsearch

kill -9 xxxx

netstat -anltp|grep 9200

显示LISTEN xxxx/java

kill -9 xxxx

 

 

 

四、安装Logstash

卸载deb版本的logstash

sudo systemctl stop logstash

dpkg -P logstash或sudo apt-get --purge autoremove logstash

rm -rf /usr/share/logstash

rm -rf /var/lib/logstash

rm -rf /var/log/logstash

 

安装

wget -P /opt/downloads https://artifacts.elastic.co/downloads/logstash/logstash-5.6.11.tar.gz

sudo tar zxvf /opt/downloads/logstash-5.6.11.tar.gz -C /opt

错误提示：Unrecognized VM option 'UseParNewGC' 

解决办法：把/opt/logstash/config/jvm.options配置文件中的-XX:UseParNewGC注释掉

错误提示：could not create the java virtual machine或找不到主类

解决办法：把/opt/logstash/bin/logstash.bat中%CLASSPATH%没有加双引号的都加上双引号。

 

修改数据文件夹读写权限：

sudo chmod 777 /opt/logstash-5.6.11/data

 

测试是否成功启动

/opt/logstash-5.6.11/bin/logstash -e 'input{stdin{}}output{stdout{codec=>rubydebug}}'

 

修改配置

略

 

关闭代码为

netstat -anltp|grep 9600

显示LISTEN xxxx/java

kill -9 xxxx

 

五、安装Kibana

卸载deb版本的kibana

sudo systemctl stop kibana

dpkg -P kibana或sudo apt-get --purge autoremove kibana

rm -rf /var/lib/kibana

 

安装

wget -P /opt/downloads https://artifacts.elastic.co/downloads/kibana/kibana-5.6.11-linux-x86_64.tar.gz

sudo tar zxvf /opt/downloads/kibana-5.6.11-linux-x86_64.tar.gz -C /opt

 

修改配置

vim /opt/kibana-5.6.11-linux-x86_64/config/kibana.yml

#配置本机ip

server.host: "0.0.0.0"

#配置es集群url

elasticsearch.url: "http://localhost:9200"

 

后台启动kibana

nohup /opt/kibana-5.6.11-linux-x86_64/bin/kibana &

然后回车返回到shell

 

关闭

netstat -anltp|grep 5601

显示LISTEN xxxx/java

kill -9 xxxx

 

六、安装Elasticsearch-head

cd /opt

git clone git://github.com/mobz/elasticsearch-head.git

 

修改配置

vim /opt/elasticsearch-head/_site/app.js

this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "http://localhost:9200";修改为当前es地址，默认es在本机localhost:9200不用修改

vim /opt/elasticsearch-head/Gruntfile.js

在原有的connect:server:options节点上加上hostname: '*',

 

修改es配置

sudo vim /opt/elasticsearch-5.6.11/config/elasticsearch.yml

network.host: 0.0.0.0

http.cors.enabled: true

http.cors.allow-origin: "*"

 

安装node

sudo apt-get install nodejs

安装npm

sudo apt-get install npm

安装grunt和grunt-cli

sudo npm install -g grunt

sudo npm install -g grunt-cli

安装模块依赖

sudo npm install

 

后台启动head

cd /opt/elasticsearch-head

nohup npm run start &

回车返回到shell

 

客户端浏览器打开x.x.x.x:9100

若显示未连接修改http://localhost:9200为es服务器ip:port

 

关闭

netstat -anltp|grep 9100

显示LISTEN xxxx/grunt

kill -9 xxxx

 

 

七、安装IK Analysis插件

/opt/elasticsearch-5.6.11/bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.6.11/elasticsearch-analysis-ik-5.6.11.zip

 

重启服务

jps|grep Elasticsearch

显示 xxxx Elasticsearch

kill -9 xxxx (xxxx为)

su - elk -c "/opt/elasticsearch-5.6.11/bin/elasticsearch -d"

 

 

八、建立守护进程(systemd)

elasticsearch

vim /lib/systemd/system/elasticsearch.service

 

[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target

[Service]
Environment=ES_HOME=/opt/elasticsearch-5.6.11
Environment=CONF_DIR=/opt/elasticsearch-5.6.11/config
Environment=DATA_DIR=/opt/elasticsearch-5.6.11/data
Environment=LOG_DIR=/opt/elasticsearch-5.6.11/logs
Environment=PID_DIR=/opt/elasticsearch-5.6.11
#EnvironmentFile=-/etc/default/elasticsearch-5.6.11

WorkingDirectory=/opt/elasticsearch-5.6.11

User=elk
Group=elk

ExecStartPre=/opt/elasticsearch-5.6.11/bin/elasticsearch-systemd-pre-exec

ExecStart=/opt/elasticsearch-5.6.11/bin/elasticsearch \
-p ${PID_DIR}/elasticsearch.pid \
--quiet \
-Edefault.path.logs=${LOG_DIR} \
-Edefault.path.data=${DATA_DIR} \
-Edefault.path.conf=${CONF_DIR}

# StandardOutput is configured to redirect to journalctl since
# some error messages may be logged in standard output before
# elasticsearch logging system is initialized. Elasticsearch
# stores its logs in /opt/elasticsearch-5.6.11/logs and does not use
# journalctl by default. If you also want to enable journalctl
# logging, you can simply remove the "quiet" option from ExecStart.
StandardOutput=journal
StandardError=inherit

# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65536

# Specifies the maximum number of processes
LimitNPROC=2048

# Specifies the maximum size of virtual memory
LimitAS=infinity

# Specifies the maximum file size
LimitFSIZE=infinity

# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0

# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM

# Send the signal only to the JVM rather than its control group
KillMode=process

# Java process is never killed
SendSIGKILL=no

# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

 

systemctl enable elasticsearch.service

 

systemctl daemon-reload

 

 

 elasticsearch-head

vim /lib/systemd/system/elasticsearch-head.service

 

[Unit]
Description=elasticsearch-head

[Service]
Type=simple
User=root
ExecStart=/opt/systemd-sh/elasticsearch-head.sh start
#ExecReload=/opt/systemd-sh/elasticsearch-head.sh restart
#ExecStop=/opt/systemd-sh/elasticsearch-head.sh stop
#ExecReload=/bin/kill -s HUP $MAINPID
#ExecStop=/bin/kill -s QUIT $MAINPID
Restart=always
PrivateTmp=true

[Install]
WantedBy=multi-user.target

 

vim /opt/systemd-sh/elasticsearch-head.sh

 

systemctl daemon-reload

 

#!/bin/bash
function exist_port()
{
  port=`netstat -lntp | grep $1 | awk '{print $4}' | cut -d \: -f 2`
  if [ "$port" = "$1" ];then
    return 1
  else
    return 0
  fi
}

function kill_by_port()
{
  port=`netstat -lntp | grep $1 | awk '{print $7}' | cut -d \/ -f 1`
  if [ "$port" != "" ];then
    sudo kill -9 $port
    echo "kill $port"
  fi
}

function run()
{if [ "$1" = "start" ];then  
    start
  elif [ "$1" = "stop" ];then
    stop
  elif [ "$1" = "restart" ];then
    stop
    start
  else
    echo "please input param, start, restart or stop"
  fi
}

function start()
{
  exist_port 9100
  if [ $? = 0 ];then
    echo 'start elasticsearch-head'
    cd /opt/elasticsearch-head
    sudo npm run start
  fi
}

function stop()
{
  kill_by_port 9100
}

run $*

 

chmod +x /opt/systemd-sh/elasticsearch-head.sh

 

systemctl enable elasticsearch-head.service

 

 

logstash

vim /lib/systemd/system/logstash.service

 

[Unit]
Description=logstash

[Service]
Type=simple
User=root
Group=root
# Load env vars from /etc/default/ and /etc/sysconfig/ if they exist.
# Prefixing the path with '-' makes it try to load, but if the file doesn't
# exist, it continues onward.
#EnvironmentFile=-/etc/default/logstash
#EnvironmentFile=-/etc/sysconfig/logstash

Environment=LS_HOME=/opt/logstash-5.6.11
Environment=LS_SETTINGS_DIR="/opt/logstash-5.6.11/config"
Environment=LS_OPTS="--path.settings /opt/logstash-5.6.11/config"
#Environment=LS_JAVA_OPTS=""
Environment=LS_PIDFILE=/opt/logstash-5.6.11/logstash.pid
Environment=LS_USER=root
Environment=LS_GROUP=root
Environment=LS_GC_LOG_FILE=/opt/logstash-5.6.11/logs/gc.log
Environment=LS_OPEN_FILES=16384
Environment=LS_NICE=19
Environment=SERVICE_NAME="logstash"
Environment=SERVICE_DESCRIPTION="logstash"

ExecStart=/opt/logstash-5.6.11/bin/logstash -f /opt/logstash-5.6.11/config/logstash.conf
Restart=always
WorkingDirectory=/
Nice=19
LimitNOFILE=16384

[Install]
WantedBy=multi-user.target

 

vim /opt/logstash-5.6.11/config/logstash.conf

自行添加conf文件，注这里写用标准输入会超时。

 

systemctl enable logstash.service

 

systemctl daemon-reload

 

 

kibana

vim /lib/systemd/system/kibana.service

 

[Unit]
Description=kibana

[Service]
Type=simple
User=root
ExecStart=/opt/kibana-5.6.11-linux-x86_64/bin/kibana "-c /opt/kibana-5.6.11-linux-x86_64/config/kibana.yml"
Restart=always
PrivateTmp=true
WorkingDirectory=/

[Install]
WantedBy=multi-user.target

 

systemctl enable kibana.service

 

systemctl daemon-reload

 

 

 

 

reboot

重启后看是否自启成功