Flume 1.11 部署

Flume官网： https://flume.apache.org/
下载地址： https://archive.apache.org/dist/flume/
文档地址： https://flume.apache.org/FlumeUserGuide.html

1.安装部署

1.将二进制包上传到服务器上[略]

2.解压二进制包

mkdir /usr/flume
tar -zxvf apache-flume-1.11.0-bin.tar.gz -C /usr/flume/

3.将lib文件夹下的guava-11.0.2.jar删除以兼容Hadoop 3.1.3+(依据Hadoop版本决定)

rm /usr/flume/apache-flume-1.11.0-bin/lib/guava-11.0.2.jar

注意：删除 guava-11.0.2.jar 的服务器节点，一定要配置Hadoop的环境变量，否则会抛出 ClassNotFoundException 异常

4.修改配置调整Flume内存

mv /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh.template /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh
vi /usr/flume/apache-flume-1.11.0-bin/conf/flume-env.sh

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# If this file is placed at FLUME_CONF_DIR/flume-env.sh, it will be sourced
# during Flume startup.

# Enviroment variables can be set here.

# export JAVA_HOME=/usr/lib/jvm/java-8-oracle

# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
# export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote"

export JAVA_OPTS="-Xms2048m -Xmx2048m -Dcom.sun.management.jmxremote"

# Let Flume write raw event data and configuration information to its log files for debugging
# purposes. Enabling these flags is not recommended in production,
# as it may result in logging sensitive user information or encryption secrets.
# export JAVA_OPTS="$JAVA_OPTS -Dorg.apache.flume.log.rawdata=true -Dorg.apache.flume.log.printconfig=true "

# Note that the Flume conf directory is always included in the classpath.
#FLUME_CLASSPATH=""