报错：Tez Compression codec com.hadoop.compression.lzo.LzoCodec not found.

报错背景

安装完成Tez之后，

在hive-shell下执行：select count(*) from student;

报错。

报错现象

22/03/03 22:34:48 INFO client.DAGClientImpl: DAG: State: FAILED Progress: 0% TotalTasks: 1 Succeeded: 0 Running: 0 Failed: 0 Killed: 0
22/03/03 22:34:48 INFO client.DAGClientImpl: DAG completed. FinalState=FAILED
22/03/03 22:34:48 INFO examples.OrderedWordCount: DAG diagnostics: [Vertex failed, vertexName=Tokenizer, vertexId=vertex_1646363809353_0002_1_00, diagnostics=[Vertex vertex_1646363809353_0002_1_00 [Tokenizer] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: Input initializer failed, vertex=vertex_1646363809353_0002_1_00 [Tokenizer], java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
        at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
        at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
        at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:58)
        at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:399)
        at org.apache.hadoop.mapreduce.split.TezGroupedSplitsInputFormat.getSplits(TezGroupedSplitsInputFormat.java:97)
        at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateNewSplits(MRInputHelpers.java:448)
        at org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:329)
        at org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
        at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2255)
        at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
        ... 18 more
], Vertex killed, vertexName=Summation, vertexId=vertex_1646363809353_0002_1_01, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1646363809353_0002_1_01 [Summation] killed/failed due to:OTHER_VERTEX_FAILURE], Vertex killed, vertexName=Sorter, vertexId=vertex_1646363809353_0002_1_02, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1646363809353_0002_1_02 [Sorter] killed/failed due to:OTHER_VERTEX_FAILURE], DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:2]

报错原因

找不到Lzo相关依赖。

报错解决

Lzo在安装的时候会将其依赖放入到hadoop中，只需要将hadoop的依赖加载到tez即可。

修改tez-site.xml，添加如下配置：

    <property>
         <name>tez.use.cluster.hadoop-libs</name>
         <value>true</value>
    </property>