需求：通过hadoop查询指定目录下创建日期为180天之前的目录，将其删除，通过crontab定期执行
编辑shell脚本文件，其中调用hadoop命令，本地sh执行没有问题，但是通过crontab调用时未执行，原因可能是环境变量未设置导致
先查询本地环境变量
echo $PATH
然后将环境变量内容写入shell脚本中
hadoop fs -ls查询指定目录下的所有目录名称
[hdp-test@hadoopclient01v ~]$ hadoop fs -ls /home/hdp-xxx/test1/data/ Found 15 items drwxr-xr-x - work hdp-test 0 2020-02-02 15:01 /home/hdp-test/test1/data/axx drwxr-xr-x - work hdp-test 0 2020-02-02 14:46 /home/hdp-test/test1/data/bxx drwxr-xr-x - work hdp-test 0 2020-02-02 14:46 /home/hdp-test/test1/data/cxx drwxr-xr-x - work hdp-test 0 2020-02-02 14:46 /home/hdp-test/test1/data/dxx drwxr-xr-x - work hdp-test 0 2020-02-02 17:26 /home/hdp-test/test1/data/exx drwxr-xr-x - work hdp-test 0 2020-02-02 17:26 /home/hdp-test/test1/data/flow drwxr-xr-x - work hdp-test 0 2020-02-02 14:46 /home/hdp-test/test1/data/ftp drwxr-xr-x - work hdp-test 0 2020-02-02 14:46 /home/hdp-test/test1/data/ldap drwxr-xr-x - work hdp-test 0 2020-02-02 14:47 /home/hdp-test/test1/data/login drwxr-xr-x - work hdp-test 0 2020-02-02 14:47 /home/hdp-test/test1/data/mail drwxr-xr-x - work hdp-test 0 2020-02-02 14:47 /home/hdp-test/test1/data/tcpxx drwxr-xr-x - work hdp-test 0 2020-02-02 13:49 /home/hdp-test/test1/data/udpxx
因为文件名称是第8个字段，所以通过 | awk '{printf $8"\n"}' 可以只得到目录名称
判断目录创建日期是否为180天以前，是的话就删除该目录（hadoop fs -rm -r）
脚本内容如下：
#!/bin/sh PATH=/bin:/home/hdp-test/.local/bin:/root/.local/bin:/root/perl5/bin:/sbin:/usr/bin:/usr/bin/hadoop/software/distcp:/usr/bin/hadoop/software/hadoop//bin:/usr/bin/hadoop/software/hbase//bin:/usr/bin/hadoop/software/hive//bin:/usr/bin/hadoop/software/java//bin:/usr/bin/hadoop/software/spark//bin:/usr/sbin export $PATH dirs=$(hadoop fs -ls /home/hdp-test/test1/data | awk '{printf $8"\n"}') for dir in $dirs do old_version=$(hadoop fs -ls $dir | awk 'BEGIN{ days_ago=strftime("%F", systime()-180*24*3600) }{ if($6<days_ago){printf "%s\n", $8} }') for version in $old_version do #printf $version'\n' hadoop fs -rm -r $version done done
