hadoop fs:du & count统计hdfs文件(目录下文件)大小的用法
hadoop fs 更多用法,请参考官网:http://hadoop.apache.org/docs/r1.0.4/cn/hdfs_shell.html
以下是我的使用hadoop fs -du统计文件时使用的记录:
[t@dv00938 ~]$ hadoop fs -ls /jc_rc/rc_hive_db/llcfpd_s_join_n_over0innerdoor_lishui 17/10/15 12:14:41 INFO hdfs.PeerCache: SocketCache disabled. Found 113 items -rwxrwx---+ 3 t_user t_user_group 282171728 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000000_0 -rwxrwx---+ 3 t_user t_user_group 281446475 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000001_0 -rwxrwx---+ 3 t_user t_user_group 281021562 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000002_0 -rwxrwx---+ 3 t_user t_user_group 280834172 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000003_0 -rwxrwx---+ 3 t_user t_user_group 280411919 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000004_0 -rwxrwx---+ 3 t_user t_user_group 279749295 2017-10-14 20:38 /t_user/my_hive_db/my_hive_table_test01/000005_0 ...
[t@dv00938 ~]$ hadoop fs -du /t_user/my_hive_db/my_hive_table_test01 17/10/15 12:14:54 INFO hdfs.PeerCache: SocketCache disabled. 282171728 /t_user/my_hive_db/my_hive_table_test01/000000_0 281446475 /t_user/my_hive_db/my_hive_table_test01/000001_0 281021562 /t_user/my_hive_db/my_hive_table_test01/000002_0 280834172 /t_user/my_hive_db/my_hive_table_test01/000003_0 280411919 /t_user/my_hive_db/my_hive_table_test01/000004_0 279749295 /t_user/my_hive_db/my_hive_table_test01/000005_0 ...
[t@dv00938 ~]$ hadoop fs -du -h /t_user/my_hive_db/my_hive_table_test01 17/10/15 12:17:05 INFO hdfs.PeerCache: SocketCache disabled. 269.1 M /t_user/my_hive_db/my_hive_table_test01/000000_0 268.4 M /t_user/my_hive_db/my_hive_table_test01/000001_0 268.0 M /t_user/my_hive_db/my_hive_table_test01/000002_0 267.8 M /t_user/my_hive_db/my_hive_table_test01/000003_0 267.4 M /t_user/my_hive_db/my_hive_table_test01/000004_0 266.8 M /t_user/my_hive_db/my_hive_table_test01/000005_0 ...
[t@dv00938 ~]$ hadoop fs -du -s /t_user/my_hive_db/my_hive_table_test01 17/10/15 12:16:52 INFO hdfs.PeerCache: SocketCache disabled. 29959873142 /t_user/my_hive_db/my_hive_table_test01 [t@dv00938 ~]$ hadoop fs -du -s -h /t_user/my_hive_db/my_hive_table_test01 17/10/15 12:17:17 INFO hdfs.PeerCache: SocketCache disabled. 27.9 G /t_user/my_hive_db/my_hive_table_test01
hadoop fs -count
[dx@d-143 conf]$ hadoop fs -count /user/dx/ 45241 110279 26340662670 /user/dx [dx@d-143 conf]$ hadoop fs -count /user/dx/* 12 4 45844 /user/dx/.Trash 15 502 2382220393 /user/dx/.sparkStaging 1 2 28116579 /user/dx/base_data 45212 109771 23930279854 /user/dx/streaming
基础才是编程人员应该深入研究的问题,比如:
1)List/Set/Map内部组成原理|区别
2)mysql索引存储结构&如何调优/b-tree特点、计算复杂度及影响复杂度的因素。。。
3)JVM运行组成与原理及调优
4)Java类加载器运行原理
5)Java中GC过程原理|使用的回收算法原理
6)Redis中hash一致性实现及与hash其他区别
7)Java多线程、线程池开发、管理Lock与Synchroined区别
8)Spring IOC/AOP 原理;加载过程的。。。
【+加关注】。