EMR日常操作

１／　数据root密码修改后emr配置修改：

 

 2/ 添加s3访问

S3:
hadoop-aws-2.8.5.jar
jar -tfv hadoop-aws-2.8.5.jar |grep S3FileSystem
/usr/lib/hadoop-current/share/hadoop/tools/lib
/usr/lib/hadoop-current/share/hadoop/common/lib
放到hadoop和hive的classpath下?

su - hadoop
echo $HADOOP_CLASSPATH

 

 

<property>
<name>fs.s3a.access.key</name>
<value>access.key</value>
</property>
<property>
<name>fs.s3a.secret.key</name>
<value>secret.key</value>
</property>
<property>
<name>fs.s3a.impl</name>
<value>org.apache.hadoop.fs.s3a.S3AFileSystem</value>
</property>

#hadoop使用S3
sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/*aws* /usr/lib/hadoop-current/share/hadoop/common/lib/
sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/*jackson* /usr/lib/hadoop-current/share/hadoop/common/lib/
sudo cp /usr/lib/hadoop-current/share/hadoop/tools/lib/joda-time-2.9.4.jar /usr/lib/hadoop-current/share/hadoop/common/lib/

#hive使用s3
包拷贝到同样的hive目录