Hadoop 常见问题 常见错误
-
启用Heap Dump
In each script, there is an export for *_OPTS, which contains the options passed to the JVM. For example, in the hadoop-env.sh script, the line that begins with export HADOOP_NAMENODE_OPTS= contains the options for the NameNode service.
Map and reduce processes are slightly different, as these are a child process of the MapReduce service. Each map or reduce process runs in a child container, and there are two entries that contain the JVM options for these. Both contained in mapred-site.xml:
- mapreduce.admin.map.child.java.opts
- mapreduce.admin.reduce.child.java.opts
-XX:+HeapDumpOnOutOfMemoryError enables heap dumps when an OutOfMemoryError occurs; The default location for the dump file is the current working directory. You can control where the file is stored using the following option -XX:HeapDumpPath=/path.
E.g. Hadoop-env.sh : export Hadoop_NAMENODE_OPTS="-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp...". When enabling heap dumps for the map or reduce child process, you will instead look for the fields labled mapreduce.admin.map.child.java.opts and mapreduce.admin.reduce.child.java.opts.
