摘要:
添加调试参数: <property> <name>mapred.child.java.opts</name> <value>-Xmx200m-Xdebug-Xrunjdwp:transport=dt_socket,address=9999,server=y,suspend=y</value> </p 阅读全文
摘要:
提交作业: hadoop jar /opt/hadoop-1.0.0/hadoop-examples-1.0.0.jar wordcount /user/admin/in/yellow2.txt /user/admin/out/128 生成2个Map、2个Reduce任务。 执行Maps[0]: a 阅读全文
摘要:
1、启动 HttpServer this.server=new HttpServer("task",httpBindAddress, httpPort, httpPort == 0, conf, aclsManager.getAdminsAcl()); server.start(); 2、生成 De 阅读全文
摘要:
提交作业: hadoop jar /opt/hadoop-1.0.0/hadoop-examples-1.0.0.jarwordcount /user/admin/in/LICENSE.txt /user/admin/out/26 , 只有一个Map任务 server3-RPC请求:getProto 阅读全文
摘要:
JobTracker无任务时处理心跳流程 HeartBeat格式:{restarted=true,initialContact=true,acceptNewTasks=true,responseId=-1, status=TaskTrackerStatus {failures=0,trackerNa 阅读全文
摘要:
2. JobTracker中的容器 2.1 nodesAtMaxLevel、hostnameToNodeMap //(hostname --> Node(NetworkTopology)) Map<String,Node> hostnameToNodeMap = Collections.synchr 阅读全文
摘要:
1. JobInProgress中的容器 1.1maps、reduces、cleanup、setup TaskInProgress maps[]= new TaskInProgress[0]; TaskInProgress reduces[]= new TaskInProgress[0]; Task 阅读全文
摘要:
1.初始化和启动JobTracker 生成一个JobQueueTaskScheduler taskScheduler=(TaskScheduler)ReflectionUtils.newInstance(schedulerClass,conf); 生成一个实现 InterTrackerProtoco 阅读全文
摘要:
1. 配置所有节点的$HADOOP_HOME/conf/hadoop-env.sh 文件 exportHADOOP_HOME=/opt/hadoop-1.0.0 exportHADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote.port=9191- 阅读全文
摘要:
命令为: hadoop_debugjar /opt/hadoop-1.0.0/hadoop-examples-1.0.0.jar wordcount/user/admin/in/yellow.txt /user/admin/out/555 首先调用org.apache.hadoop.util.run 阅读全文
摘要:
一、服务器端线程 1、主线程main,启动其余线程后不参与交互 2、1个Listener线程,监听连接请求,并在有连接的时候唤醒Reader线程 public void run() { while(running){ selector.select(); Iterator<SelectionKey> 阅读全文
摘要:
一、服务器端线程 1、主线程main,启动其余线程后不参与交互 2、1个Listener线程,监听连接请求,并在有连接的时候唤醒Reader线程 public void run() { while(running){ selector.select(); Iterator<SelectionKey> 阅读全文
摘要:
考虑一个生产者/消费者模型。 商品类Goods: class Goods { private String name; public Goods(String name) { this.name =name; } public String toString() { return "Goods[" 阅读全文
摘要:
示例代码: public class SyncTest { private List<String> list= new ArrayList<String>(); public void syncAddPrint(){ synchronized(list){ for(int i = 0; i < 1 阅读全文
摘要:
示例代码: public class SyncTest { private List<String> list= new ArrayList<String>(); public void syncAddPrint(){ synchronized(list){ for(int i = 0; i < 1 阅读全文
摘要:
分析任务reduce_1 args =[127.0.0.1, 42767, attempt_201405060431_0003_r_000001_0,/opt/hadoop-1.0.0/logs/userlogs/job_201405060431_0003/attempt_201405060431_ 阅读全文
摘要:
提交命令行:hadoop jar /opt/hadoop-1.0.0/hadoop-examples-1.0.0.jar wordcount /user/admin/in/yellow.txt /user/admin/out/3 JobTracker已生成JobInProgress(job_2014 阅读全文
摘要:
输入文件:hdfs://server1:9000/user/admin/in/yellow.txt 1.splits formatMinSplitSize:1; minSplitSize=conf("mapred.min.split.size"):1; minSize=Math.max(format 阅读全文
摘要:
提交命令行:hadoop jar/opt/hadoop-1.0.0/hadoop-examples-1.0.0.jar wordcount/user/admin/in/yellow.txt /user/admin/out/3 RPC请求:getProtocolVersion(org.apache.h 阅读全文
摘要:
List<Integer> list = new ArrayList<Integer>(); for (int i = 0; i < 10; i++) { list.add(i); } System.out.println(list); //[0, 1,2, 3, 4, 5, 6, 7, 8, 9] 阅读全文