强制DataNode向NameNode上报blocks
正常情况下,什么时候上报blocks,是由NameNode通过回复心跳响应的方式触发的。
一次机房搬迁中,原机房hadoop版本为2.7.2,新机房版本为2.8.0,采用先扩容再缩容的方式搬迁。由于新旧机房机型不同和磁盘数不同,操作过程搞混过hdfs-site.xml,因为两种不同的机型,hdfs-site.xml不便做到相同,导致了NameNode报大量“missing block”。
然而依据NameNode所报信息,在DataNode能找到那些被标记为“missing”的blocks。修复配置问题后,“missing block”并没有消失。结合DataNode源代码,估计是因为DataNode没有向NameNode上报blocks。
结合DataNode的源代码,发现了HDFS自带的工具triggerBlockReport,它可以强制指定的DataNode向NameNode上报块,使用方法为:
hdfs dfsadmin -triggerBlockReport datanode_host:ipc_port
如:hdfs dfsadmin -triggerBlockReport 192.168.31.35:50020
正常情况下NameNode启动时,会要求DataNode上报一次blocks(通过fullBlockReportLeaseId值来控制),相关源代码如下:
DataNode相关代码(BPServiceActor.java):
private void offerService() throws Exception {
HeartbeatResponse resp = sendHeartBeat(requestBlockReportLease); // 向NameNode发向心跳
long fullBlockReportLeaseId = resp.getFullBlockReportLeaseId(); // 心跳响应
boolean forceFullBr = scheduler.forceFullBlockReport.getAndSet(false); // triggerBlockReport强制上报仅一次有效
if (forceFullBr) {
LOG.info("Forcing a full block report to " + nnAddr);
}
if ((fullBlockReportLeaseId != 0) || forceFullBr) {
cmds = blockReport(fullBlockReportLeaseId);
fullBlockReportLeaseId = 0;
}
}
// NameNode相关代码(FSNamesystem.java):
/**
* The given node has reported in. This method should:
* 1) Record the heartbeat, so the datanode isn't timed out
* 2) Adjust usage stats for future block allocation
*
* If a substantial amount of time passed since the last datanode
* heartbeat then request an immediate block report.
*
* @return an array of datanode commands
* @throws IOException
*/
HeartbeatResponse handleHeartbeat(DatanodeRegistration nodeReg,
StorageReport[] reports, long cacheCapacity, long cacheUsed,
int xceiverCount, int xmitsInProgress, int failedVolumes,
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease) throws IOException {
readLock();
try {
//get datanode commands
final int maxTransfer = blockManager.getMaxReplicationStreams() - xmitsInProgress;
DatanodeCommand[] cmds = blockManager.getDatanodeManager().handleHeartbeat(
nodeReg, reports, blockPoolId, cacheCapacity, cacheUsed,
xceiverCount, maxTransfer, failedVolumes, volumeFailureSummary);
long fullBlockReportLeaseId = 0;
if (requestFullBlockReportLease) {
fullBlockReportLeaseId = blockManager.requestBlockReportLeaseId(nodeReg);
}
//create ha status
final NNHAStatusHeartbeat haState = new NNHAStatusHeartbeat(
haContext.getState().getServiceState(),
getFSImage().getCorrectLastAppliedOrWrittenTxId());
return new HeartbeatResponse(cmds, haState, rollingUpgradeInfo, fullBlockReportLeaseId);
} finally {
readUnlock("handleHeartbeat");
}
}
一次机房搬迁中,原机房hadoop版本为2.7.2,新机房版本为2.8.0,采用先扩容再缩容的方式搬迁。由于新旧机房机型不同和磁盘数不同,操作过程搞混过hdfs-site.xml,因为两种不同的机型,hdfs-site.xml不便做到相同,导致了NameNode报大量“missing block”。
然而依据NameNode所报信息,在DataNode能找到那些被标记为“missing”的blocks。修复配置问题后,“missing block”并没有消失。结合DataNode源代码,估计是因为DataNode没有向NameNode上报blocks。
结合DataNode的源代码,发现了HDFS自带的工具triggerBlockReport,它可以强制指定的DataNode向NameNode上报块,使用方法为:
hdfs dfsadmin -triggerBlockReport datanode_host:ipc_port
如:hdfs dfsadmin -triggerBlockReport 192.168.31.35:50020
正常情况下NameNode启动时,会要求DataNode上报一次blocks(通过fullBlockReportLeaseId值来控制),相关源代码如下:
DataNode相关代码(BPServiceActor.java):
private void offerService() throws Exception {
HeartbeatResponse resp = sendHeartBeat(requestBlockReportLease); // 向NameNode发向心跳
long fullBlockReportLeaseId = resp.getFullBlockReportLeaseId(); // 心跳响应
boolean forceFullBr = scheduler.forceFullBlockReport.getAndSet(false); // triggerBlockReport强制上报仅一次有效
if (forceFullBr) {
LOG.info("Forcing a full block report to " + nnAddr);
}
if ((fullBlockReportLeaseId != 0) || forceFullBr) {
cmds = blockReport(fullBlockReportLeaseId);
fullBlockReportLeaseId = 0;
}
}
// NameNode相关代码(FSNamesystem.java):
/**
* The given node has reported in. This method should:
* 1) Record the heartbeat, so the datanode isn't timed out
* 2) Adjust usage stats for future block allocation
*
* If a substantial amount of time passed since the last datanode
* heartbeat then request an immediate block report.
*
* @return an array of datanode commands
* @throws IOException
*/
HeartbeatResponse handleHeartbeat(DatanodeRegistration nodeReg,
StorageReport[] reports, long cacheCapacity, long cacheUsed,
int xceiverCount, int xmitsInProgress, int failedVolumes,
VolumeFailureSummary volumeFailureSummary,
boolean requestFullBlockReportLease) throws IOException {
readLock();
try {
//get datanode commands
final int maxTransfer = blockManager.getMaxReplicationStreams() - xmitsInProgress;
DatanodeCommand[] cmds = blockManager.getDatanodeManager().handleHeartbeat(
nodeReg, reports, blockPoolId, cacheCapacity, cacheUsed,
xceiverCount, maxTransfer, failedVolumes, volumeFailureSummary);
long fullBlockReportLeaseId = 0;
if (requestFullBlockReportLease) {
fullBlockReportLeaseId = blockManager.requestBlockReportLeaseId(nodeReg);
}
//create ha status
final NNHAStatusHeartbeat haState = new NNHAStatusHeartbeat(
haContext.getState().getServiceState(),
getFSImage().getCorrectLastAppliedOrWrittenTxId());
return new HeartbeatResponse(cmds, haState, rollingUpgradeInfo, fullBlockReportLeaseId);
} finally {
readUnlock("handleHeartbeat");
}
}
【推荐】国内首个AI IDE,深度理解中文开发场景,立即下载体验Trae
【推荐】编程新体验,更懂你的AI,立即体验豆包MarsCode编程助手
【推荐】抖音旗下AI助手豆包,你的智能百科全书,全免费不限次数
【推荐】轻量又高性能的 SSH 工具 IShell:AI 加持,快人一步
· .NET Core 中如何实现缓存的预热?
· 从 HTTP 原因短语缺失研究 HTTP/2 和 HTTP/3 的设计差异
· AI与.NET技术实操系列:向量存储与相似性搜索在 .NET 中的实现
· 基于Microsoft.Extensions.AI核心库实现RAG应用
· Linux系列:如何用heaptrack跟踪.NET程序的非托管内存泄露
· TypeScript + Deepseek 打造卜卦网站:技术与玄学的结合
· 阿里巴巴 QwQ-32B真的超越了 DeepSeek R-1吗?
· 【译】Visual Studio 中新的强大生产力特性
· 【设计模式】告别冗长if-else语句:使用策略模式优化代码结构
· 10年+ .NET Coder 心语 ── 封装的思维:从隐藏、稳定开始理解其本质意义