Hdfs Active Namenode向JournalNode同步editlog流程
FSEditLog
Active namenode进程上使用的类,达到条件后执行logSync()
向journalnode同步最新的EditLog段
调用过程
FSEditLog.logSync(long mytxid)
->EditLogOutputStream.flush()
->EditLogOutputStream.flush(boolean durable)
->EditLogOutputStream.flushAndSync(boolean durable)
->JournalSet.flushAndSync(boolean durable)
->JournalSet.mapJournalsAndReportErrors(
JournalClosure closure, String status)
此过程最终的调用函数为JournalSet中的
/**
* Apply the given operation across all of the journal managers, disabling
* any for which the closure throws an IOException.
* @param closure {@link JournalClosure} object encapsulating the operation.
* @param status message used for logging errors (e.g. "opening journal")
* @throws IOException If the operation fails on all the journals.
*/
private void mapJournalsAndReportErrors(
JournalClosure closure, String status) throws IOException{
List<JournalAndStream> badJAS = Lists.newLinkedList();
for (JournalAndStream jas : journals) {
try {
closure.apply(jas);
} catch (Throwable t) {
if (jas.isRequired()) {
final String msg = "Error: " + status + " failed for required journal ("
+ jas + ")";
LOG.fatal(msg, t);
// If we fail on *any* of the required journals, then we must not
// continue on any of the other journals. Abort them to ensure that
// retry behavior doesn't allow them to keep going in any way.
abortAllJournals();
// the current policy is to shutdown the NN on errors to shared edits
// dir. There are many code paths to shared edits failures - syncs,
// roll of edits etc. All of them go through this common function
// where the isRequired() check is made. Applying exit policy here
// to catch all code paths.
terminate(1, msg);
} else {
LOG.error("Error: " + status + " failed for (journal " + jas + ")", t);
badJAS.add(jas);
}
}
}
disableAndReportErrorOnJournals(badJAS);
if (!NameNodeResourcePolicy.areResourcesAvailable(journals,
minimumRedundantJournals)) {
String message = status + " failed for too many journals";
LOG.error("Error: " + message);
throw new IOException(message);
}
}
mapJournalsAndReportErrors中如果成功个数没有达到minimumRedundantJournals
,则namenode会主动shutdown。