语种检测

语种检测language detector工具https://github.com/optimaize/language-detector

官网示例小改动后的代码：

//load all languages:
List<LanguageProfile> languageProfiles = new LanguageProfileReader().readAllBuiltIn();

//build language detector:
LanguageDetector languageDetector = LanguageDetectorBuilder.create(NgramExtractors.standard())
        .withProfiles(languageProfiles)
        .build();

//create a text object factory
TextObjectFactory textObjectFactory = CommonTextObjectFactories.forDetectingOnLargeText();

//query:
//TextObject textObject = textObjectFactory.forText("my text");
//上面的my text这个字符串不会检测到英语，因为示例中CommonTextObjectFactories创建的默认TextObjectFactory附带文本URL过滤器和0.3阈值的RemoveMinorityScriptsTextFilter，后者会把那么短的文本全过滤掉，导致不能检检测出语种。
TextObject textObject = textObjectFactory.forText("Song of the Sea (Irish: Amhrán na Mara) is a 2014 animated fantasy drama film directed and co-produced by Tomm Moore");   // long enough text containing many words....
Optional<LdLocale> lang = languageDetector.detect(textObject);

if(lang.isPresent())
    System.out.println(lang.get());
else
    System.out.println("nothing");

官网的“my text”不能检测出语种，通过查看源代码，并分析逻辑，得知原因是文本过短。涉及的具体类看代码中注释。