语言检测 language detection langid langdetect

比较 langid 和 langdetect

python2.7

import langid
langid.classify("This is a test")
('en', -54.41310358047485)
langid.classify("你好 世界")
('zh', -74.89392423629761)
langid.classify("(")
('en', 9.061840057373047)
langid.classify("（")
('ja', -0.8925070762634277)
langid.classify("中鼎集團有限公司")
('zh', -77.54190039634705)

 

from langdetect import detect
detect("Hello World")
'en'

 

 

For example, the sentence "Presidencia de la República - Mexico", the word "de" will be classified as "en", but if i change it to " de ", as add space ,it will work.

can this be worked?

import langid
str='de'
langid.classify(str)
('en', 9.061840057373047)
str=' de '
langid.classify(str)
('es', 1.0269873142242432)`

 

 

detect("CTCI(中鼎集團有限公司)")

报错，编码问题

detect(u"CTCI(中鼎集團有限公司)")
'ko'

 

总体觉得 langid 更合适。