利用SOLR搭建企业搜索平台 之十一（中文分词之IK）

在经过使用了庖丁以后，这里说说怎么将目前很火很流行的IK集成进SOLR，其实方法真的很简单，比paoding方便不少。这里很感谢IK的作者，蓝山咖啡，很感谢你为中文分词做出的贡献。 作者博客：http://linliangyi2007.javaeye.com

入正题：
1》请先去作者博客参看IK下载地址，主要就是一个IKAnalyzer3.1.1Stable.jar。我这里用的是最新版！

package com.yeedoo.slor.tokenizer;

import java.io.Reader;

import org.apache.lucene.analysis.TokenStream;
import org.apache.solr.analysis.BaseTokenizerFactory;
import org.wltea.analyzer.lucene.IKAnalyzer;

public class ChineseTokenizerFactory extends BaseTokenizerFactory {

    @Override
    public TokenStream create(Reader reader) {
        return new IKAnalyzer().tokenStream("text", reader);
    }

}

package com.yeedoo.slor.tokenizer;  import java.io.Reader;  import org.apache.lucene.analysis.TokenStream; import org.apache.solr.analysis.BaseTokenizerFactory; import org.wltea.analyzer.lucene.IKAnalyzer;  public class ChineseTokenizerFactory extends BaseTokenizerFactory {  	@Override 	public TokenStream create(Reader reader) { 		return new IKAnalyzer().tokenStream("text", reader); 	} 	 }

从代码就可以看得出来真的很方便！将它打包放入solr.war中同时还有IK的jar包。如果你不想打包，请去附件下载已经打好的包。

2》配置文件

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
            <analyzer type="index">
                <tokenizer class="com.yeedoo.slor.tokenizer.ChineseTokenizerFactory" />
                <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" />
                <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" />
                <filter class="solr.LowerCaseFilterFactory" />
                <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt" />
                <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
            </analyzer>
            <analyzer type="query">
                <tokenizer class="com.yeedoo.slor.tokenizer.ChineseTokenizerFactory" />
                <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" />
                <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" />
                <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" />
                <filter class="solr.LowerCaseFilterFactory" />
                <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt" />
                <filter class="solr.RemoveDuplicatesTokenFilterFactory" />
            </analyzer>
        </fieldType>

<fieldType name="text" class="solr.TextField" positionIncrementGap="100"> 			<analyzer type="index"> 				<tokenizer class="com.yeedoo.slor.tokenizer.ChineseTokenizerFactory" /> 				<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" enablePositionIncrements="true" /> 				<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0" splitOnCaseChange="1" /> 				<filter class="solr.LowerCaseFilterFactory" /> 				<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt" /> 				<filter class="solr.RemoveDuplicatesTokenFilterFactory" /> 			</analyzer> 			<analyzer type="query"> 				<tokenizer class="com.yeedoo.slor.tokenizer.ChineseTokenizerFactory" /> 				<filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true" /> 				<filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt" /> 				<filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="0" catenateNumbers="0" catenateAll="0" splitOnCaseChange="1" /> 				<filter class="solr.LowerCaseFilterFactory" /> 				<filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt" /> 				<filter class="solr.RemoveDuplicatesTokenFilterFactory" /> 			</analyzer> 		</fieldType>

关于这里的配置文件，如果你更详细的了解意思，请参见我的另一篇博客，博客地址：http://lianj-lee.javaeye.com/blog/424474

3》如果在启动tomcat没有报错的情况下，可以去solr的admin那里尝试看看该分词器的效果，效果如下图：

这样就ok了。简单、方便、“无公害”... ...