君子博学而日参省乎己 则知明而行无过矣

博客园 首页 新随笔 联系 订阅 管理
企业搜索引擎开发之连接器connector（十六）

本人有一段时间没有接触企业搜索引擎之连接器的开发了，连接器是涉及企业搜索引擎一个重要的组件，在数据源与企业搜索引擎中间起一个桥梁的作用，类似于数据库之JDBC，通过连接器将不同数据源的数据适配到企业搜索引擎系统

本文将描述怎样构建连接器应用实例，并开发一个简单的客户端进行测试

我们先用myeclipse建立一个web应用，命名为connector-manager，并添加相关依赖，包括连接器的相关jar依赖

本人当前引入http连接器和db连接器，还可以包括更多的连接器类型，比如数据仓库、共享文件系统、企业邮件等

pom.xml文件如下：

<dependencies>
        <dependency>
            <groupId>com.google.enterprise.connector</groupId>
            <artifactId>connector</artifactId>
            <version>3.0.8</version>
        </dependency>

        <dependency>
            <groupId>com.google.enterprise.connector</groupId>
            <artifactId>connector-http</artifactId>
            <version>2.0M3</version>
        </dependency>
        <dependency>
            <groupId>com.google.enterprise.connector</groupId>
            <artifactId>connector-db</artifactId>
            <version>3.0.0</version>
        </dependency>
    </dependencies>

 然后在myeclipse里面将该web应用编译并添加到tomcat服务器，运行tomcat

我们这时在浏览器打开http://localhost:8080/connector-manager/就可以看到如下显示：

连接器提供了一系列的servlet供客户端以编程方式调用，从而实现对连接器的管理（请求这些servlet有ge方式和post方式），我们来测试查看连接器提供了哪些数据源连接类型，servlet请求相对路径为/getConnectorList，在浏览器打开http://localhost:8080/connector-manager/getConnectorList

上面可以看到，目前提供了数据库连接类型和http连接类型

如果我们要建立一个连接实例，则需要通过post方式请求相关servlet，首先需要获取需要提交哪些post参数，可以通过相对路径为/getConfigForm的servlet获取，在浏览器打开http://localhost:8080/connector-manager/getConfigForm?ConnectorType=DBConnector可以获取到数据库连接实例的post参数（如果是http连接器，则将参数修改为ConnectorType=http-connector）

上面只是简单的通过浏览器来请求连接器的servlet，更丰富的功能则需要以编程方式来请求这些servlet，从而实现连接器的管理

接下来本人在eclipse建立一个java project，命名为SerchEngine，在pom.xml文件添加相关依赖

<dependencies>
        <!-- httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpmime</artifactId>
            <version>4.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient-cache</artifactId>
            <version>4.2.3</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>fluent-hc</artifactId>
            <version>4.2.3</version>
        </dependency>

        <dependency>
            <groupId>dom4j</groupId>
            <artifactId>dom4j</artifactId>
            <version>1.6.1</version>
        </dependency>        
    </dependencies>

本人这里的servlet请求是通过httpclient组件实现的，所以添加了httpclient组件的相关依赖；然后提交的xml数据和返回的xml数据都是通过dom4j来解析的，所以同时添加了dom4j的相关依赖

为了实现方法的公用，本人这里建立一个servlet请求类ConnectorManagerRequestUtils，包括get方式与 post方式的请求，完整代码如下：

public class ConnectorManagerRequestUtils {

    /**
     * @param args
     */
    public static void main(String[] args) {
        // TODO Auto-generated method stub
        ConnectorManager connectorManager=new ConnectorManager();
        connectorManager.setUrl("http://localhost:8080/connector-manager");
        
         Map<String, String> paramsMap=new HashMap<String, String>();
         paramsMap.put("ConnectorType", "DBConnector");
         
         Element element=ConnectorManagerRequestUtils.sendGet(connectorManager, "/getConfigForm", paramsMap);
         //Element element=ConnectorManagerRequestUtils.sendGet(connectorManager, "/getConnectorList", paramsMap);
         System.out.println(element.asXML());

    }
    private static final Logger LOGGER = Logger.getLogger(ConnectorManagerRequestUtils.class.getName());
    
    public static Element sendGet(ConnectorManager connectorManager, String servletPath, Map<String, String> paramsMap) {
        if (paramsMap == null) {
            paramsMap = new HashMap<String, String>();
        }
        
        try {
            HttpParams params = new BasicHttpParams();
            for (Iterator<String> it = paramsMap.keySet().iterator(); it.hasNext();) {
                String paramName = (String) it.next();
                String paramValue = (String) paramsMap.get(paramName);
                params.setParameter(paramName, paramValue);
            }

            HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
            HttpProtocolParams.setContentCharset(params, "UTF-8");
            HttpProtocolParams.setUserAgent(params, "HttpComponents/1.1");
            HttpProtocolParams.setUseExpectContinue(params, true);

            BasicHttpProcessor httpproc = new BasicHttpProcessor();
            // Required protocol interceptors
            httpproc.addInterceptor(new RequestContent());
            httpproc.addInterceptor(new RequestTargetHost());
            // Recommended protocol interceptors
            httpproc.addInterceptor(new RequestConnControl());
            httpproc.addInterceptor(new RequestUserAgent());
            httpproc.addInterceptor(new RequestExpectContinue());

            HttpRequestExecutor httpexecutor = new HttpRequestExecutor();

            HttpContext context = new BasicHttpContext(null);
            URL connectorManagerURL = new URL(connectorManager.getUrl());
            HttpHost host = new HttpHost(connectorManagerURL.getHost(), connectorManagerURL.getPort());

            DefaultHttpClientConnection conn = new DefaultHttpClientConnection();
            ConnectionReuseStrategy connStrategy = new DefaultConnectionReuseStrategy();

            context.setAttribute(ExecutionContext.HTTP_CONNECTION, conn);
            context.setAttribute(ExecutionContext.HTTP_TARGET_HOST, host);

            try {                
                String target = connectorManager.getUrl() + servletPath;
                boolean firstParam = true;
                for (Iterator<String> it = paramsMap.keySet().iterator(); it.hasNext();) {
                    String paramName = (String) it.next();
                    String paramValue = (String) paramsMap.get(paramName);

                    if (firstParam) {
                        target += "?";
                        firstParam = false;
                    } else {
                        target += "&";
                    }
                    target += paramName + "=" + paramValue;
                }

                if (!conn.isOpen()) {
                    Socket socket = new Socket(host.getHostName(), host.getPort());
                    conn.bind(socket, params);
                }
                BasicHttpEntityEnclosingRequest request = new BasicHttpEntityEnclosingRequest("GET", target);
                LOGGER.fine(">> Request URI: " + request.getRequestLine().getUri());

                request.setParams(params);
                httpexecutor.preProcess(request, httpproc, context);
                HttpResponse response = httpexecutor.execute(request, conn, context);
                response.setParams(params);
                httpexecutor.postProcess(response, httpproc, context);

                LOGGER.fine("<< Response: " + response.getStatusLine());
                String entityText = EntityUtils.toString(response.getEntity());
                LOGGER.fine(entityText);
                LOGGER.fine("==============");
                if (!connStrategy.keepAlive(response, context)) {
                    conn.close();
                } else {
                    LOGGER.fine("Connection kept alive...");
                }

                try {
                    Document xml = DocumentHelper.parseText(entityText);
                    return xml.getRootElement();
                } catch (Exception e) {
                    LOGGER.severe("Error caused by text : " + entityText);
                    throw e;
                }
            } finally {
                conn.close();
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
    public static Element sendPost(ConnectorManager connectorManager, String servletPath, Document document) {
        try {
            HttpParams params = new BasicHttpParams();
            HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1);
            HttpProtocolParams.setContentCharset(params, "UTF-8");
            HttpProtocolParams.setUserAgent(params, "HttpComponents/1.1");
            HttpProtocolParams.setUseExpectContinue(params, true);

            BasicHttpProcessor httpproc = new BasicHttpProcessor();
            // Required protocol interceptors
            httpproc.addInterceptor(new RequestContent());
            httpproc.addInterceptor(new RequestTargetHost());
            // Recommended protocol interceptors
            httpproc.addInterceptor(new RequestConnControl());
            httpproc.addInterceptor(new RequestUserAgent());
            httpproc.addInterceptor(new RequestExpectContinue());

            HttpRequestExecutor httpexecutor = new HttpRequestExecutor();

            HttpContext context = new BasicHttpContext(null);

            URL connectorManagerURL = new URL(connectorManager.getUrl());
            HttpHost host = new HttpHost(connectorManagerURL.getHost(), connectorManagerURL.getPort());

            DefaultHttpClientConnection conn = new DefaultHttpClientConnection();
            ConnectionReuseStrategy connStrategy = new DefaultConnectionReuseStrategy();

            context.setAttribute(ExecutionContext.HTTP_CONNECTION, conn);
            context.setAttribute(ExecutionContext.HTTP_TARGET_HOST, host);

            try {
                HttpEntity requestBody;
                if (document != null) {
//                    OutputFormat format = OutputFormat.createPrettyPrint();
                    OutputFormat format = OutputFormat.createCompactFormat();
                    StringWriter stringWriter = new StringWriter();
                    XMLWriter xmlWriter = new XMLWriter(stringWriter, format);
                    xmlWriter.write(document);
                    String xmlAsString = stringWriter.toString();
                    requestBody = new StringEntity(xmlAsString, "UTF-8");
                } else {
                    requestBody = null;
                }
                if (!conn.isOpen()) {
                    Socket socket = new Socket(host.getHostName(), host.getPort());
                    conn.bind(socket, params);
                }

                String target = connectorManager.getUrl() + servletPath;

                BasicHttpEntityEnclosingRequest request = new BasicHttpEntityEnclosingRequest("POST", target);
                request.setEntity(requestBody);
                LOGGER.info(">> Request URI: " + request.getRequestLine().getUri());

                request.setParams(params);
                httpexecutor.preProcess(request, httpproc, context);
                HttpResponse response = httpexecutor.execute(request, conn, context);
                response.setParams(params);
                httpexecutor.postProcess(response, httpproc, context);

                LOGGER.info("<< Response: " + response.getStatusLine());
                String entityText = EntityUtils.toString(response.getEntity());
                LOGGER.info(entityText);
                LOGGER.info("==============");
                if (!connStrategy.keepAlive(response, context)) {
                    conn.close();
                } else {
                    LOGGER.info("Connection kept alive...");
                }

                Document xml = DocumentHelper.parseText(entityText);
                return xml.getRootElement();
            } finally {
                conn.close();
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

}

现在运行该类的main方法，是不是在控制台看到和上面浏览器方式相同的输出呢？

更丰富的功能待后文接着分析，本文先到这里

---------------------------------------------------------------------------

本系列企业搜索引擎开发之连接器connector系本人原创

转载请注明出处 博客园 刺猬的温驯

本人邮箱： chenying998179@163#com （#改为.）

本文链接 http://www.cnblogs.com/chenying99/p/3222137.html

posted on 2013-07-29 05:27  刺猬的温驯  阅读(913)  评论(0)  编辑  收藏  举报