saiku执行过程代码跟踪

使用了很久的saiku，决定跟踪一下代码，看看它的执行核心过程：

一、入口controller代码

1.1、页面打开之后，会发送一个ajax请求

Request URL:

http://l-tdata2.tkt.cn6.qunar.com:8080/saiku/rest/saiku/api/query/execute

Request Method:

POST

1.2、controller，java文件org.saiku.web.rest.resources.Query2Resource

如果有缓存，直接输出数据

没有缓存，计算在输出数据

二、service代码

2.1、service，执行核心代码 org.saiku.service.olap.ThinQueryService的private CellDataSet execute(ThinQuery tq, ICellSetFormatter formatter)方法

执行mdx语句

            Long start = (new Date()).getTime();
            log.debug("Query Start");
            CellSet cellSet =  executeInternalQuery(tq); //这是执行mdx语句的地方，需要较长时间
            log.debug("Query End");
            String runId = "RUN#:" + ID_GENERATOR.get();
            Long exec = (new Date()).getTime();

三、核心代码

执行mdx语句 org.saiku.service.olap.ThinQueryService的CellSet executeInternalQuery(ThinQuery query) throws Exception 方法

    CellSet executeInternalQuery(ThinQuery query) throws Exception {
        String runId = "RUN#:" + ID_GENERATOR.getAndIncrement();
        QueryContext queryContext = context.get(query.getName());

        if (queryContext == null) {
            queryContext = new QueryContext(Type.OLAP, query);
            this.context.put(query.getName(), queryContext);
        }

        // 根据数据立方体建立olap的jdbc链接
        OlapConnection con = olapDiscoverService.getNativeConnection(query.getCube().getConnection());
        if (StringUtils.isNotBlank(query.getCube().getCatalog())) {
            con.setCatalog(query.getCube().getCatalog());
        }

        if (queryContext.contains(ObjectKey.STATEMENT)) {
            Statement s = queryContext.getStatement();
            s.cancel();
            s.close();
            s = null;
            queryContext.remove(ObjectKey.STATEMENT);
        }

        OlapStatement stmt = con.createStatement(); // 实例化Statement对象
        queryContext.store(ObjectKey.STATEMENT, stmt);

        query = updateQuery(query);

        try {
            String mdx = query.getParameterResolvedMdx();
            log.info(runId + "\tType:" + query.getType() + ":\n" + mdx);

            CellSet cs = stmt.executeOlapQuery(mdx); //这里是执行mdx语句的过程，耗时最久
            queryContext.store(ObjectKey.RESULT, cs);
            //追踪代码cs使用
            log.info("cs:" + cs.toString());
            if (query != null) {
                queryContext.store(ObjectKey.QUERY, query);
            }
            //追踪代码query使用
            log.info("query:" + query.toString());
            return cs;
        } finally {
            stmt.close();
            queryContext.remove(ObjectKey.STATEMENT);
        }
    }

四、执行日志：

上面的注释，是通过日志来作证的，日志如下：

2016-06-12 14:46:21,571 DEBUG [org.saiku.web.rest.resources.Query2Resource] TRACK        /query/F7CE71C7-3E29-0A6A-9BC9-FDDA1A129BB7    POST     tq:false file:/homes/saiku_search.saiku
2016-06-12 14:46:21,686 DEBUG [org.saiku.service.olap.ThinQueryService] Query Start
2016-06-12 14:46:21,814 INFO  [org.saiku.service.olap.ThinQueryService] RUN#:1    Type:QUERYMODEL:
WITH
SET [~COLUMNS] AS
    {[category_name_id].[category_name_id].[category_name].Members}
SET [~ROWS_rpt_date_rpt_date] AS
    {[rpt_date].[rpt_date].[2016-06-07]}
SET [~ROWS_partner_partner] AS
    Hierarchize({{[partner].[partner].[All partners]}, {[partner].[partner].[name].Members}})
SET [~ROWS_from_area_id_from_area_id] AS
    Hierarchize({{[from_area_id].[from_area_id].[All from_area_ids]}, {[from_area_id].[from_area_id].[name].Members}})
SET [~ROWS_utmr_page_id_utmr_page_id] AS
    {[utmr_page_id].[utmr_page_id].[All utmr_page_ids]}
SET [~ROWS_in_track_in_track] AS
    {[in_track].[in_track].[All in_tracks]}
SET [~ROWS_dist_city_dist_city] AS
    {[dist_city].[dist_city].[All dist_citys]}
SET [~ROWS_current_city_current_city] AS
    {[current_city].[current_city].[All current_citys]}
SET [~ROWS_from_value_id_from_value_id] AS
    {[from_value_id].[from_value_id].[All from_value_ids]}
SET [~ROWS_page_id_page_id] AS
    {[page_id].[page_id].[All page_ids]}
SELECT
NON EMPTY CrossJoin([~COLUMNS], {[Measures].[num], [Measures].[gid]}) ON COLUMNS,
NON EMPTY NonEmptyCrossJoin([~ROWS_rpt_date_rpt_date], NonEmptyCrossJoin([~ROWS_partner_partner], NonEmptyCrossJoin([~ROWS_from_area_id_from_area_id], NonEmptyCrossJoin([~ROWS_utmr_page_id_utmr_page_id], NonEmptyCrossJoin([~ROWS_in_track_in_track], NonEmptyCrossJoin([~ROWS_dist_city_dist_city], NonEmptyCrossJoin([~ROWS_current_city_current_city], NonEmptyCrossJoin([~ROWS_from_value_id_from_value_id], [~ROWS_page_id_page_id])))))))) ON ROWS
FROM [saiku_search_detail_cube]
2016-06-12 14:50:58,344 INFO  [org.saiku.service.olap.ThinQueryService] cs:mondrian.olap4j.FactoryJdbc41Impl$MondrianOlap4jCellSetJdbc41@2c72fc4f
2016-06-12 14:50:58,344 INFO  [org.saiku.service.olap.ThinQueryService] query:org.saiku.olap.query2.ThinQuery@3112bd55
2016-06-12 14:50:58,344 DEBUG [org.saiku.service.olap.ThinQueryService] Query End
2016-06-12 14:50:58,442 DEBUG [org.saiku.service.olap.ThinQueryService] cellSet2Matrix End
2016-06-12 14:50:58,443 DEBUG [org.saiku.service.olap.ThinQueryService] calculateTotals End
2016-06-12 14:50:58,443 INFO  [org.saiku.service.olap.ThinQueryService] RUN#:2    Size: 23/76    Execute:    276658ms    Format:    98ms    Totals:    1ms     Total: 276757ms

上面是执行了一个超级数据的日志，红色部分标志出了执行时间最久的部分，日志是我重新编译代码得出的，可见执行的核心代码就是第三部分标出的红色部分代码

五、olap4j引擎

第四部分的代码，核心是建立olap的jdbc链接。下面是原文：http://www.olap4j.org/

olap4j is an open Java API for OLAP.
Think of it like JDBC, but for accessing multi-dimensional data.

olap4j is a common API for any OLAP server, so you can write an analytic application on one server and easily switch it to another. Built on that API, there is a growing collection of tools and components.

这个略微有点抽象，走到这一步，说明大家已经明白了数据立方体的定义，以及上传的xml文件就定义了一个多维数据库（不明白的同学翻看以前的博客：http://www.cnblogs.com/liqiu）。那么定义好了多维数据库，就需要获取里面的数据，olap4j就是这样的一个实现了jdbc规范的多为数据库查询引擎！

总结：

看了上面的过程，大家就能了解saiku的执行过程了吧

saiku前端发送mdx查询ajax请求
saiku后端接收mdx语句
包装一下查询内容
调用olap4j引擎查询数据库结果
修饰数据并返回
saiku前端展示出来

预告：下一期会讨论一下saiku的缓存机制

posted @ 2016-06-12 15:26 李秋阅读(1727) 评论(0) 编辑收藏举报

刷新页面返回顶部