多线程处理慢sql查询小笔记~

多线程处理慢sql查询以及List（Array）的拆分

系统数据量不大，但是访问速度特别慢，使用多线程优化一下！！！

优化结果：访问时间缩短了十几秒  25s --> 8s

一、List的拆分：Iterables.partition

注意: 引入的包为google名下的

　　  Iterables.partitions(List list , int size);  // list表示需要被拆分的集合，size表示拆分后的集合的元素个数

　　 例如：size=3,表示会拆分为2个集合，第一个集合中的元素为{1,2,3},第二个集合中的元素为{4,5}

　　　　　 size=2，表示会拆分为3个集合，分别为{1,2} {3,4} {5}  以此类推

ps: 集合的合并：

List list = new ArrayList();

List list1 = new ArrayList();

 list.addAll(list1);

import com.google.common.collect.Iterables;  

List<String> strList = new ArrayList<String>();
strList.add(1);
strList.add(2);
strList.add(3);
strList.add(4);
strList.add(5);
Iterable<List<String>> splitStringlList = Iterables.partition(strList, 3); //3表示拆分后的集合元素的数量

 

二、Array的拆分：Arrays.asList(ArrayObj)

先将数组转换为List,然后再按照第一方法去拆分。

String[] validRiderArray = [1,2,3,4,5];
ArrayList<String> validArrayList = new ArrayList<String>(Arrays.asList(validRiderArray));

 

三、多线程处理类：

 

Runtime.getRuntime().availableProcessors();//返回到Java虚拟机的可用的处理器数量

 

package com.chinalife.proposal.common.util;

import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import org.springframework.stereotype.Component;

import com.google.common.util.concurrent.ThreadFactoryBuilder;
/**
 * 固定线程个数的线程池工具类
 * */
 
public class SimpleFixedThreadPoolUtil {
    /** 默认线程个数 */
    private static final int DEFAULT_THREAD = Runtime.getRuntime().availableProcessors();//返回到Java虚拟机的可用的处理器数量
    /** 线程池个数: 默认为： Runtime.getRuntime().availableProcessors() * 2 */
    public static int THREADPOOL_NUM = DEFAULT_THREAD * 2;
    private static ExecutorService executor = Executors.newFixedThreadPool(THREADPOOL_NUM,
            new ThreadFactoryBuilder().setNameFormat("SimpleFixedThreadPool- %d: ").build());

    /**
     * 提交任务,任务执行后无返回值
     * 
     * @param task
     */
    public static void addTask(Runnable task) {
        executor.execute(task);
    }

    /**
     * 提交具有返回值的任务
     * @param task
     * @return
     */
    public static <T> Future<T> addTask(Callable<T> task) {
        Future<T> future = executor.submit(task);
        return future;
    }

    /**
     * 关闭线程池
     */
    /*public void shutdown() {
        executor.shutdown();
    }*/
}

 

四、多线程处理慢查询使用示例：

JunitTest:

/**
     * 多线程处理慢sql查询示例
     * @throws Exception
     */
    @Test
    public void testAddTaskCallables() throws Exception {
        String role = "Agent";
        String nodeid = "2";
        String subnodeid = "3";
        long time = System.currentTimeMillis();
        List<OnlProposalRolePrdRelation> rolePrdlist = webProposalService.getAvailableProductListByRoleAndID(role,nodeid, subnodeid);
        Iterable<List<OnlProposalRolePrdRelation>> result = Iterables.partition(rolePrdlist, 30);
        Iterator<List<OnlProposalRolePrdRelation>> iterator = result.iterator();
        List<Future<List<OnlProposalRolePrdRelation>>> futures = new ArrayList<>();
        while (iterator.hasNext()) {
            List<OnlProposalRolePrdRelation> listTemp = iterator.next();
            Future<List<OnlProposalRolePrdRelation>> future = SimpleFixedThreadPoolUtil
                    .addTask((Callable<List<OnlProposalRolePrdRelation>>) () -> {
                        List<OnlProposalRolePrdRelation> relations = null;
                        try {
                            relations = webProposalService.filterByCriteria(listTemp, 20, "1", "1", "0");
                            return relations;
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                        return relations;
                    });
            futures.add(future);
        }
        List<OnlProposalRolePrdRelation> proposalResultList = new ArrayList<OnlProposalRolePrdRelation>();
        for (Future<List<OnlProposalRolePrdRelation>> future : futures) {
            while (!future.isDone()) {
                Thread.sleep(100);
            }
            List<OnlProposalRolePrdRelation> relations = future.get();
            System.out.println(relations.size());
            proposalResultList.addAll(relations);

        }
        logger.debug("time 3 =" + (System.currentTimeMillis() - time) / 1000);
        System.out.println("stop");
    }

 

 实际应用：

        ArrayList<String> validArrayList = new ArrayList<String>(Arrays.asList(validRiderArray)); 
        
        //step1 method start
        //List<TblProposalRiderAttr> tblProposalRiderAttrList = getValidRiderListStep1(validRiderArray, parameter,tblProposalProductSetting, tPresentrelationForRider );
        Iterable<List<String>> splitValidArrayList = Iterables.partition(validArrayList, 15);
        Iterator<List<String>> iterator = splitValidArrayList.iterator();
        List<Future<List<TblProposalRiderAttr>>> tblRiderFutures = new ArrayList<Future<List<TblProposalRiderAttr>>>();
        while (iterator.hasNext()) {
            List<String> tmpValidArrayList = iterator.next();
            //调用多线程执行  future为单个list的返回结果; tblRiderFutures为所有返回结果的集合
            Future<List<TblProposalRiderAttr>> future = SimpleFixedThreadPoolUtil
                    .addTask((Callable<List<TblProposalRiderAttr>>) () -> {
                        List<TblProposalRiderAttr> tmpTblRiderList = null;
                        try {
                            //调用step1方法进行筛选附加产品信息  进行多线程处理的方法
                            tmpTblRiderList = getValidRiderListStep1(tmpValidArrayList, parameter,tblProposalProductSetting,tPresentrelationForRider);
                            return tmpTblRiderList;
                        } catch (Exception e) {
                            e.printStackTrace();
                        }
                        return tmpTblRiderList;
                    });
            tblRiderFutures.add(future);
        }
        //将筛选后的结果添加到集合中
        List<TblProposalRiderAttr> tblProposalRiderAttrList = new ArrayList<TblProposalRiderAttr>();
        for (Future<List<TblProposalRiderAttr>> future : tblRiderFutures) {
            while (!future.isDone()) {
                try {
                    Thread.sleep(100);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
            try {
                tblProposalRiderAttrList.addAll(future.get());
            } catch (InterruptedException e) {
                e.printStackTrace();
            } catch (ExecutionException e) {
                e.printStackTrace();
            }
        }
        //step1 method  end

 

参考链接：

https://www.cnblogs.com/dolphin0520/p/3949310.html

https://blog.csdn.net/qq1137623160/article/details/79772505