针对一百万以内数据高频变化、聚合统计、实时性要求的解决方案

背景需求：

 

 

解决方法程序：

package org.mem;

import java.util.*;

public class Test1 {



    public static void main(String[] args) {

        final Map<String,Moci> map = new HashMap<>();

        Date d1 = new Date();
        for(int i=0;i<1000000;i++){
            Random rand = new Random();
            Moci m = new Moci();
            m.setLat(rand.nextDouble() * 90);
            m.setLng(rand.nextDouble() * 180);
            m.setP1(rand.nextDouble());
            m.setP2(rand.nextDouble());
            m.setP3(rand.nextDouble());

            m.setType1(rand.nextInt(100));
            m.setType2(rand.nextInt(50));
            m.setVin(String.valueOf(i));
            map.put(String.valueOf(i),m);

        }

        System.out.println(new Date()+":"+d1);

        new Thread(){
            @Override
            public void run() {
                try {
                    int idx = 0;
                    Random rand = new Random();
                    while(true){

                        Moci m = map.get(String.valueOf(idx));

                        m.setLat(rand.nextDouble() * 90);
                        m.setLng(rand.nextDouble() * 180);
                        m.setP1(rand.nextDouble());
                        m.setP2(rand.nextDouble());
                        m.setP3(rand.nextDouble());

                        m.setType1(rand.nextInt(100));
                        m.setType2(rand.nextInt(50));
                        idx++;
                        if (idx %100001 == 0){
                            Thread.sleep(100);
                        }

                        if (idx >=1000000)
                            idx = 0;


                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }.start();


        for(int i=0;i<10;i++){

            new Thread(){
                @Override
                public void run() {
                    try{

                        while(true){



                            final Set<Map.Entry<String, Moci>> entries = map.entrySet();

                            final Iterator<Map.Entry<String, Moci>> it = entries.iterator();

                            double sp1 = 0;

                            Date d1 = new Date();

                            while(it.hasNext()){

                                Map.Entry<String, Moci> m = it.next();

                                Moci moci = m.getValue();



                                sp1 += moci.getP1();
                            }

                            System.out.println(Thread.currentThread()+"--"+(new Date().getTime() - d1.getTime())+"--"+d1+"----"+sp1);


                        }

                    }catch(Exception e){
                        e.printStackTrace();
                    }
                }
            }.start();


        }





//        while(true){
//            d1 = new Date();
//            for(int i=0;i<1000000;i++){
//
//                Random rand = new Random();
//                Moci m = map.get(String.valueOf(i));
//                m.setLat(rand.nextDouble() * 90);
//                m.setLng(rand.nextDouble() * 180);
//                m.setP1(rand.nextDouble());
//                m.setP2(rand.nextDouble());
//                m.setP3(rand.nextDouble());
//
//                m.setType1(rand.nextInt(100));
//                m.setType2(rand.nextInt(50));
//                m.setVin(String.valueOf(i));
//
//
//            }
//
//
//
//            System.out.println(new Date()+"====="+d1);
//
//            try {
//                Thread.sleep(1000);
//            } catch (InterruptedException e) {
//                e.printStackTrace();
//            }
//        }

    }


    public static class Moci{
        public volatile double lng;
        public volatile double lat;
        public String vin;
        public int type1;
        public int type2;
        public volatile double p1;
        public volatile double p2;
        public volatile double p3;

        public double getLng() {
            return lng;
        }

        public void setLng(double lng) {
            this.lng = lng;
        }

        public double getLat() {
            return lat;
        }

        public void setLat(double lat) {
            this.lat = lat;
        }

        public String getVin() {
            return vin;
        }

        public void setVin(String vin) {
            this.vin = vin;
        }

        public int getType1() {
            return type1;
        }

        public void setType1(int type1) {
            this.type1 = type1;
        }

        public int getType2() {
            return type2;
        }

        public void setType2(int type2) {
            this.type2 = type2;
        }

        public double getP1() {
            return p1;
        }

        public void setP1(double p1) {
            this.p1 = p1;
        }

        public double getP2() {
            return p2;
        }

        public void setP2(double p2) {
            this.p2 = p2;
        }

        public double getP3() {
            return p3;
        }

        public void setP3(double p3) {
            this.p3 = p3;
        }
    }
}

 

性能：

个人PC：

Thread[Thread-6,5,main]--253--Mon Nov 08 18:30:50 CST 2021----499760.4343774934
Thread[Thread-1,5,main]--231--Mon Nov 08 18:30:50 CST 2021----499750.3747759595
Thread[Thread-9,5,main]--250--Mon Nov 08 18:30:50 CST 2021----499787.23510245676
Thread[Thread-5,5,main]--236--Mon Nov 08 18:30:50 CST 2021----499750.72346231336
Thread[Thread-10,5,main]--175--Mon Nov 08 18:30:50 CST 2021----499856.9988989521
Thread[Thread-8,5,main]--212--Mon Nov 08 18:30:50 CST 2021----499842.2538334175
Thread[Thread-7,5,main]--198--Mon Nov 08 18:30:50 CST 2021----499856.9988989521
Thread[Thread-4,5,main]--202--Mon Nov 08 18:30:50 CST 2021----499880.970231565
Thread[Thread-3,5,main]--181--Mon Nov 08 18:30:50 CST 2021----499821.5141410899
Thread[Thread-2,5,main]--269--Mon Nov 08 18:30:50 CST 2021----499807.83603098075
Thread[Thread-6,5,main]--257--Mon Nov 08 18:30:50 CST 2021----499771.20976216684
Thread[Thread-1,5,main]--256--Mon Nov 08 18:30:51 CST 2021----499828.2464070499
Thread[Thread-8,5,main]--204--Mon Nov 08 18:30:51 CST 2021----499823.4357772191
Thread[Thread-7,5,main]--198--Mon Nov 08 18:30:51 CST 2021----499817.38504959363
Thread[Thread-4,5,main]--202--Mon Nov 08 18:30:51 CST 2021----499817.4829666695
Thread[Thread-9,5,main]--275--Mon Nov 08 18:30:51 CST 2021----499825.024023482
Thread[Thread-5,5,main]--266--Mon Nov 08 18:30:51 CST 2021----499790.3105707482
Thread[Thread-3,5,main]--224--Mon Nov 08 18:30:51 CST 2021----499823.9917041906
Thread[Thread-10,5,main]--267--Mon Nov 08 18:30:51 CST 2021----499806.9031451534
Thread[Thread-7,5,main]--173--Mon Nov 08 18:30:51 CST 2021----499684.4828482913

服务器上：

Thread[Thread-10,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499707.98572607624
Thread[Thread-1,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499707.98572607624
Thread[Thread-2,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499707.98572607624
Thread[Thread-3,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499707.98572607624
Thread[Thread-6,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499708.74318016105
Thread[Thread-4,5,main]--41--Mon Nov 08 18:32:34 CST 2021----499827.66904025123
Thread[Thread-10,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499844.09975824325
Thread[Thread-7,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499843.9539189926
Thread[Thread-3,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499851.4043842572
Thread[Thread-2,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499851.76995954016
Thread[Thread-8,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499843.9539189926
Thread[Thread-5,5,main]--37--Mon Nov 08 18:32:34 CST 2021----499851.4043842572
Thread[Thread-9,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499844.09975824325
Thread[Thread-1,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499843.9539189926
Thread[Thread-6,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499851.4043842572
Thread[Thread-3,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-10,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-1,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-9,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-8,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-6,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-2,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-4,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-7,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-5,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-5,5,main]--35--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-2,5,main]--35--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-6,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-3,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-10,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-4,5,main]--35--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-1,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-7,5,main]--35--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-8,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741
Thread[Thread-9,5,main]--36--Mon Nov 08 18:32:34 CST 2021----499808.3218557741

 

优点：

  • 性能高、并发性强

  • 支持水平扩展

缺点：

  • 自定义查询条件

  • 需要考虑线程安全

  • 元数据更改，要及时对应

 

posted on 2021-11-08 18:35  李雷  阅读(192)  评论(0)  编辑  收藏  举报

导航