Java8 stream-List去重distinct、和指定字段去重

使用Java8的Stream的distinct方法去重，我们的对象需要实现hashcode()和equals()方法。

把学生类修改后如下：

public class Student {
    private Long id;
    private String name;
    private int age;
    private Double height;

    public Student(Long id, String name, int age, Double height) {
        this.id = id;
        this.name = name;
        this.age = age;
        this.height = height;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public int getAge() {
        return age;
    }

    public void setAge(int age) {
        this.age = age;
    }

    public Double getHeight() {
        return height;
    }

    public void setHeight(Double height) {
        this.height = height;
    }

    @Override
    public String toString() {
        return "Student{" +
                "id=" + id +
                ", name='" + name + '\'' +
                ", age=" + age +
                ", height=" + height +
                '}';
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Student student = (Student) o;
        return age == student.age &&
                Objects.equals(id, student.id) &&
                Objects.equals(name, student.name) &&
                Objects.equals(height, student.height);
    }

    @Override
    public int hashCode() {
        return Objects.hash(id, name, age, height);
    }
}

测试例子：

import com.top.test.dto.Student;

import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

public class ListStreamDistinctTest {
    public static void main(String[] args) {
        // 一个集合中放入4个学生对象
        List<Student> list = new ArrayList<>();
        list.add(new Student(10002L, "ZhangSan", 18, 175.2));
        list.add(new Student(10001L, "LiSi", 19, 175.2));
        list.add(new Student(10004L, "Peter", 19, 170.8));
        list.add(new Student(10004L, "Peter", 19, 170.8));

        // 按整个个对象去重
        System.out.println("整个对象去重：");
        list.stream().distinct()
                .forEach(System.out::println);

        // 指定某个字段去重
        System.out.println("指定age属性去重（方法一）：");
        list.stream().filter(distinctByKey1(s -> s.getAge()))
                .forEach(System.out::println);

        // 方法二（用循环)
        System.out.println("指定age属性去重（方法二）：");
        TreeSet<Student> students = new TreeSet<>(Comparator.comparing(s -> s.getAge()));
        for (Student student : list) {
            students.add(student);
        }
        new ArrayList<>(students)
                .forEach(System.out::println);

        // 方法三：（是方法二的变形），此方法来源于：https://blog.csdn.net/qq_28988969/article/details/81119587
        System.out.println("指定age属性去重（方法三）：");
        list.stream().collect(Collectors.collectingAndThen(Collectors.toCollection(() -> new TreeSet<>(Comparator.comparing(s -> s.getAge()))), ArrayList::new))
                .forEach(System.out::println);

    }

    // 次方法来源于：https://blog.csdn.net/haiyoung/article/details/80934467
    static <T> Predicate<T> distinctByKey1(Function<? super T, ?> keyExtractor) {
        Map<Object, Boolean> seen = new ConcurrentHashMap<>();
        return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
    }
}

执行结果：

整个对象去重：
Student{id=10002, name='ZhangSan', age=18, height=175.2}
Student{id=10001, name='LiSi', age=19, height=175.2}
Student{id=10004, name='Peter', age=19, height=170.8}
指定age属性去重（方法一）：
Student{id=10002, name='ZhangSan', age=18, height=175.2}
Student{id=10001, name='LiSi', age=19, height=175.2}
指定age属性去重（方法二）：
Student{id=10002, name='ZhangSan', age=18, height=175.2}
Student{id=10001, name='LiSi', age=19, height=175.2}
指定age属性去重（方法三）：
Student{id=10002, name='ZhangSan', age=18, height=175.2}
Student{id=10001, name='LiSi', age=19, height=175.2}

其中方法三使用起来比较方便，还可以借助排序规则（数据库排好序或者stream排序 ）去重，例如取最新一条数据。

ArrayList<ProductProcessDrawbackDto> collect = records1.stream().collect(Collectors.collectingAndThen(
                        Collectors.toCollection(() -> new TreeSet<>(
                                Comparator.comparing(
                                        ProductProcessDrawbackDto::getId))), ArrayList::new));

其中records1是处理的对象，改对象的list集合，collect是处理后返回的结果

其中的ProductProcessDrawbackDto是处理的list中每一个对象，id是判断是否重复的条件（去除id相同的重复元素，只保留一条）

多个字段或者多个条件去重

 ArrayList<PatentDto> collect1 = patentDtoList.stream().collect(Collectors.collectingAndThen(
                Collectors.toCollection(() -> new TreeSet<>(
                        Comparator.comparing(p->p.getPatentName() + ";" + p.getLevel()))), ArrayList::new)

 

 

参考文章：

https://blog.csdn.net/kurry4ever_/article/details/109638367