List集合之元素和对象去重
1 List元素去重
1.1 移除List中指定某一元素
1.1.1 For循环移除
1.1.1.1 For移除不彻底问题
假如去除List中的Morning元素
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
for (int i = 0; i < l1.size(); i++) {
if("Morning".equals(l1.get(i))){
l1.remove(i);
}
}
System.out.println(l1);
}
执行结果：
[Midday, Evening, Night, Morning, Midday]
查看执行结果发现还有元素Morning没有去掉
1.1.1.2 用 i-- 解决问题
产生的原因就是：ArrayList 是一个数组元素的集合当删掉第一个元素Morning后，集合后面的元素会往前移动，但是此时 i 又指向下一个元素
解决办法：在 list.remove(i) 后 i--
修改后：
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
for (int i = 0; i < l1.size(); i++) {
if("Morning".equals(l1.get(i))){
l1.remove(i);
i--;
}
}
System.out.println(l1);
}
执行结果：
[Midday, Evening, Night, Midday]
1.1.1.3 倒序遍历移除元素
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
for (int i = l1.size()-1; i >=0 ; i--) {
if("Morning".equals(l1.get(i))){
l1.remove(i);
}
}
System.out.println(l1);
}
执行结果：
[Midday, Evening, Night, Midday]
1.1.2 ForEach移除
1.1.2.1 ConcurrentModificationException异常
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
for (String str:l1) {
if("Morning".equals(str:l1)){
l1.remove(i);
}
}
System.out.println(l1);
}
执行结果：
[Midday, Evening, Night, Midday]
会抛出 ConcurrentModificationException 异常
ConcurrentModificationException：当方法检测到对象的并发修改，但不允许这种修改时，抛出此异常。
原因：ArraayList 迭代器有个Itr，其内部有个属性expectedModCount。而集合有个fail-fast 快速失败检测机制，当进行remove()操作时，会比对expectedModCount是否与modCount相等，而前者一般不会改变，但 remove 操作会导致 modCount 发生改变。一旦两者不等，就会抛出ConcurrentModificationException异常。
解决办法：使用迭代器remove()方法
1.1.2.2 iterator遍历
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
Iterator<String> iterator = l1.iterator();
while (iterator.hasNext()){
String next = iterator.next();
if("Morning".equals(next)){
iterator.remove();
}
}
System.out.println(l1);
}
执行结果：
[Midday, Evening, Night, Midday]
1.2 移除List中重复元素
1.2.1 ForEach添加去重
这个是创建一个空的List，添加前判断下存在与否，不存在才添加，这样就抱着了元素不重复
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
List<String> l2 = new ArrayList<>();
for(String str:l1){
if(!l2.contains(str)){
l2.add(str);
}
}
System.out.println(l2);
}
执行结果：
[Midday, Evening, Night, Midday]
1.2.2 For双循环去重
从头和尾一起遍历，判断是否有相等，再移除
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
for (int i = 0; i < l1.size()-1; i++) {
for(int j=l1.size()-1;j>i;j--){
if(l1.get(j).equals(l1.get(i))){
l1.remove(j);
System.out.println("i=:"+i+",j=:"+j+l1);
}
}
}
System.out.println(l1);
}
执行结果：
i=:0,j=:6[Morning, Midday, Evening, Night, Morning, Morning, Midday]
i=:0,j=:5[Morning, Midday, Evening, Night, Morning, Midday]
i=:0,j=:4[Morning, Midday, Evening, Night, Midday]
i=:1,j=:4[Morning, Midday, Evening, Night]
[Morning, Midday, Evening, Night]
1.2.3 ForEach循环重复坐标去重
这个方式需要先复制出一个List2，再循环遍历List2，判断List中的元素首尾出现的坐标位置是否一致，若一致，则说明没有重复的，否则重复，并移除重复位置的元素
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
List<String> l2= new ArrayList<>(strings);
for(String s2:l2){
if(l1.indexOf(s2)!=l1.lastIndexOf(s2)){
l1.remove(l1.lastIndexOf(s2));
}
}
System.out.println(l1);
}
执行结果：
[Morning, Midday, Evening, Night]
1.2.4 Set去重
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
List<String> l2= new ArrayList<>(new HashSet(l1));
System.out.println(l2);
}
执行结果：
[Evening, Night, Morning, Midday]
这个方式去重重复元素最简单，但是不能保证顺序，可以把HashSet替换成LinkedHashSet，就可以保证原来的顺序了
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
List<String> l2= new ArrayList<>(new LinkedHashSet(l1));
System.out.println(l2);
}
执行结果：
[Morning, Midday, Evening, Night]
1.2.5 Stream去重
@Test
public void testRemoveDuplicate(){
List<String> strings = Arrays.asList("Morning", "Midday", "Evening", "Night","Morning","Morning","Morning","Midday");
List<String> l1= new ArrayList<>(strings);
List<String> collect = l1.stream().distinct().collect(Collectors.toList());
System.out.println(collect);
}
执行结果：
[Morning, Midday, Evening, Night]
2 List对象去重
2.1 使用的实体类
@Data
@NoArgsConstructor
@AllArgsConstructor
@String
public class User(){
private String userId;
private String userName;
private Integer userAge;
}
2.2 利用Collectors.toMap去重
2.2.1 toMap去重说明
List<User> userList = new ArrayList<>();
userList.add(new User("a", "xiaoming",12));
userList.add(new User("b", "xiaoming",13));
userList.add(new User("d", "xiaoming",15));
userList.add(new User("a", "xiaoming",14));
System.out.println("利用Collectors.toMap去重:");
//利用Collectors.toMap去重
userList.stream()
//或者这样写 Collectors.toMap(m -> m.getUserId(),
.collect(Collectors.toMap(User::getUserId,
Function.identity(), (oldValue, newValue) -> oldValue))
.values()
.stream()
.forEach(System.out::println); //打印
输出结果：
[{"userAge":12,"userId":"a","userName":"xiaoming"},
{"userAge":13,"userId":"b","userName":"xiaoming"},
{"userAge":15,"userId":"d","userName":"xiaoming"}]
其中，Collectors.toMap需要使用三个参数的版本，前两个参数一个是keyMapper函数一个是valueMapper函数的，用过toMap的都知道，去重的关键在于第三个参数BinaryOperator函数接口。
这个BinaryOperator函数接收两个参数，如上面代码，一个oldValue，一个newValue，字面意思，第一个旧值，第二个是新值。当stream构造Map时，会先调用Map的get方法获取该key对应节点的旧值，如果该值为null，则不会调用BinaryOperator函数，如果不为null，则会把获取的旧值与新值作为参数传给函数执行，然后把函数的返回值作为新值put到Map中。如果看不懂，请看源码吧
2.2.2 Funcion.identity()解释
Java 8允许在接口中加入具体方法。接口中的具体方法有两种，default方法和static方法，identity()就是Function接口的一个静态方法。
Function.identity()返回一个输出跟输入一样的Lambda表达式对象，等价于形如t -> t形式的Lambda表达式。
identity() 方法JDK源码如下：
static Function identity() {
return t -> t;
}
下面的代码中，Task::getTitle需要一个task并产生一个仅有一个标题的key
task -> task是一个用来返回自己的lambda表达式，上例中返回一个task
private static Map<String, Task> taskMap(List<Task> tasks) {
return tasks.stream().collect(toMap(Task::getTitle, task -> task));
}
可以使用Function接口中的默认方法identity来让上面的代码代码变得更简洁明了、传递开发者意图时更加直接，下面是采用identity函数的代码。
import static java.util.function.Function.identity;
private static Map<String, Task> taskMap(List<Task> tasks) {
return tasks.stream().collect(toMap(Task::getTitle, identity()));
}
2.3 利用Collectors.toCollection和TreeSet去重
List<User> userList = new ArrayList<>();
userList.add(new User("a", "xiaoming",12));
userList.add(new User("b", "xiaoming",13));
userList.add(new User("d", "xiaoming",15));
userList.add(new User("a", "xiaoming",14));
System.out.println("利用Collectors.toMap去重:");
//利用Collectors.toMap去重
userList.stream()
.collect(Collectors.toCollection(() ->
new TreeSet<>(Comparator.comparing(User::getUserId))))
.values()
.stream()
.forEach(System.out::println); //打印
输出结果：
[{"userAge":12,"userId":"a","userName":"xiaoming"},
{"userAge":13,"userId":"b","userName":"xiaoming"},
{"userAge":15,"userId":"d","userName":"xiaoming"}]
利用TreeSet原理去重，TreeSet内部使用的是TreeMap，使用指定Comparator比较元素，如果元素相同，则新元素代替旧元素，
TreeMap的put方法来放入元素的，有兴趣可以自己找源码
如果不想要返回TreeSet类型，那也可以使用Collectors.collectingAndThen转换成ArrayList，也可以用new ArrayList(set)，原理一样，如下：
List<User> distinctList = userList.stream()
.collect(Collectors.collectingAndThen(Collectors.toCollection(()
-> new TreeSet<>(Comparator.comparing(User::getUserId))), ArrayList::new));
注意：如果想根据两个或三个字段去重，可以在上述三个方法中的Comparator.comparing(User::getUserId))修改为Comparator.comparing(u -> u.getUserId() +"#" + u.getUserName() )) 这样就是根据两个字段去重，中间的#作用就是为了增加辨识度，也可以不加这个#，无论多少个字段去重只用在这里用+连接就可以了
