Python实现的txt文件去重功能示例
更新时间:2018年07月07日 09:00:36 作者:人饭子 我要评论
这篇文章主要介绍了Python实现的txt文件去重功能,涉及Python针对txt文本文件的读写、字符串遍历、判断相关操作技巧,需要的朋友可以参考下
本文实例讲述了Python实现的txt文件去重功能。分享给大家供大家参考,具体如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
# -*- coding:utf-8 -*- #! python2 import shutil a = 0 readDir = "/Users/Administrator/Desktop/old.txt" #old writeDir = "/Users/Administrator/Desktop/new.txt" #new # txtDir = "/home/Administrator/Desktop/1" lines_seen = set () outfile = open (writeDir, "w" ) f = open (readDir, "r" ) for line in f: if line not in lines_seen: a + = 1 outfile.write(line) lines_seen.add(line) print (a) print ( '\n' ) outfile.close() print ( "success" ) |
其中old.tx如下:
www.jb51.net
www.baidu.com
www.sina.com.cn
www.jb51.net
www.google.com
www.sohu.com
www.jb51.net
www.163.com
运行后new.txt内容如下:
www.jb51.net
www.baidu.com
www.sina.com.cn
www.google.com
www.sohu.com
www.163.com