基于TF-IDF算法进行关键词抽取(jieba分词，lagoujobdetails.txt)

import jieba
import sys
import jieba.analyse
from optparse import OptionParser
tfidf = jieba.analyse.extract_tags

with open('lagoujobdatails.txt',encoding='utf-8') as f:
    tmp_line=f.read()
    jieba_cut=jieba.cut(tmp_line)
    ans=''.join(jieba_cut)
    text=ans
    k=tfidf(text)
    for i in k:
        print(i)

['爬虫', '熟悉', '抓取', '经验', '开发', '网页', '优先', '数据', '负责', 'spanclass', '精通', '分布式', '技术', '系统', '算法', '岗位职责', '工作', '网络', '设计', '编程']

 其中"爬虫‘，’spanclass','岗位职责‘是垃圾数据，数据清理时没清理完