中文词频统计及词云制作

1、中软国际华南区技术总监曾老师还会来上两次课，同学们希望曾老师讲些什么内容？（认真想一想回答）

a、关于这门课的相关工作经历

b、自己对于这门课的看法

2、中文分词

a、将要测试词频的文章放在一个TXT里

>>> fo = open('sanguo.txt','w')
>>> fo.write('''A Chinese company offering sex dolls for rent has withdrawn its services just days after launching.

Touch had begun offering five different sex doll types for daily or longer-term rent on Thursday in Beijing but quickly drew complaints and criticism.

The company said in a statement on Weibo it "sincerely apologised for the negative impact" of the concept.

But the firm stressed sex was "not vulgar" and said it would keep working towards more people enjoying it.

Touch told the BBC the rental service had operated for two days.

"We prepared ten dolls for the trial operation," a company spokesperson said via email, adding that they received very positive feedback from users.

"But it’s really hard in China," the firm wrote, saying there had been a lot of controversy with the police over the issue.

The company had offered the sex dolls for a daily fee of 298 yuan ($46), according to Chinese media.

The models on offer were marketed as Chinese, Korean and Russian women, with one also modelled on the movie character Wonder Woman, complete with a sword and shield.

In its Weibo statement, the firm said its original intention had been to make expensive silicone dolls more affordable but conceded that the service triggered a heated public debate.

The company also said it would pay out compensation to users worth double the amount they had paid as a deposit for reserving a doll.

The statement added that Touch would in future pay more attention to its "social duty", and would actively promote a "healthier and more harmonious sex lifestyle".

Aside from its short-lived rental offering, the firm sells an array of sex toys, including sex dolls.''')
1664
>>> fo.close()
>>> fr=open('kang.txt','r')
>>> fr.readline()
'A Chinese company offering sex dolls for rent has withdrawn its services just days after launching.\n'
>>> 

引用：

fo=open('sanguo.txt','r')
news = fo.read()

news= news.lower()
for i in ',./-_"":；':
    news=news.replace(i,' ')
words=news.split(' ')
exp = {'','the','\n\nthe','its','that','it','a','for','and','had','said','to','of','in','on','as','they','also','or','an','\n\nin','\n\n','\n\ntouch'}
dict={}
keys=set(words)-exp
for i in keys:
    dict[i]=words.count(i)

tj=list(dict.items())
tj.sort(key=lambda x:x[1],reverse=True)
for i in range(10):
    print(tj[i])
fo.close()

结果：

使用jieba库，进行中文词频统计，输出TOP20的词及出现次数

import jieba
aa=open('sanguo.txt','r').read()
bb=jieba.cut(aa)
news=list(bb)
dic={}
exp={'','。','“','”','、','，','\n','》','《'}
keys=set(news)-exp
for i in keys:
    dic[i]=news.count(i)
a=list(dic.items())
a.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    print(a[i])

import jieba
xs=open('xs.txt','w')
xs.write('''三国演义''')
xs.close()
fr=open('xs.txt','r',encoding='GBK').read()
zs=jieba.cut(fr)

dic={}
for z in zs:
    if len(z)==1:
        continue
    else:
        rez=z
        dic[z] = dic.get(z,0) + 1
keys=set(z)
a=sorted(dic.items())

tj=list(dic.items())
tj.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    print(tj[i])

>>> 
 RESTART: C:/Users/Administrator/AppData/Local/Programs/Python/Python36/zhongwen222.py 
Building prefix dict from the default dictionary ...
Loading model from cache C:\Users\ADMINI~1\AppData\Local\Temp\jieba.cache
Loading model cost 0.706 seconds.
Prefix dict has been built succesfully.
('刘备', 227)
('曹操', 210)
('一个', 206)
('张飞', 204)
('众人', 203)
('说道', 201)
('关羽', 174)
('诸葛亮', 136)
('自己', 132)
('他们', 132)
('心想', 127)
('赵云', 127)
('黄忠', 126)
('司马懿', 124)
('曹丕', 121)
('攻打', 114)
('兵弩', 113)
('不知', 110)
('不想', 106)
('哪里', 104)
>>>