综合练习：词频统计

news = '''In the modern Chinese history of fine arts education, Xu Beihong (1895~ 1953), is to adopt the western art long modern painting master, prodromal type of art educators. Xu Beihong was born in rural areas, young the family was poor, his father Xu Dazhang is a village school teacher, good at flower, figure painting. At the age of 4 Xu Beihong began reading in school, and was interested in painting. At the age of 9, from whom he learned painting at the age of 10, has been able to make father's assistant. When busy season, and poor farming, working life, so that a child nurturance hard-working, simple style and honest and upright character.
When Xu Beihong was 13 years old in the year of great famine, father to walk the political arena, rely on vend word selling paintings. At the age of 17, his father contracted the illness, go from bad to worse family, a family of eight living burden, then fell to Xu Beihong 's shoulder. He started in the primary school, middle school teachers have to Shanghai pictures, such as selling paintings. 19 years old when his father died, more impoverished family.
In 1915, Xu Beihong again goes to Shanghai, the friends help," I always in time" ( at the time of the casino corner ) stay, hard creation, at the same time to night school to learn french. He drew a horse, send aesthetic books curator Gao Jianfu. Gao Jianfu and his brother Gao Qifeng read, appreciate his art. At this time, Xu Beihong was admitted to the aurora University, but without Qian Jinxue, but Gao Qifeng grant. Then met Kang Youwei, have the opportunity to observe the Kang collection of rubbings. And in the view of art and was influenced by Kang Youwei.
In 1917, Xu Beihong went to Japan to study art in Tokyo, fall back to Beijing, invited by Cai Yuanpei Ren Peking University descriptive research mentor, and met Chen Shiceng. Chen Shiceng is a poet Chen Sanyuan's eldest son, his grandfather is the renowned Hunan governor Chen Baozhen. He is the beginning of the twentieth Century China 's most outstanding art master, but also a bole type character, Qi Baishi fame is due to his strong push tours. After Chen Shiceng's death, Qi Baishi had the poem mourning, said: "I'm no king is back, you no I do not enter."
Xu Beihong 1919France, famous French painter Yang karma. Dangan early Sciences in nineteenth Century master Connor 's gate, later became one of the leaders of the French state. He is especially good at depicting the life of the Breton fishermen and farmers, the" blessing"," bread"," for" men such as painting as early as the Xu Beihong adore. Since then, Xu Beihong every week are almost always works to Hickey Road No. 65Dangan studio for advice, and joined the painter in the tea party, with Meniere, times to your conversation greatly benefit.
1921after going to Germany, Xu Beihong studied at the studio of painter CommScope, next to Paris. In 1925 by the Singapore home. Spring of second, he went to Paris, and went to Belgium Brussels Pro painting, travelled to Switzerland, Italy.
After returning home in 1927, served as a professor in the Department of Centre College, southern Shanghai art academy fine arts Dean, Peking University School of art director. In 1933with the Chinese modern painting in France, Germany, Belgium, Italy and the Soviet Union exhibition. During the war of resistance against Japan, the work to the Southeast Asia, India and other Southeast Asia exhibition, and all income donated to the refugees.
On the eve of the liberation, the Kuomintang government to send aircraft to pick up Xu Beihong and a number of well-known professor to go to Nanjing, by Xu Beihong refused. After the liberation, he was invited to China to attend the World Peace Congress, and served as president of the China Central Academy of Fine Arts, and was elected to the National Federation of standing committee, CPPCC representatives and the Chinese National Artists Association president. 1952is ill, his life and all the country donated collection of works creation. Died in 1953, only live 59 years old. The country is this great artist in Beijing established the Xu Beihong Memorial, saved his one thousand works. He composed works amounted to thousands of pieces, training and found a large number of excellent art talents.
Xu Beihong is good at Chinese painting, oil painting, especially fine sketch. His paintings are full of passion, skill is very high. Famous paintings are" my"" after the creek, the five hundred people", traditional Chinese painting has" nine party" Gao", great determination and courage"," Reunion", tokyo. Most can reflect the Xu Beihong personality, to express his thoughts and feelings than his horse. He on equine muscle, skeletal and expression dynamics, made long-term observational study, draw thousands of sketches. So he painted horse figure painting and earned, not arrogant, not trivial subtle, reinforcing bone Zhuang, of great momentum, spirit all foot. Some figures, lions, cats and other works, is the large amount of high quality. His painting" learning from nature, seek the truth " principle.'''

sep=''','?!:.()'''
exclude={'the','and','of','to','in'}

#将所有,.？！’:等分隔符全部替换为空格
for c in sep:
    news=news.replace(c,'')

#将所有大写转换为小写
wordList=news.lower().split()

# for w in wordList:
#     wordDict[w]=wordDict.get(w,0)+1
# for w in exclude:
#     del(wordDict[w])

#排除语法型词汇，代词、冠词、连词
wordDict = {}
wordSet=set(wordList)-exclude

for w in wordSet:
    wordDict[w]=wordList.count(w)

#排序
dictList=list(wordDict.items())
dictList.sort(key=lambda x:x[1],reverse=True)

输出词频最大TOP20
for i in range(20):
    print(dictList[i])

# print(dictList)
# for w in wordDict:
#     print(w,wordDict[w])

#将分析对象存为utf-8编码的文件，通过文件读取的方式获得词频分析内容。
f=open('news.txt','a')
for i in range(25):
    f.write(dictList[i][0]+''+str(dictList[1])+'\n')
f.close()

　　

 

import jieba
 
 
f = open('sanguoyanyi.txt', 'r',encoding='utf-8')
text = f.read()
f.close()
 
jieba.add_word('曹操')
jieba.add_word('诸葛亮')
jieba.add_word('孔明')
punctuation = '''，。‘’“”：；（）！？、 '''
a = {'的','\n','\u3000','曰','之','不','人','军','操','一','将',
     '大','马','来','德','有','于','下','兵','此',
     '玄','公','见','为','何','中','而','可','吾',
     '出','也','以','与','上','后','今','其','去',
     '日','明','言'}
for i in punctuation:
    text = text.replace(i, '')
print(list(jieba.cut(text)))
tempwords = list(jieba.cut(text))
print(tempwords)
count = {}
words = list(set(tempwords) - a)
print(words)
 
 
for i in range(0, len(words)):
    count[words[i]] = text.count(str(words[i]))
 
 
countList = list(count.items())
countList.sort(key=lambda x: x[1], reverse=True)
print(countList)
 
f = open('zzzCount.txt', 'a')
for i in range(20):
    f.write(countList[i][0] + ':' + str(countList[i][1]) + '\n')
f.close()