【Python】Python获取TIOBE排行榜 && 数据可视化

✨TIOBE

TIOBE排行榜是根据互联网上有经验的程序员、课程和第三方厂商的数量，并使用搜索引擎（如Google、Bing、Yahoo!）以及Wikipedia、Amazon、YouTube统计出排名数据，只是反映某个编程语言的热门程度，并不能说明一门编程语言好不好，或者一门语言所编写的代码数量多少。

TIOBE开发语言排行榜每月更新一次，依据的指数是基于世界范围内的资深软件工程师和第三方供应商提供，其结果作为当前业内程序开发语言的流行使用程度的有效指标。

该指数可以用来检阅开发者的编程技能能否跟上趋势，或是否有必要作出战略改变，以及什么编程语言是应该及时掌握的。观察认为，该指数反应的虽并非当前最流行或应用最广的语言，但对世界范围内开发语言的走势仍具有重要参考意义。

✨Python已位居TIOBE榜首

此博客为2.0版本

1.0版本详情可见：

【Python】Python获取TIOBE排行榜 绘制图表及词云 - 双份浓缩馥芮白 - 博客园 (cnblogs.com)

由于写第一篇博客时刚学Python

还没有学习数据分析相关模块（ numpy、pandas等）

因此数据清洗部分写的非常繁琐

虽然2.0版本也懒得改就是了

时隔一年多代码不是还能跑

又不是不能用

2022.7.18重新爬取时 Python已位于榜首！！

七月标题：TIOBE 指数中的夏季

上个月，TIOBE 指数没有太大变化。目前排名前 4 位的语言 Python、C、Java 和 C++ 的市场份额合计接近 50%。Fortran 和 Matlab 以 Lua 和 Prolog 为代价再次进入前 20 名。今年已经过半，到目前为止，TIOBE 2022 年度语言的最大候选者是 Python、C、C++ 和 C#。我们预计 Rust、Dart、Kotlin 或 TypeScript 等任何相对较新和热门的语言都不会进入前 20 名。让我们期待 8 月份有更多激动人心的消息。——保罗·詹森 CEO TIOBE Software

✨目标网站

https://www.tiobe.com/tiobe-index/

✨目标数据

TIOBE Index for July 2022

TIOBE Programming Community Index

Very Long Term History

✨需求分析

• 利用Python解析页面信息

• 数据分析

• 数据可视化

• 保存相关榜单信息

✨数据可视化展示

TOP20 编程语言使用率

TOP20 编程语言变化趋势

各类编程语言长期排名变化情况

各类编程语言使用率长期变化情况

TOP20 编程语言使用率词云

✨前端页面分析

分别针对如下目标数据分析前端页面：

• TIOBE Index for July 2022
• TIOBE Programming Community Index
• Very Long Term History

TIOBE Index for July 2022

此部分数据位于 id为top20 class为table table-striped table-top20的节点内

TIOBE Programming Community Index

此部分数据直接分析页面元素并不能直接得到

通过查看页面源代码

可以发现这张动态的曲线实际上是在前端利用JS动态渲染的

而我们需要的数据在series中

整个曲线的源数据为JSON格式

Very Long Term History

此部分数据位于 id为VLTH class为table table-striped的节点内

✨代码实现

导入模块

import requests
from requests.exceptions import RequestException
from lxml import etree
import matplotlib.pyplot as plt
import wordcloud
from bs4 import BeautifulSoup
import re
import demjson

这里pip install demjson如果遇到报错"error in demjson setup command: use_2to3 is invalid."

解决方案如下

pip install "setuptools<58.0.0"
pip install demjson

或者新建个conda环境试试

重装下python/anaconda/miniconda试试

数据获取

# 获取html
def getHTMLText(url):
    headers = {
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36'
    }
    try:
        r = requests.get(url, headers=headers)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        return r.text
    except RequestException as e:
        print('error', e)


# 解析html
def analyzeRankHtml(html):
    html = etree.HTML(html, etree.HTMLParser())
    uhead = html.xpath('//table[contains(@class,"table-top20")]/thead/tr//text()')
    ulist = html.xpath('//table[contains(@class,"table-top20")]/tbody/tr//text()')
    for i in range(7):
        uhead.remove('\n')
    uhead.pop(2)
    # print(uhead)
    # print(ulist)
    return uhead, ulist


# 获取json
def getJson(url):
    response = requests.get(url)
    soup = BeautifulSoup(response.text, features='html.parser')
    parseText = soup.body
    # print(parseText)
    # 定义查找正则表达式，series:开头，中间任意匹配，}\);结尾的文本，后边多一个?表示懒惰模式
    pattern = re.compile(r"series:(.*?)}\);", re.MULTILINE | re.DOTALL)
    # 正则表达式匹配查找，group(0) 是获取取得的字符串整体，group(1)是取出括号里面我们要匹配的内容
    languageRank = pattern.search(str(parseText)).group(1)
    # print(languageRank)
    languageRank = languageRank.replace('\n', '').replace('\r', '').replace(" ", "").replace("\t", "")
    # print(languageRank)
    # 定义替换正则表达式，series:开头，中间任意匹配，}\);结尾的文本
    # JavaScript Date.UTC 表示月份的整数，介于 0 ~ 11。
    languageRank = languageRank.replace('Date.UTC(', '"').replace(')', '"')
    jsonRank = demjson.decode(languageRank)
    # printList(jsonRank)
    # print(jsonRank[0].get("name"))
    # print(jsonRank[0].get("data")[10])
    # 接下来继续处理与展示
    return jsonRank

数据清洗

写的时候还没有学习数据分析相关模块（ numpy、pandas等）

数据清洗部分写的非常繁琐

# 数据清洗 1
def dataPreTreat1(uinfo: list) -> list:
    language = []
    ratings = []
    change = []

    for i in range(len(uinfo)):
        if i % 5 == 2:
            language.append(uinfo[i])
        elif i % 5 == 3:
            ratings.append(uinfo[i])
        elif i % 5 == 4:
            change.append(uinfo[i])
        else:
            pass

    ratings_data = []
    for i in ratings:
        ratings_data.append(float(i.rstrip('%')) / 100)

    change_data = []
    for i in change:
        change_data.append(float(i.rstrip('%')) / 100)

    return language, ratings_data, change_data


# 数据清洗 2
def dataPreTreat2(vhead, vlth: list) -> list:
    month = []
    for i in range(len(vhead)):
        month.append(vhead[i])
    month.pop(0)
    month.sort()
    # print(month)

    vlth = [40 if i == '-' else i for i in vlth]
    tmp = []
    data = []
    for i in range(13):
        for j in range(9):
            tmp.append(vlth.pop(0))
        # print(tmp)
        data.append(tmp)
        tmp = []
    # print(data)
    language = []
    for i in data:
        language.append(i.pop(0))

    data = [[float(j) for j in i] for i in data]
    # print(language)
    # print(month)
    # print(data)

    return language, month, data


# 数据清洗 3
def preTreat3(jsonRank):
    language = []
    for i in range(len(jsonRank)):
        language.append(jsonRank[i].get('name'))


    tmp = []
    for i in range(len(jsonRank)):
        tmp.append(jsonRank[i].get('data'))
    # printList(tmp)

    date = []  # 存放每种语言的date的临时变量
    dateall = []  # 存放所有语言的date

    value = []  # 存放每种语言的value的临时变量
    valueall = []  # 存放所有语言的value

    # JavaScript Date.UTC 表示月份的整数，介于 0 ~ 11。
    # 需要把所有日期格式中的 month + 1

    for i in range(len(tmp)):
        # print(tmp[i])
        for j in range(len(tmp[i])):
            # print((tmp[i][j][0]), end=',')
            date.append((tmp[i][j][0]).replace(',', '-'))  # 标准化日期格式
            value.append(tmp[i][j][1])
        dateall.append(date)
        valueall.append(value)
        # 临时变量清零
        date = []
        value = []

    # printList(dateall)
    # printList(valueall)

    # print(len(dateall))  # 没错都是十种语言
    # print(len(valueall))  # 没错都是十种语言

    time = dateall
    value = valueall

    return language, time, value

数据显示及保存

# 打印列表
def printList(ls):
    for i in ls:
        print(i)


# 打印排行信息
def printRank(uhead, ulist):
    for i in range(len(uhead)):
        if i == 2:
            print('{:^25}\t'.format(uhead[i]), end='')
        else:
            print('{:^10}\t'.format(uhead[i]), end='')
    print()

    tplt = "{0:^10}\t{1:^10}\t{2:^25}\t{3:^10}\t{4:^10}"

    j = 0
    for i in range(20):
        if i == 0:
            print(tplt.format(ulist[0], ulist[1], ulist[2], ulist[3], ulist[4]))
        else:
            print(tplt.format(ulist[j], ulist[j + 1], ulist[j + 2], ulist[j + 3], ulist[j + 4]))
        j += 5
    print()


# 保存TOP20编程语言情况
def saveRank(uhead, ulist):
    with open('TOP20 编程语言情况.txt', 'w+') as f:
        for i in range(len(uhead)):
            if i == 2:
                f.write('{:^25}\t'.format(uhead[i]))
            else:
                f.write('{:^10}\t'.format(uhead[i]))
        f.write('\n')

        tplt = "{0:^10}\t{1:^10}\t{2:^25}\t{3:^10}\t{4:^10}\t{5:}"
        j = 0
        for i in range(20):
            if i == 0:
                f.write(tplt.format(ulist[0], ulist[1], ulist[2], ulist[3], ulist[4], '\n'))
            else:
                f.write(tplt.format(ulist[j], ulist[j + 1], ulist[j + 2], ulist[j + 3], ulist[j + 4], '\n'))
            j += 5

    print('TOP20 编程语言情况.txt 已保存...\n')


# 获取各类编程语言长期排名情况
def analyzeVlthHtml(html):
    html = etree.HTML(html, etree.HTMLParser())
    vlth = html.xpath('//table[contains(@id,"VLTH")]//tr//text()')
    vhead = []
    for i in range(9):
        vhead.append(vlth.pop(0))
    # print(vhead)
    # print(vlth)
    return vhead, vlth


# 打印长期排行信息
def printLongTermHistoryRank(vhead, vlth):
    for i in range(len(vhead)):
        if i == 0:
            print('{0:^25}\t'.format(vhead[i]), end='')
        else:
            print('{:^5}\t'.format(vhead[i]), end='')
    print()

    tplt = "{0:^25}\t{1:^5}\t{2:^5}\t{3:^5}\t{4:^5}\t{5:^5}\t{6:^5}\t{7:^5}\t{8:^5}"
    j = 0
    for i in range(13):
        if i == 0:
            print(tplt.format(vlth[0], vlth[1], vlth[2], vlth[3], vlth[4], vlth[5], vlth[6], vlth[7], vlth[8]))
        else:
            print(tplt.format(vlth[j], vlth[j + 1], vlth[j + 2], vlth[j + 3], vlth[j + 4], vlth[j + 5], vlth[j + 6],
                              vlth[j + 7], vlth[j + 8]))
        j += 9
    print()


# 保存各类编程语言长期排名变化情况
def saveLongTermHistoryRank(vhead, vlth):
    with open('各类编程语言长期排名变化情况.txt', 'w+') as f:
        for i in range(len(vhead)):
            if i == 0:
                f.write('{0:^25}\t'.format(vhead[i]))
            else:
                f.write('{:^5}\t'.format(vhead[i]))
        f.write('\n')

        tplt = "{0:^25}\t{1:^5}\t{2:^5}\t{3:^5}\t{4:^5}\t{5:^5}\t{6:^5}\t{7:^5}\t{8:^5}\t{9:}"

        j = 0
        for i in range(13):
            if i == 0:
                f.write(tplt.format(vlth[0], vlth[1], vlth[2], vlth[3], vlth[4], vlth[5], vlth[6], vlth[7], vlth[8], '\n'))
            else:
                f.write(tplt.format(vlth[j], vlth[j + 1], vlth[j + 2], vlth[j + 3], vlth[j + 4], vlth[j + 5], vlth[j + 6],
                                vlth[j + 7], vlth[j + 8], '\n'))
            j += 9

    print('各类编程语言长期排名情况.txt 已保存...\n')

数据可视化

# 解决title中文乱码
def handleFont():
    # plt.rcParams['font.sans-serif'] = ['SimHei']  # Windows
    plt.rcParams['font.sans-serif'] = ['Arial Unicode MS']  # macOS
    plt.rcParams['axes.unicode_minus'] = False


# 绘制图表
def drawPic(ratings_data, change_data):
    handleFont()
    print('开始绘制->TOP20 编程语言使用率...\n')

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')

    plt.figure(figsize=(12, 8))
    plt.pie(x=ratings_data, explode=ratings_data, labels=language, autopct='%1.1f%%', normalize=True, shadow=False,
            startangle=150)
    plt.title("TOP20 编程语言使用率")
    plt.legend(loc='best', fontsize='x-small')
    plt.savefig('TOP20 编程语言使用率.png')
    plt.show()
    print('TOP20 编程语言使用率.png 已保存...\n')

    print('开始绘制->TOP20 编程语言变化趋势...\n')

    color = []
    for i in change_data:
        if i >= 0:
            color.append('red')
        else:
            color.append('green')

    plt.figure(figsize=(16, 10))
    bar_width = 0.5
    plt.bar(language, change_data, bar_width,
            color=color,
            align="center", label="change", alpha=0.5)
    plt.xlabel("Programming Language")
    plt.ylabel("Percentage")
    plt.xticks(rotation=30)
    plt.title("TOP20 编程语言变化趋势")
    plt.savefig('TOP20 编程语言变化趋势.png')
    plt.show()
    print('TOP20 编程语言变化趋势.png 已保存...\n')


# 绘制词云图
def drawWordcloud(language, ratings):
    handleFont()
    print('开始绘制词云图...\n')
    ratings = [i * 1000 for i in ratings]
    # print(ratings)
    words = []
    for i in range(len(language)):
        for j in range(int(ratings[i])):
            words.append(language[i])

    txt = ",".join(words)
    # print(txt)
    w = wordcloud.WordCloud(width=800, height=600, background_color="white",
                            max_words=15,
                            collocations=False)
    w.generate(txt)
    plt.imshow(w)
    plt.axis('off')  # 是否显示坐标轴
    plt.savefig("TOP20 编程语言使用率词云.png")
    plt.show()
    # w.to_file("TOP20 编程语言使用率词云.png")
    print('TOP20 编程语言使用率词云.png 已保存...\n')


# 绘制折线图
def drawLineChart(language, month, data):
    handleFont()
    print('开始绘制->各类编程长期排名情况...\n')

    # print(month)
    # print(language)

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')

    for i in range(len(data)):
        # print(data[i])
        plt.plot(month, data[i], label=language[i])
    plt.title("各类编程语言长期排名变化情况")
    plt.legend(loc='best', fontsize='x-small')
    plt.ylim(0, 40)  # 设置y轴
    plt.gca().invert_yaxis()  # y轴逆序
    plt.savefig('各类编程语言长期排名变化情况.png')
    plt.show()
    print('各类编程语言长期排名变化情况.png 已保存...\n')


# 绘制 各类编程语言长期使用率变化情况
def drawTPCI(language, time, value):
    print('各类编程语言使用率长期变化情况...\n')
    # 解决title中文乱码
    # plt.rcParams['font.sans-serif'] = ['SimHei']  # Windows
    plt.rcParams['font.sans-serif'] = ['Arial Unicode MS']  # macOS
    plt.rcParams['axes.unicode_minus'] = False

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')
    plt.figure(figsize=(10, 4.8))
    # printList(time)
    # for i in time:
    #     print(len(i))

    for i in range(len(language)):
        plt.plot(time[i], value[i], label=language[i])
    plt.title("各类编程语言使用率长期变化情况")
    plt.legend(loc='best', fontsize='x-small')
    # 坐标轴映射
    plt.xticks(range(0, 235, 22), range(2000, 2022, 2))
    plt.xlabel('时间', multialignment='center')
    plt.ylabel('使用率(%)', multialignment='center')
    # plt.ylim(0, 30)  # 设置y轴
    plt.savefig('各类编程语言使用率长期变化情况.png')
    plt.show()
    print('各类编程语言使用率长期变化情况.png 已保存...\n')

完整实现代码

import requests
from requests.exceptions import RequestException
from lxml import etree
import matplotlib.pyplot as plt
import wordcloud
from bs4 import BeautifulSoup
import re
import demjson


# 获取html
def getHTMLText(url):
    headers = {
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36'
    }
    try:
        r = requests.get(url, headers=headers)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        return r.text
    except RequestException as e:
        print('error', e)


# 解析html
def analyzeRankHtml(html):
    html = etree.HTML(html, etree.HTMLParser())
    uhead = html.xpath('//table[contains(@class,"table-top20")]/thead/tr//text()')
    ulist = html.xpath('//table[contains(@class,"table-top20")]/tbody/tr//text()')
    for i in range(7):
        uhead.remove('\n')
    uhead.pop(2)
    # print(uhead)
    # print(ulist)
    return uhead, ulist


# 获取json
def getJson(url):
    response = requests.get(url)
    soup = BeautifulSoup(response.text, features='html.parser')
    parseText = soup.body
    # print(parseText)
    # 定义查找正则表达式，series:开头，中间任意匹配，}\);结尾的文本，后边多一个?表示懒惰模式
    pattern = re.compile(r"series:(.*?)}\);", re.MULTILINE | re.DOTALL)
    # 正则表达式匹配查找，group(0) 是获取取得的字符串整体，group(1)是取出括号里面我们要匹配的内容
    languageRank = pattern.search(str(parseText)).group(1)
    # print(languageRank)
    languageRank = languageRank.replace('\n', '').replace('\r', '').replace(" ", "").replace("\t", "")
    # print(languageRank)
    # 定义替换正则表达式，series:开头，中间任意匹配，}\);结尾的文本
    # JavaScript Date.UTC 表示月份的整数，介于 0 ~ 11。
    languageRank = languageRank.replace('Date.UTC(', '"').replace(')', '"')
    jsonRank = demjson.decode(languageRank)
    # printList(jsonRank)
    # print(jsonRank[0].get("name"))
    # print(jsonRank[0].get("data")[10])
    # 接下来继续处理与展示
    return jsonRank


# 打印列表
def printList(ls):
    for i in ls:
        print(i)


# 打印排行信息
def printRank(uhead, ulist):
    for i in range(len(uhead)):
        if i == 2:
            print('{:^25}\t'.format(uhead[i]), end='')
        else:
            print('{:^10}\t'.format(uhead[i]), end='')
    print()

    tplt = "{0:^10}\t{1:^10}\t{2:^25}\t{3:^10}\t{4:^10}"

    j = 0
    for i in range(20):
        if i == 0:
            print(tplt.format(ulist[0], ulist[1], ulist[2], ulist[3], ulist[4]))
        else:
            print(tplt.format(ulist[j], ulist[j + 1], ulist[j + 2], ulist[j + 3], ulist[j + 4]))
        j += 5
    print()


# 保存TOP20编程语言情况
def saveRank(uhead, ulist):
    with open('TOP20 编程语言情况.txt', 'w+') as f:
        for i in range(len(uhead)):
            if i == 2:
                f.write('{:^25}\t'.format(uhead[i]))
            else:
                f.write('{:^10}\t'.format(uhead[i]))
        f.write('\n')

        tplt = "{0:^10}\t{1:^10}\t{2:^25}\t{3:^10}\t{4:^10}\t{5:}"
        j = 0
        for i in range(20):
            if i == 0:
                f.write(tplt.format(ulist[0], ulist[1], ulist[2], ulist[3], ulist[4], '\n'))
            else:
                f.write(tplt.format(ulist[j], ulist[j + 1], ulist[j + 2], ulist[j + 3], ulist[j + 4], '\n'))
            j += 5

    print('TOP20 编程语言情况.txt 已保存...\n')


# 获取各类编程语言长期排名情况
def analyzeVlthHtml(html):
    html = etree.HTML(html, etree.HTMLParser())
    vlth = html.xpath('//table[contains(@id,"VLTH")]//tr//text()')
    vhead = []
    for i in range(9):
        vhead.append(vlth.pop(0))
    # print(vhead)
    # print(vlth)
    return vhead, vlth


# 打印长期排行信息
def printLongTermHistoryRank(vhead, vlth):
    for i in range(len(vhead)):
        if i == 0:
            print('{0:^25}\t'.format(vhead[i]), end='')
        else:
            print('{:^5}\t'.format(vhead[i]), end='')
    print()

    tplt = "{0:^25}\t{1:^5}\t{2:^5}\t{3:^5}\t{4:^5}\t{5:^5}\t{6:^5}\t{7:^5}\t{8:^5}"
    j = 0
    for i in range(13):
        if i == 0:
            print(tplt.format(vlth[0], vlth[1], vlth[2], vlth[3], vlth[4], vlth[5], vlth[6], vlth[7], vlth[8]))
        else:
            print(tplt.format(vlth[j], vlth[j + 1], vlth[j + 2], vlth[j + 3], vlth[j + 4], vlth[j + 5], vlth[j + 6],
                              vlth[j + 7], vlth[j + 8]))
        j += 9
    print()


# 保存各类编程语言长期排名变化情况
def saveLongTermHistoryRank(vhead, vlth):
    with open('各类编程语言长期排名变化情况.txt', 'w+') as f:
        for i in range(len(vhead)):
            if i == 0:
                f.write('{0:^25}\t'.format(vhead[i]))
            else:
                f.write('{:^5}\t'.format(vhead[i]))
        f.write('\n')

        tplt = "{0:^25}\t{1:^5}\t{2:^5}\t{3:^5}\t{4:^5}\t{5:^5}\t{6:^5}\t{7:^5}\t{8:^5}\t{9:}"

        j = 0
        for i in range(13):
            if i == 0:
                f.write(tplt.format(vlth[0], vlth[1], vlth[2], vlth[3], vlth[4], vlth[5], vlth[6], vlth[7], vlth[8], '\n'))
            else:
                f.write(tplt.format(vlth[j], vlth[j + 1], vlth[j + 2], vlth[j + 3], vlth[j + 4], vlth[j + 5], vlth[j + 6],
                                vlth[j + 7], vlth[j + 8], '\n'))
            j += 9

    print('各类编程语言长期排名情况.txt 已保存...\n')


# 数据清洗 1
def dataPreTreat1(uinfo: list) -> list:
    language = []
    ratings = []
    change = []

    for i in range(len(uinfo)):
        if i % 5 == 2:
            language.append(uinfo[i])
        elif i % 5 == 3:
            ratings.append(uinfo[i])
        elif i % 5 == 4:
            change.append(uinfo[i])
        else:
            pass

    ratings_data = []
    for i in ratings:
        ratings_data.append(float(i.rstrip('%')) / 100)

    change_data = []
    for i in change:
        change_data.append(float(i.rstrip('%')) / 100)

    return language, ratings_data, change_data


# 数据清洗 2
def dataPreTreat2(vhead, vlth: list) -> list:
    month = []
    for i in range(len(vhead)):
        month.append(vhead[i])
    month.pop(0)
    month.sort()
    # print(month)

    vlth = [40 if i == '-' else i for i in vlth]
    tmp = []
    data = []
    for i in range(13):
        for j in range(9):
            tmp.append(vlth.pop(0))
        # print(tmp)
        data.append(tmp)
        tmp = []
    # print(data)
    language = []
    for i in data:
        language.append(i.pop(0))

    data = [[float(j) for j in i] for i in data]
    # print(language)
    # print(month)
    # print(data)

    return language, month, data


# 数据清洗 3
def preTreat3(jsonRank):
    language = []
    for i in range(len(jsonRank)):
        language.append(jsonRank[i].get('name'))


    tmp = []
    for i in range(len(jsonRank)):
        tmp.append(jsonRank[i].get('data'))
    # printList(tmp)

    date = []  # 存放每种语言的date的临时变量
    dateall = []  # 存放所有语言的date

    value = []  # 存放每种语言的value的临时变量
    valueall = []  # 存放所有语言的value

    # JavaScript Date.UTC 表示月份的整数，介于 0 ~ 11。
    # 需要把所有日期格式中的 month + 1

    for i in range(len(tmp)):
        # print(tmp[i])
        for j in range(len(tmp[i])):
            # print((tmp[i][j][0]), end=',')
            date.append((tmp[i][j][0]).replace(',', '-'))  # 标准化日期格式
            value.append(tmp[i][j][1])
        dateall.append(date)
        valueall.append(value)
        # 临时变量清零
        date = []
        value = []

    # printList(dateall)
    # printList(valueall)

    # print(len(dateall))  # 没错都是十种语言
    # print(len(valueall))  # 没错都是十种语言

    time = dateall
    value = valueall

    return language, time, value


# 解决title中文乱码
def handleFont():
    # plt.rcParams['font.sans-serif'] = ['SimHei']  # Windows
    plt.rcParams['font.sans-serif'] = ['Arial Unicode MS']  # macOS
    plt.rcParams['axes.unicode_minus'] = False


# 绘制图表
def drawPic(ratings_data, change_data):
    handleFont()
    print('开始绘制->TOP20 编程语言使用率...\n')

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')

    plt.figure(figsize=(12, 8))
    plt.pie(x=ratings_data, explode=ratings_data, labels=language, autopct='%1.1f%%', normalize=True, shadow=False,
            startangle=150)
    plt.title("TOP20 编程语言使用率")
    plt.legend(loc='best', fontsize='x-small')
    plt.savefig('TOP20 编程语言使用率.png')
    plt.show()
    print('TOP20 编程语言使用率.png 已保存...\n')

    print('开始绘制->TOP20 编程语言变化趋势...\n')

    color = []
    for i in change_data:
        if i >= 0:
            color.append('red')
        else:
            color.append('green')

    plt.figure(figsize=(16, 10))
    bar_width = 0.5
    plt.bar(language, change_data, bar_width,
            color=color,
            align="center", label="change", alpha=0.5)
    plt.xlabel("Programming Language")
    plt.ylabel("Percentage")
    plt.xticks(rotation=30)
    plt.title("TOP20 编程语言变化趋势")
    plt.savefig('TOP20 编程语言变化趋势.png')
    plt.show()
    print('TOP20 编程语言变化趋势.png 已保存...\n')


# 绘制词云图
def drawWordcloud(language, ratings):
    handleFont()
    print('开始绘制词云图...\n')
    ratings = [i * 1000 for i in ratings]
    # print(ratings)
    words = []
    for i in range(len(language)):
        for j in range(int(ratings[i])):
            words.append(language[i])

    txt = ",".join(words)
    # print(txt)
    w = wordcloud.WordCloud(width=800, height=600, background_color="white",
                            max_words=15,
                            collocations=False)
    w.generate(txt)
    plt.imshow(w)
    plt.axis('off')  # 是否显示坐标轴
    plt.savefig("TOP20 编程语言使用率词云.png")
    plt.show()
    # w.to_file("TOP20 编程语言使用率词云.png")
    print('TOP20 编程语言使用率词云.png 已保存...\n')


# 绘制折线图
def drawLineChart(language, month, data):
    handleFont()
    print('开始绘制->各类编程长期排名情况...\n')

    # print(month)
    # print(language)

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')

    for i in range(len(data)):
        # print(data[i])
        plt.plot(month, data[i], label=language[i])
    plt.title("各类编程语言长期排名变化情况")
    plt.legend(loc='best', fontsize='x-small')
    plt.ylim(0, 40)  # 设置y轴
    plt.gca().invert_yaxis()  # y轴逆序
    plt.savefig('各类编程语言长期排名变化情况.png')
    plt.show()
    print('各类编程语言长期排名变化情况.png 已保存...\n')


# 绘制 各类编程语言长期使用率变化情况
def drawTPCI(language, time, value):
    print('各类编程语言使用率长期变化情况...\n')
    # 解决title中文乱码
    # plt.rcParams['font.sans-serif'] = ['SimHei']  # Windows
    plt.rcParams['font.sans-serif'] = ['Arial Unicode MS']  # macOS
    plt.rcParams['axes.unicode_minus'] = False

    # 输出所有可使用style
    # print(plt.style.available)
    # 设置绘图style
    plt.style.use('bmh')
    plt.figure(figsize=(10, 4.8))
    # printList(time)
    # for i in time:
    #     print(len(i))

    for i in range(len(language)):
        plt.plot(time[i], value[i], label=language[i])
    plt.title("各类编程语言使用率长期变化情况")
    plt.legend(loc='best', fontsize='x-small')
    # 坐标轴映射
    plt.xticks(range(0, 235, 22), range(2000, 2022, 2))
    plt.xlabel('时间', multialignment='center')
    plt.ylabel('使用率(%)', multialignment='center')
    # plt.ylim(0, 30)  # 设置y轴
    plt.savefig('各类编程语言使用率长期变化情况.png')
    plt.show()
    print('各类编程语言使用率长期变化情况.png 已保存...\n')


if __name__ == '__main__':

    url = 'https://www.tiobe.com/tiobe-index/'

    print('正在解析页面信息...请稍等...\n')

    html = getHTMLText(url)

    uhead, uinfo = analyzeRankHtml(html)

    language, ratings, change = dataPreTreat1(uinfo)

    print('解析：https://www.tiobe.com/tiobe-index/ 获取到 TOP20 编程语言情况如下：\n')

    printRank(uhead, uinfo)
    saveRank(uhead, uinfo)
    drawPic(ratings, change)
    drawWordcloud(language, ratings)

    print('正在解析各类编程语言长期排名情况...请稍等...\n')

    vhead, vlth = analyzeVlthHtml(html)

    language, month, data = dataPreTreat2(vhead, vlth)

    print('解析：https://www.tiobe.com/tiobe-index/ 获取到 各类编程语言长期排名情况如下：\n')

    printLongTermHistoryRank(vhead, vlth)
    saveLongTermHistoryRank(vhead, vlth)
    drawLineChart(language, month, data)

    print('解析：https://www.tiobe.com/tiobe-index/ 获取到 各类编程语言使用率长期变化情况如下：\n')

    jsonRank = getJson(url)
    language, time, value = preTreat3(jsonRank)

    drawTPCI(language, time, value)

✨参考及引用

https://stackoverflow.com/questions/72414481/error-in-anyjson-setup-command-use-2to3-is-invalid

⭐转载请注明出处

本文作者：双份浓缩馥芮白

原文链接：https://www.cnblogs.com/Flat-White/p/16491583.html

版权所有，如需转载请注明出处。