Python aiohttp raise RuntimeError(‘Event loop is closed‘)

问题描述

aiohttp 的 getting started 入门案例是这样写的

 import aiohttp
import asyncio
 
 
async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get('https://python.org') as response:
            print("Status:", response.status)
            print("Content-type:", response.headers['content-type'])
 
            html = await response.text()
            print("Body:", html[:15], "...")
 
 
loop = asyncio.get_event_loop()
loop.run_until_complete(main())

运行结果为

 Status: 200
Content-type: text/html; charset=utf-8
Body: <!doctype html> ...

看上去没问题，但是在 Python3.7 后对 asyncio 进行了改进，可以直接调用 asyncio.run() 执行协程程序，而不需管底层 API 如事件循环 loop 的操作，所以上述代码的

 loop = asyncio.get_event_loop()
loop.run_until_complete(main())

可以直接替换为

 asyncio.run()

Linux 和 Mac 上这样运行是没问题的，但是在 Windows 上运行会报如下错误

 Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B1FFE978B0>
Traceback (most recent call last):
  File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

原因分析

像 aiohttp 这类第三方协程库都是依赖于标准库 asyncio 的，而 asyncio 对 Windows 的支持本来就不好。Python3.8 后默认 Windows 系统上的事件循环采用 ProactorEventLoop （仅用于 Windows ）这篇文档描述了其在 Windows 下的缺陷：https://docs.python.org/zh-cn/3/library/asyncio-platforms.html#windows 👈

引发异常的函数是 _ProactorBasePipeTransport.__del__ ，所以 aiohttp 铁定使用了 _ProactorBasePipeTransport，并且在程序退出释放内存时自动调用了其__del__ 方法

在这里插入图片描述

就是上述一串连环反应最终抛出了 RuntimeError: Event loop is closed

一般的协程程序是不会使用 _ProactorBasePipeTransport 的，所以下面的代码还是可以正常的运行

 import asyncio
 
 
async def main():
    print('Hello...')
    await asyncio.sleep(1)
    print('...World')
 
 
asyncio.run(main())

我特意写了个装饰器来验证这一点：

 import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
 
 
def explicit_call(func):
    @wraps(func)
    def wrappers(self, *args, **kwargs):
        print('call _ProactorBasePipeTransport.__del__')
        return func(self, *args, **kwargs)
 
    return wrappers
 
 
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
 
 
async def main():
    print('Hello...')
    await asyncio.sleep(1)
    print('...World')
 
 
asyncio.run(main())

正常执行，没有使用 _ProactorBasePipeTransport

 Hello...
...World

 import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
 
import aiohttp
 
 
async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get('https://www.baidu.com') as response:
            print("Status:", response.status)
            print("Content-type:", response.headers['content-type'])
 
            html = await response.text()
            print("Body:", html[:15], "...")
 
 
def explicit_call(func):
    @wraps(func)
    def wrappers(self, *args, **kwargs):
        print('call _ProactorBasePipeTransport.__del__')
        return func(self, *args, **kwargs)
 
    return wrappers
 
 
_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)
 
asyncio.run(main())

先打印 call _ProactorBasePipeTransport.__del__ 然后报错，说明使用了 _ProactorBasePipeTransport

 Status: 200
Content-type: text/html
Body: <html>
 ...
call _ProactorBasePipeTransport.__del__
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B11653EF70>
Traceback (most recent call last):
  File "D:\Codes\SpiderProject\BingImageSpider\demo.py", line 16, in wrappers
    return func(self, *args, **kwargs)
  File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
    self.close()
  File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
    self._check_closed()
  File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed
 
进程已结束，退出代码为 0

解决方案

如果执意要在 Windows 下继续开发，有这几个方案可以选择

1. 不要使用 run 函数

既然 _ProactorBasePipeTransport 会在程序结束后自动关闭事件循环，那就不要用 run 函数了，用官网的例子，乖乖使用 loop 吧

 loop = asyncio.get_event_loop()
loop.run_until_complete(main())

2. 替换事件循环

在调用 run 函数前，替换默认的 ProactorEventLoop 为 SelectorEventLoop

 asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
asyncio.run(main())

但是 SelectorEventLoop 是有一些缺点的，比如不支持子进程等

3. 忽略异常

这是 Github 上一个外国大佬的方法，在不改变源码的前提下，使用装饰器忽略掉异常

 import asyncio
from asyncio.proactor_events import _ProactorBasePipeTransport
from functools import wraps
 
import aiohttp
 
 
async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get('https://www.baidu.com') as response:
            print("Status:", response.status)
            print("Content-type:", response.headers['content-type'])
 
            html = await response.text()
            print("Body:", html[:15], "...")
 
 
def silence_event_loop_closed(func):
    @wraps(func)
    def wrapper(self, *args, **kwargs):
        try:
            return func(self, *args, **kwargs)
        except RuntimeError as e:
            if str(e) != 'Event loop is closed':
                raise
 
    return wrapper
 
 
_ProactorBasePipeTransport.__del__ = silence_event_loop_closed(_ProactorBasePipeTransport.__del__)
 
asyncio.run(main())

更详细的信息可以在这个 issue 上找到：https://github.com/aio-libs/aiohttp/issues/4324 👈

相关链接：

https://github.com/aio-libs/aiohttp/issues/4324

https://stackoverflow.com/questions/45600579/asyncio-event-loop-is-closed-when-getting-loop

https://docs.python.org/zh-cn/3/library/asyncio-platforms.html#windows

https://bugs.python.org/issue39232

喜欢我的文章的话，欢迎关注👇点赞👇评论👇收藏👇 谢谢支持！！！

posted @ 2022-04-07 11:30 王舰阅读(2500) 评论(0) 编辑收藏举报

刷新页面返回顶部

登录后才能查看或发表评论，立即登录或者逛逛博客园首页

相关博文：

· Python Python3.9使用uvloop抛出NotImplementedError异常

· Python 协程asyncio+aiohttp“百万并发”高速爬取英雄联盟皮肤（1385次get请求和图片下载，用时45s）

· asyncio+aiohttp出现Exception ignored：RuntimeError('Event loop is closed')

· asyncio --错误信息处理： RuntimeError: Event loop is closed

· Python - asyncio

阅读排行：
· 无需6万激活码！GitHub神秘组织3小时极速复刻Manus，手把手教你使用OpenManus搭建本
· Manus爆火，是硬核还是营销？
· 终于写完轮子一部分：tcp代理了，记录一下
· 别再用vector＜bool＞了！Google高级工程师：这可能是STL最大的设计失误
· 单元测试从入门到精通

王舰的博客

网络江湖中的安全屋

Python aiohttp raise RuntimeError(‘Event loop is closed‘)

问题描述

原因分析

解决方案

1. 不要使用 run 函数

2. 替换事件循环

3. 忽略异常

我的标签

随笔分类

阅读排行榜

评论排行榜

推荐排行榜

最新评论

	import aiohttp
	import asyncio


	async def main():
	async with aiohttp.ClientSession() as session:
	async with session.get('https://python.org') as response:
	print("Status:", response.status)
	print("Content-type:", response.headers['content-type'])

	html = await response.text()
	print("Body:", html[:15], "...")


	loop = asyncio.get_event_loop()
	loop.run_until_complete(main())

	Status: 200
	Content-type: text/html; charset=utf-8
	Body: <!doctype html> ...

	Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B1FFE978B0>
	Traceback (most recent call last):
	File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
	self.close()
	File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
	self._loop.call_soon(self._call_connection_lost, None)
	File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
	self._check_closed()
	File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
	raise RuntimeError('Event loop is closed')
	RuntimeError: Event loop is closed

	import asyncio


	async def main():
	print('Hello...')
	await asyncio.sleep(1)
	print('...World')


	asyncio.run(main())

	import asyncio
	from asyncio.proactor_events import _ProactorBasePipeTransport
	from functools import wraps


	def explicit_call(func):
	@wraps(func)
	def wrappers(self, args, *kwargs):
	print('call _ProactorBasePipeTransport.__del__')
	return func(self, args, *kwargs)

	return wrappers


	_ProactorBasePipeTransport.__del__ = explicit_call(_ProactorBasePipeTransport.__del__)


	async def main():
	print('Hello...')
	await asyncio.sleep(1)
	print('...World')


	asyncio.run(main())

	Status: 200
	Content-type: text/html
	Body: <html>
	...
	call _ProactorBasePipeTransport.__del__
	Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001B11653EF70>
	Traceback (most recent call last):
	File "D:\Codes\SpiderProject\BingImageSpider\demo.py", line 16, in wrappers
	return func(self, args, *kwargs)
	File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 116, in __del__
	self.close()
	File "D:\DevTools\Python\lib\asyncio\proactor_events.py", line 108, in close
	self._loop.call_soon(self._call_connection_lost, None)
	File "D:\DevTools\Python\lib\asyncio\base_events.py", line 746, in call_soon
	self._check_closed()
	File "D:\DevTools\Python\lib\asyncio\base_events.py", line 510, in _check_closed
	raise RuntimeError('Event loop is closed')
	RuntimeError: Event loop is closed

	进程已结束，退出代码为 0

	asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
	asyncio.run(main())