tornado解决高并发的初步认识牵扯出的一些问题
#!/bin/env python
# -*- coding:utf-8 -*-
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.gen
from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor
import time
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class SleepHandler(tornado.web.RequestHandler):
executor = ThreadPoolExecutor(2)
@tornado.web.asynchronous
@tornado.gen.coroutine
def get(self):
# 假如你执行的异步会返回值被继续调用可以这样(只是为了演示),否则直接yield就行
res = yield self.sleep()
self.write("when i sleep %s s" % res)
self.finish()
@run_on_executor
def sleep(self):
time.sleep(5)
return 5
class JustNowHandler(tornado.web.RequestHandler):
def get(self):
self.write("i hope just now see you")
if __name__ == "__main__":
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[
(r"/sleep", SleepHandler), (r"/justnow", JustNowHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
上面的代码示例转载自：https://blog.csdn.net/qq_28893679/article/details/69437496
本来通过线程池我觉得就可以实现高并发了，但是这位道友的@tornado.gen.coroutine这个装饰器，让我重新认识了协程，之前我一直以为协程是用来在单线程中解决io多路服用的，其实我还忽略了一个重要的地方，就是协程会主动让出CPU资源，这点我居然会遗忘，哎，天杀的，所以协程还有个重要的作用就是主动让出CPU资源。
@tornado.web.asynchronous的源码和自己瞎胡分析
@asynchronous源码:
def asynchronous(method):
"""Wrap request handler methods with this if they are asynchronous.
This decorator is for callback-style asynchronous methods; for
coroutines, use the ``@gen.coroutine`` decorator without
``@asynchronous``. (It is legal for legacy reasons to use the two
decorators together provided ``@asynchronous`` is first, but
``@asynchronous`` will be ignored in this case)
This decorator should only be applied to the :ref:`HTTP verb
methods <verbs>`; its behavior is undefined for any other method.
This decorator does not *make* a method asynchronous; it tells
the framework that the method *is* asynchronous. For this decorator
to be useful the method must (at least sometimes) do something
asynchronous.
If this decorator is given, the response is not finished when the
method returns. It is up to the request handler to call
`self.finish() <RequestHandler.finish>` to finish the HTTP
request. Without this decorator, the request is automatically
finished when the ``get()`` or ``post()`` method returns. Example:
.. testcode::
class MyRequestHandler(RequestHandler):
@asynchronous
def get(self):
http = httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed.com/", self._on_download)
def _on_download(self, response):
self.write("Downloaded!")
self.finish()
.. testoutput::
:hide:
.. versionchanged:: 3.1
The ability to use ``@gen.coroutine`` without ``@asynchronous``.
.. versionchanged:: 4.3 Returning anything but ``None`` or a
yieldable object from a method decorated with ``@asynchronous``
is an error. Such return values were previously ignored silently.
"""
# Delay the IOLoop import because it's not available on app engine.
from tornado.ioloop import IOLoop
@functools.wraps(method)
def wrapper(self, *args, **kwargs):
self._auto_finish = False
with stack_context.ExceptionStackContext(
self._stack_context_handle_exception):
result = method(self, *args, **kwargs)
if result is not None:
result = gen.convert_yielded(result)
# If @asynchronous is used with @gen.coroutine, (but
# not @gen.engine), we can automatically finish the
# request when the future resolves. Additionally,
# the Future will swallow any exceptions so we need
# to throw them back out to the stack context to finish
# the request.
def future_complete(f):
f.result()
if not self._finished:
self.finish()
IOLoop.current().add_future(result, future_complete)
# Once we have done this, hide the Future from our
# caller (i.e. RequestHandler._when_complete), which
# would otherwise set up its own callback and
# exception handler (resulting in exceptions being
# logged twice).
return None
return result
return wrapper
蹩脚的翻译:
def asynchronous(method):
如果处理请求的方法是异步的，就用此方法来包装它(它代指get,post等方法)
这个装饰器用于回调类型的异步方法;对于协程,请用"@gen.coroutine"装饰器而不是"@asynchronous".(如果要同时合法的使用这两个装饰器,前提是"@asynchronous"要放在第一个也就是最外面,但在这种情况下"@asynchronous"将被忽略)
这个装饰器应该只能被应用到:ref:'HTTP 动态方法(get...)';它的行为对任何其它的方法都是未定义的.这个装饰器不制作异步方法;它告诉框架这个方法是异步的.对于这个装饰器包装的方法一定(至少一段时间)要做一些异步的事情.
如果给出了这个装饰器,被包装的方法已经return的时候响应还没有完成.完不完成取决于处理请求的方法去调用"self.finish()<RequestHandler.finish>"去结束这次HTTP请求.没有这个装饰器,请求会在"get()"或"post()"方法return的时候结束.例如：
.. 测试代码::
class MyRequestHandler(RequestHandler):
@asynchronous
def get(self):
http = httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed.com/", self._on_download)
def _on_download(self, response):
self.write("Downloaded!")
self.finish()
.. 测试输出::
:隐藏:
.. 版本改变:3.1
能用"@gen.coroutine"实现就不用"@asynchronous"
.. 版本改变:4.3 在被该装饰器包装的方法中返回"None"或者一个yieldable对象是一个错误.这种返回值在之前是默认忽略的.
# 延迟IOLoop导入因为它不能在app引擎上用
from tornado.ioloop import IOLoop
@functools.wraps(method)
def wrapper(self,*args,**kwargs):
self._auto_finish = False
with stack_context.ExceptionStackContext(
self._stack_context_handle_exception):
result = method(self,*args,**kwargs)
if result is not None:
result = gen.convert_yielded(result)
#如果 @asynchronous 是和@gen.coroutine(但不是@gen.engine)一起用的,我们
#能自动完成请求当future对象resolves时(我理解处于解决状态).此外,future对象
#将吞下所有异常,因此我们需要让他们退出堆栈上下文去完成请求.
def future_complete(f):
f.result()
if not self._finished:
self.finish()
IOLoop.current().add_future(result,future_complete)
#一旦我们完成了这一步，隐藏来自调用者的Future
#(即RequestHandler._when_complete),
#否则它会设置自己的回调和异常处理程序(导致异常被记录两次).
return None
return result
return wrapper
随后我又开始看了@gen.conroutine然后在源码里发现了一个神奇的模块types模块，大开这个模块后发现了各种类型的定义，然后我又莫名奇妙的想到了数据结构和数据类型，接下来就一发不可收拾了，简明python教程中说，数据结构是一种结构，能够将一些数据聚合在一起，也就是用来存储一系列相关数据的集合。python内置的数据结构有四种列表，集合，字典，元组。然后我又想起python的数据类型整形，字符串，列表，字典，集合，元组。到这里我开始犯糊涂了，怎么回事，数据结构也可以是数据类型么，活着数据类型也可以同时是数据结构么。查阅了很多资料(就是自己想了半天)后我总结除了自己的理解：
数据结构=数据+组合方式，就是把好多数据多放在一起了，对吧
数据类型=数据+处理方式，就是把原生数据(0101)加工处理了一下，变成了现有的数据
不要打我。。。
