使用monkey技术修改python requests模块

例如请求前和请求后各来一条日志，这样就不需要在自己的每个代码都去加日志了。

其实也可以直接记录'urllib3.connectionpool'  logger name的日志。

 

修改了requests，所有requests的地方将自动打印日志，前提要给custom_request添加handlers

import requests
from app.utils.utils_ydf import LogManager

logger = LogManager('custom_request').get_logger_without_handlers()


class CustomSession(requests.Session):
    def request(self, method, url,
                params=None, data=None, headers=None, cookies=None, files=None,
                auth=None, timeout=None, allow_redirects=True, proxies=None,
                hooks=None, stream=None, verify=None, cert=None, json=None):
        """Constructs a :class:`Request <Request>`, prepares it and sends it.
        Returns :class:`Response <Response>` object.

        :param method: method for the new :class:`Request` object.
        :param url: URL for the new :class:`Request` object.
        :param params: (optional) Dictionary or bytes to be sent in the query
            string for the :class:`Request`.
        :param data: (optional) Dictionary, bytes, or file-like object to send
            in the body of the :class:`Request`.
        :param json: (optional) json to send in the body of the
            :class:`Request`.
        :param headers: (optional) Dictionary of HTTP Headers to send with the
            :class:`Request`.
        :param cookies: (optional) Dict or CookieJar object to send with the
            :class:`Request`.
        :param files: (optional) Dictionary of ``'filename': file-like-objects``
            for multipart encoding upload.
        :param auth: (optional) Auth tuple or callable to enable
            Basic/Digest/Custom HTTP Auth.
        :param timeout: (optional) How long to wait for the server to send
            data before giving up, as a float, or a :ref:`(connect timeout,
            read timeout) <timeouts>` tuple.
        :type timeout: float or tuple
        :param allow_redirects: (optional) Set to True by default.
        :type allow_redirects: bool
        :param proxies: (optional) Dictionary mapping protocol or protocol and
            hostname to the URL of the proxy.
        :param stream: (optional) whether to immediately download the response
            content. Defaults to ``False``.
        :param verify: (optional) Either a boolean, in which case it controls whether we verify
            the server's TLS certificate, or a string, in which case it must be a path
            to a CA bundle to use. Defaults to ``True``.
        :param cert: (optional) if String, path to ssl client cert file (.pem).
            If Tuple, ('cert', 'key') pair.
        :rtype: requests.Response
        """
        # Create the Request.
        req = requests.Request(
            method=method.upper(),
            url=url,
            headers=headers,
            files=files,
            data=data or {},
            json=json,
            params=params or {},
            auth=auth,
            cookies=cookies,
            hooks=hooks,
        )
        logger.debug('start {} this url -->  {} '.format(method, url))
        prep = self.prepare_request(req)

        proxies = proxies or {}

        settings = self.merge_environment_settings(
            prep.url, proxies, stream, verify, cert
        )

        # Send the request.
        send_kwargs = {
            'timeout': timeout,
            'allow_redirects': allow_redirects,
        }
        send_kwargs.update(settings)
        resp = self.send(prep, **send_kwargs)
        logger.debug('{}  {}  {}  {}  {}  {}'.format(method, resp.url, resp.status_code, resp.elapsed.total_seconds(), resp.is_redirect, resp.text.__len__()))
        return resp


def custom_request(method, url, **kwargs):
    with CustomSession() as session:
        return session.request(method=method, url=url, **kwargs)


def patch_request():
    LogManager('custom_request').get_logger_and_add_handlers()
    requests.api.request = custom_request


if __name__ == '__main__':
    patch_request()
    resp = requests.get('http://www.sina.com.cn')
    requests.get('https://www.baidu.com')

 

1、结果就是这样。这样所有地方的requests请求都会生效了。

正常情况下我有自己围绕Session类实例包装的SessionWrapper类，通常情况下我不会直接去使用requests的那些函数请求接口，但同事都是直接requests，然后每个地方打印print一下请求和返回，以便能看看到发了什么请求，那样有些重复。而且项目有几百个文件，突然蹦出来的print语句，根本不知道哪里print的。

使用这种猴子技术后，不用修改库源码，就能生效。

 

2、使用猴子技术要注意的是，patch的地方并不是随便替换就能生效的。

如果库里面的那个地方是from xx import yy

你直接yy.function = yourfunction

这是无效的，一定要找到正确的地方替换，要搞清除到底怎么才能生效，必须要熟悉python的模块导入机制。可以看我的上上篇的三个试验文件。

python 模块会导入几次？猴子补丁为什么可以实现？