最近工作中涉及到和app相关的测试工作，需要用到mock，特意网上查了些资料，发现有很多工具可以实现app的mock，但是经过我反复试用后，发现mitmproxy这个工具非常的强大
我认为mitmproxy的最大优势有2个
a、使用简单，上手成本极低
b、可以完美的python脚本集成，可以对http请求/websock请求做高度定制化且自动化的处理，更重要的是，他可以热加载
另外如果大家的工作涉及到app的测试，我强烈建议大家要使用代理抓包分析接口请求，有以下几个优势
a、可以比较不同平台app的接口处理逻辑，请求频率、触发条件 是否一样
b、可以观察app的请求频率是否合理，不合理的请求频率会浪费客户的流量
下面我们进入正题
一、前期准备
1、官网地址（有非常详细的介绍，建议去看下）
https://mitmproxy.org/
2、安装（直接使用pip安装即可）
pip install mitmproxy
安装后大家可以使用搜索工具搜索如下文件：mitmproxy.exe\mitmweb.exe\mitmweb.exe，然后执行下面的命令，有输出即可证明安装成功
>mitmdump.exe --version Mitmproxy: 8.0.0 Python: 3.8.10 OpenSSL: OpenSSL 1.1.1n 15 Mar 2022 Platform: Windows-10-10.0.19045-SP0 >mitmproxy.exe --version Mitmproxy: 8.0.0 Python: 3.8.10 OpenSSL: OpenSSL 1.1.1n 15 Mar 2022 Platform: Windows-10-10.0.19045-SP0 >mitmweb.exe --version Mitmproxy: 8.0.0 Python: 3.8.10 OpenSSL: OpenSSL 1.1.1n 15 Mar 2022 Platform: Windows-10-10.0.19045-SP0
3、启动
其实3个命令的启动是一样的，这里我认为mitmweb.exe是最适合的最强大，既有web界面，也可以集成python脚本，所以今天以介绍mitmweb.exe这个命令为主
>mitmweb.exe --help
usage: mitmweb [options]
optional arguments:
-h, --help show this help message and exit
--version show version number and exit
--options Show all options and their default values
--commands Show all commands and their signatures
--set option[=value] Set an option. When the value is omitted, booleans are set to true, strings and integers are
set to None (if permitted), and sequences are emptied. Boolean values can be true, false or
toggle. Sequences are set using multiple invocations to set for the same option.
-q, --quiet Quiet.
-v, --verbose Increase log verbosity.
--mode MODE, -m MODE Mode can be "regular", "transparent", "socks5", "reverse:SPEC", or "upstream:SPEC". For
reverse and upstream proxy modes, SPEC is host specification in the form of
"http[s]://host[:port]".
--no-anticache
--anticache Strip out request headers that might cause the server to return 304-not-modified.
--no-showhost
--showhost Use the Host header to construct URLs for display.
--rfile PATH, -r PATH
Read flows from file.
--scripts SCRIPT, -s SCRIPT
Execute a script. May be passed multiple times.
--stickycookie FILTER
Set sticky cookie filter. Matched against requests.
--stickyauth FILTER Set sticky auth filter. Matched against requests.
--save-stream-file PATH, -w PATH
Stream flows to file as they arrive. Prefix path with + to append. The full path can use
python strftime() formating, missing directories are created as needed. A new file is opened
every time the formatted string changes.
--no-anticomp
--anticomp Try to convince servers to send us un-compressed data.
Mitmweb:
--no-web-open-browser
--web-open-browser Start a browser.
--web-port PORT Web UI port.
--web-host HOST Web UI host.
--web-columns WEB_COLUMNS
Columns to show in the flow list May be passed multiple times.
Proxy Options:
--listen-host HOST Address to bind proxy to.
--listen-port PORT, -p PORT
Proxy service port.
--no-server, -n
--server Start a proxy server. Enabled by default.
--ignore-hosts HOST Ignore host and forward all traffic without processing it. In transparent mode, it is
recommended to use an IP address (range), not the hostname. In regular mode, only SSL traffic
is ignored and the hostname should be used. The supplied value is interpreted as a regular
expression and matched on the ip or the hostname. May be passed multiple times.
--allow-hosts HOST Opposite of --ignore-hosts. May be passed multiple times.
--tcp-hosts HOST Generic TCP SSL proxy mode for all hosts that match the pattern. Similar to --ignore-hosts,
but SSL connections are intercepted. The communication contents are printed to the log in
verbose mode. May be passed multiple times.
--upstream-auth USER:PASS
Add HTTP Basic authentication to upstream proxy and reverse proxy requests. Format:
username:password.
--proxyauth SPEC Require proxy authentication. Format: "username:pass", "any" to accept any user/pass
combination, "@path" to use an Apache htpasswd file, or
"ldap[s]:url_server_ldap[:port]:dn_auth:password:dn_subtree" for LDAP authentication.
--no-rawtcp
--rawtcp Enable/disable raw TCP connections. TCP connections are enabled by default.
--no-http2
--http2 Enable/disable HTTP/2 support. HTTP/2 support is enabled by default.
SSL:
--certs SPEC SSL certificates of the form "[domain=]path". The domain may include a wildcard, and is equal
to "*" if not specified. The file at path is a certificate in PEM format. If a private key is
included in the PEM, it is used, else the default key in the conf dir is used. The PEM file
should contain the full certificate chain, with the leaf certificate as the first entry. May
be passed multiple times.
--cert-passphrase PASS
Passphrase for decrypting the private key provided in the --cert option. Note that passing
cert_passphrase on the command line makes your passphrase visible in your system's process
list. Specify it in config.yaml to avoid this.
--no-ssl-insecure
--ssl-insecure, -k Do not verify upstream server SSL/TLS certificates.
--key-size KEY_SIZE TLS key size for certificates and CA.
Client Replay:
--client-replay PATH, -C PATH
Replay client requests from a saved file. May be passed multiple times.
Server Replay:
--server-replay PATH, -S PATH
Replay server responses from a saved file. May be passed multiple times.
--no-server-replay-kill-extra
--server-replay-kill-extra
Kill extra requests during replay (for which no replayable response was found).
--no-server-replay-nopop
--server-replay-nopop
Don't remove flows from server replay state after use. This makes it possible to replay same
response multiple times.
--no-server-replay-refresh
--server-replay-refresh
Refresh server replay responses by adjusting date, expires and last-modified headers, as well
as adjusting cookie expiration.
Map Remote:
--map-remote PATTERN, -M PATTERN
Map remote resources to another remote URL using a pattern of the form "[/flow-filter]/url-
regex/replacement", where the separator can be any character. May be passed multiple times.
Map Local:
--map-local PATTERN Map remote resources to a local file using a pattern of the form "[/flow-filter]/url-
regex/file-or-directory-path", where the separator can be any character. May be passed
multiple times.
Modify Body:
--modify-body PATTERN, -B PATTERN
Replacement pattern of the form "[/flow-filter]/regex/[@]replacement", where the separator can
be any character. The @ allows to provide a file path that is used to read the replacement
string. May be passed multiple times.
Modify Headers:
--modify-headers PATTERN, -H PATTERN
Header modify pattern of the form "[/flow-filter]/header-name/[@]header-value", where the
separator can be any character. The @ allows to provide a file path that is used to read the
header value string. An empty header-value removes existing header-name headers. May be passed
multiple times.
Filters:
See help in mitmproxy for filter expression syntax.
--intercept FILTER Intercept filter expression.
说实话有这么命令参数，其实我们常用的就2个
mitmweb.exe -p 8888 -s D:\code\test\mitm4.py
-p指定代理服务的启动端口
-s指定我们执行的脚本
4、配置代理
正常情况下，我们app和笔记本连在同一个wifi就可以代理抓包，但是在我实际测试工程中，会发现有的时候会抓不到包。所以这个时候建议大家在笔记本上开热点，然后手机连这个热点，然后在配置代理，就基本上没问题
5、安装证书
由于目前的请求基本上都是https，所以我们需要安装证书,以便mitmproxy可以帮我们解密https的报文
如果mitmweb已经启动成功，我们就可以访问下面这个地址
http://mitm.it/
如果出现这个，就说明我们的代理还没有配置对
如果出现下面的窗口，则证明正常，我们根据当前的客户端类型安装证书即可
二、进入实战（这里我们用51cto这个app来做测试）
1、脚本的模板（当然还有其他钩子方法，但是基本不常用，我也没有去使用，所以这里不讲，这里只讲request和response，其他可以看官网）
import mitmproxy.http
from mitmproxy import ctx, http
class Mit():
def request(self,flow:mitmproxy.http.HTTPFlow):
#在这里写对request请求的流程处理
pass
def response(self,flow:mitmproxy.http.HTTPFlow):
#在这里写对response请求的流程处理
pass
addons = [
Mit(),
]
1、修改请求的报文（ctx.log是proxy自己写的一个日志工具，可以根据日志的级别在控制台打印不同的颜色，你也可以用print打印）
a、先用下面的方法获取请求这个对象有什么方法
class Mit():
def request(self,flow:mitmproxy.http.HTTPFlow):
#在这里写对request请求的流程处理
if flow.request.url == "edu.51cto.com/app.php":
ctx.log.error("匹配到请求报文")
ctx.log.error(dir(flow.request))
def response(self,flow:mitmproxy.http.HTTPFlow):
#在这里写对response请求的流程处理
pass
addons = [
Mit(),
]
可以看到控制台输出，request这个请求有如下方法，所有大家可想而知mitmproxy这个工具有多强大了，也就是说我们几乎可以对请求字任何字段删除和更改，甚至可以对新增字段
['__abstractmethods__',
'__annotations__',
'__class__',
'__delattr__',
'__dict__',
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__gt__',
'__hash__',
'__init__',
'__init_subclass__',
'__le__',
'__lt__',
'__module__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__sizeof__',
'__str__',
'__subclasshook__',
'__weakref__',
'_abc_impl',
'_get_content_type_charset',
'_get_cookies',
'_get_multipart_form',
'_get_query',
'_get_urlencoded_form',
'_guess_encoding',
'_set_cookies',
'_set_multipart_form',
'_set_query',
'_set_urlencoded_form',
'anticache',
'anticomp',
'authority',
'constrain_encoding',
'content',
'cookies',
'copy',
'data',
'decode',
'encode',
'first_line_format',
'from_state',
'get_content',
'get_state',
'get_text',
'headers',
'host',
'host_header',
'http_version',
'is_http10',
'is_http11',
'is_http2',
'json',
'make',
'method',
'multipart_form',
'path',
'path_components',
'port',
'pretty_host',
'pretty_url',
'query',
'raw_content',
'scheme',
'set_content',
'set_state',
'set_text',
'stream',
'text',
'timestamp_end',
'timestamp_start',
'trailers',
'url',
'urlencoded_form']
b、获取参数和修改参数的例子
我们常用用法，下面的代码我演示了获取参数和修改的参数的用法，大家注意看我的注释
def request(self,flow:mitmproxy.http.HTTPFlow):
#在这里写对request请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
ctx.log.error("匹配到请求报文")
# ctx.log.error(dir(flow.request))
ctx.log.error(flow.request.query)
# 返回url的参数的值
ctx.log.error(flow.request.query.items())
# 返回url参数值的dict形式
ctx.log.debug(flow.request.query.keys())
# url参数的key
ctx.log.info(flow.request.query.values())
# url参数的values值
ctx.log.info(flow.request.method)
# 获取请求的方法
ctx.log.info(flow.request.scheme)
# 获取请求的类型，http还是https
ctx.log.info(flow.request.host)
# 获取请求的host
ctx.log.info(flow.request.headers)
# 获取请求的头部
ctx.log.info(flow.request.url)
# 获取请求的url
flow.request.url = "https://www.baidu.com"
#修改url
flow.request.query.set_all("wd",["test"])
#设置url参数
大家看到我在app上请求51cto的地址，被代理到了百度，且新增了url的参数
c、新增参数的例子
另外，我们不仅仅可以修改已有参数，还可以给请求头新增参数
flow.request.headers["test"] = "abc"
2、修改响应
a、先用下面的方法看下响应请求有什么属性和方法
class Mit():
def request(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对request请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
pass
def response(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对response请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
ctx.log.error(dir(flow.response))
addons = [
Mit(),
]
可以看到控制台输出，response这个请求有如下方法，所有大家可想而知mitmproxy这个工具有多强大了，也就是说我们几乎可以对请求的任何做增删改
['__abstractmethods__',
'__annotations__',
'__class__',
'__delattr__',
'__dict__',
'__dir__',
'__doc__',
'__eq__',
'__format__',
'__ge__',
'__getattribute__',
'__gt__',
'__hash__',
'__init__',
'__init_subclass__',
'__le__',
'__lt__',
'__module__',
'__ne__',
'__new__',
'__reduce__',
'__reduce_ex__',
'__repr__',
'__setattr__',
'__sizeof__',
'__str__',
'__subclasshook__',
'__weakref__',
'_abc_impl',
'_get_content_type_charset',
'_get_cookies',
'_guess_encoding',
'_set_cookies',
'content',
'cookies',
'copy',
'data',
'decode',
'encode',
'from_state',
'get_content',
'get_state',
'get_text',
'headers',
'http_version',
'is_http10',
'is_http11',
'is_http2',
'json',
'make',
'raw_content',
'reason',
'refresh',
'set_content',
'set_state',
'set_text',
'status_code',
'stream',
'text',
'timestamp_end',
'timestamp_start',
'trailers']
b、演示一个拒绝响应的例子
class Mit():
def request(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对request请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
pass
def response(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对response请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
ctx.log.error(dir(flow.response))
ctx.log.error("测试拒绝响应")
flow.response = flow.response.make(status_code=404,
content="<h1>被代理</h1>",
headers={"content-type":"text/html"}
)
addons = [
Mit(),
]
c、演示一个获取参数的例子
import json
class Mit():
def request(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对request请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
pass
def response(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对response请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
res_text = json.loads(flow.response.text,encoding="utf-8")
ctx.log.info(res_text)
res_get_text = flow.response.get_text("result")
ctx.log.warn(res_get_text)
res_get_header = flow.response.headers
ctx.log.error(res_get_header)
addons = [
Mit(),
]
看一下返回的结果
flow.response.text的值，也就是响应体的值
d、演示一个修改参数的例子
import json
class Mit():
def request(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对request请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
pass
def response(self, flow: mitmproxy.http.HTTPFlow):
# 在这里写对response请求的流程处理
if "edu.51cto.com/app.php" in flow.request.url:
res_text = json.loads(flow.response.text,encoding="utf-8")
res_text["msg"] = "test_mitmproxy"
flow.response.text = json.dumps(res_text,ensure_ascii=True)
addons = [
Mit(),
]
我们修改了msg的信息
至此我认为主要的作用已经讲完了
