Fork me on GitHub

使用HttpWebrequest对网站进行模拟操作(附登陆百度demo)

 

   这篇文章是在博客园正式的第一篇文章。

 

不出意外 以后将在web的方向发展，前段时间把老早以前做过的webqq机器人重做了一遍，算是对winform的告别吧，巩固了C#方面的知识。

   本篇主要介绍了我对模拟http请求方式的介绍和理解。(博客的样式是自己写的，有木有感觉好看呢(•‾̑⌣‾̑•)✧˖°)

首先 看一个GET请求

 

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
public string GetHtml(string url, string Referer, Encoding encode, bool SaveCookie)
        {
            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
            req.Method = "GET";
            req.CookieContainer = this.CookieContainer;
            req.Proxy = null;
            if (!string.IsNullOrEmpty(Referer))
                req.Referer = Referer;
            using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)
            {
                if (SaveCookie)
                {
                    this.CookieCollection = hwr.Cookies;
                    this.CookieContainer.GetCookies(req.RequestUri);
                }
                using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))
                {
                    return SR.ReadToEnd();
                }
            }
        }

 

然后 再看POST

  

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
public string PostHtml(string url, string Referer, string data, Encoding encode, bool SaveCookie)
       {
           HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
           req.CookieContainer = this.CookieContainer;
           req.ContentType = "application/x-www-form-urlencoded";
           req.Method = "POST";
           req.UserAgent = "Mozilla/5.0 (Windows NT 5.1; rv:30.0) Gecko/20100101 Firefox/30.0";
           req.Proxy = null;
           req.ProtocolVersion = HttpVersion.Version10;
           if (!string.IsNullOrEmpty(Referer))
               req.Referer = Referer;
           byte[] mybyte = Encoding.Default.GetBytes(data);
           req.ContentLength = mybyte.Length;
           using (Stream stream = req.GetRequestStream())
           {
               stream.Write(mybyte, 0, mybyte.Length);
           }
           using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)
           {
               if (SaveCookie)
               {
                   this.CookieCollection = hwr.Cookies;
                   this.CookieContainer.GetCookies(req.RequestUri);
               }
               using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))
               {
                   return SR.ReadToEnd();
               }
           }
       }

 

 

小结

1.这是我封装的一个httphelp类中的一部分，get貌似不需要那些像什么UserAgent,ContentType之类的东西

2.POST请求中一般要设置ContentType和UserAgent

3.默认请求是GET

 

 

如何 保持Cookie

每次请求之后保存CookieContainer即可，赋给下一次请求的httpwebrequest

 

区别 HttpWebRequest.CookieContainer和HttpWebResponse.CookieCollection

此处说的是进行一次http请求获得响应之后

CookieContainer是当前域的所有Cookie

CookieCollection是该次请求相关的所有Cookie

 

Referer 是什么

即本次请求的来源，从哪个页面进行http请求的

这里常被服务器用来检测请求是否合法

 

proxy？

请求的代理，对此我了解不多，忘有人能告之

在请求之前设置proxy=null，可以让请求跳过检查代理，http请求速度立刻上升一个档次！尤其是以前蹭wifi的时候我深有体会

 

 

 

 

这里是我封装的一个http请求辅助类，大家有兴趣可以参考一下：

 

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
public class HttpHelp
    {
        public CookieContainer CookieContainer { get; set; }
 
        public CookieCollection CookieCollection { get; set; }
 
        public HttpHelp()
        {
            this.CookieCollection = new CookieCollection();
            this.CookieContainer = new CookieContainer();
        }
 
        public static string GetHtml(string url, string Referer, Encoding encode)
        {
            return new HttpHelp().GetHtml(url, Referer, encode, false);
        }
 
        public static string PostHtml(string url, string Referer, string data, Encoding encode)
        {
            return new HttpHelp().PostHtml(url, Referer, data, encode, false);
        }
 
        public string GetHtml(string url, string Referer, Encoding encode, bool SaveCookie)
        {
            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
            req.Method = "GET";
            req.CookieContainer = this.CookieContainer;
            req.Proxy = null;
            if (!string.IsNullOrEmpty(Referer))
                req.Referer = Referer;
            using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)
            {
                if (SaveCookie)
                {
                    this.CookieCollection = hwr.Cookies;
                    this.CookieContainer.GetCookies(req.RequestUri);
                }
                using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))
                {
                    return SR.ReadToEnd();
                }
            }
        }
 
        public string PostHtml(string url, string Referer, string data, Encoding encode, bool SaveCookie)
        {
            HttpWebRequest req = WebRequest.Create(url) as HttpWebRequest;
            req.CookieContainer = this.CookieContainer;
            req.ContentType = "application/x-www-form-urlencoded";
            req.Method = "POST";
            req.UserAgent = "Mozilla/5.0 (Windows NT 5.1; rv:30.0) Gecko/20100101 Firefox/30.0";
            req.Proxy = null;
            req.ProtocolVersion = HttpVersion.Version10;
            if (!string.IsNullOrEmpty(Referer))
                req.Referer = Referer;
            byte[] mybyte = Encoding.Default.GetBytes(data);
            req.ContentLength = mybyte.Length;
            using (Stream stream = req.GetRequestStream())
            {
                stream.Write(mybyte, 0, mybyte.Length);
            }
            using (HttpWebResponse hwr = req.GetResponse() as HttpWebResponse)
            {
                if (SaveCookie)
                {
                    this.CookieCollection = hwr.Cookies;
                    this.CookieContainer.GetCookies(req.RequestUri);
                }
                using (StreamReader SR = new StreamReader(hwr.GetResponseStream(), encode))
                {
                    return SR.ReadToEnd();
                }
            }
        }
 
        ///
        /// 上传文件
        ///
        ///上传地址
        ///文件路径
        ///原网页file控件name
        ///请求流中的contentType
        ///返回的encoding
        ///post参数字典
        ///
        public static string PostFile(string url, string filepath, string paramName, string contentType, Encoding encode, Dictionary<string, string> dict)
        {
            HttpWebRequest hrq = WebRequest.Create(url) as HttpWebRequest;
            string boundary = "---------------------------" + DateTime.Now.Ticks.ToString("x");
            byte[] boundarybytes = System.Text.Encoding.Default.GetBytes("\r\n--" + boundary + "\r\n");
            hrq.ContentType = "multipart/form-data; boundary=" + boundary;
            hrq.Method = "POST";
 
            using (Stream stream = hrq.GetRequestStream())   //请求流
            {
                //写入post参数
                string formdataTemplete = "Content-Disposition: form-data; name=\"{0}\"\r\n\r\n{1}";
                if (dict != null && dict.Count > 0)
                {
                    foreach (KeyValuePair<string, string> pair in dict)
                    {
                        stream.Write(boundarybytes, 0, boundarybytes.Length);
                        string formitem = string.Format(formdataTemplete, pair.Key, pair.Value);
                        byte[] formitemBytes = Encoding.Default.GetBytes(formitem);
                        stream.Write(formitemBytes, 0, formitemBytes.Length);
                    }
                }
                stream.Write(boundarybytes, 0, boundarybytes.Length);
 
                //写入头信息
                string headerTemplate = "Content-Disposition: form-data; name=\"{0}\"; filename=\"{1}\"\r\nContent-Type: {2}\r\n\r\n";
                string header = string.Format(headerTemplate, paramName, Path.GetFileName(filepath), contentType);
                byte[] headerBytes = Encoding.UTF8.GetBytes(header);
                stream.Write(headerBytes, 0, headerBytes.Length);
 
                //写入文件
                byte[] fileBytes = File.ReadAllBytes(filepath);
                stream.Write(fileBytes, 0, fileBytes.Length);
 
                //写入尾部
                byte[] footerBytes = Encoding.Default.GetBytes("\r\n--" + boundary + "--\r\n");
                stream.Write(footerBytes, 0, footerBytes.Length);
 
                using (HttpWebResponse hrp = hrq.GetResponse() as HttpWebResponse)//响应流
                {
                    using (StreamReader SR = new StreamReader(hrp.GetResponseStream(), encode))
                    {
                        return SR.ReadToEnd();
                    }
                }
            }
 
        }
 
 
 
    }

 

 

 

登陆百度示例：

登陆百度的一个demo，我在13年写的 =。= ，现在登陆加密了，不过以前的这种方式仍然可以用，呵呵

 

吐槽一下，从来没写过博客，无论是新浪，QQ，还是其它。今天才发现这排版太累。

 

 

 之后打算写一下，模拟webqq请求来实现QQ机器人。尽量写成一个系列吧...么么哒(*ﾟ∀ﾟ*)

 

 

.

posted on   上位者的怜悯  阅读(8581)  评论(83)  编辑  收藏  举报

编辑推荐：
· Linux系列：如何用 C#调用 C方法造成内存泄露
· AI与.NET技术实操系列（二）：开始使用ML.NET
· 记一次.NET内存居高不下排查解决与启示
· 探究高空视频全景AR技术的实现原理
· 理解Rust引用及其生命周期标识（上）
阅读排行：
· 阿里最新开源QwQ-32B，效果媲美deepseek-r1满血版，部署成本又又又降低了！
· 单线程的Redis速度为什么快？
· SQL Server 2025 AI相关能力初探
· 展开说说关于C#中ORM框架的用法！
· AI编程工具终极对决：字节Trae VS Cursor，谁才是开发者新宠？
< 2025年3月 >
日 一 二 三 四 五 六
23 24 25 26 27 28 1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31 1 2 3 4 5

统计

点击右上角即可分享
微信分享提示
一起组团烧烤秋刀鱼
menu