c#使用WebClient登录网站抓取登录后的网页
C#登录网站实际上就是模拟浏览器提交表单,然后记录浏览器响应返回的会话Cookie值,再次发送请求时带着这个会话cookie值去请求就可以实现模拟登录的效果了。
如下类CookieAwareWebClient实现在发送请求时都带着cookie。
public class CookieAwareWebClient : WebClient {/* 何问起 hwq2.com */ private CookieContainer cookie = new CookieContainer(); protected override WebRequest GetWebRequest(Uri address) { WebRequest request = base.GetWebRequest(address); if (request is HttpWebRequest) { (request as HttpWebRequest).CookieContainer = cookie; } return request; } }/* 何问起 hovertree.com */
如下是模拟表单提交登录的使用示例:
var client = new CookieAwareWebClient(); client.BaseAddress = @"https://hovertree.net/any/base/url/"; var loginData = new NameValueCollection(); loginData.Add("login", "YourLogin"); loginData.Add("password", "YourPassword"); client.UploadValues("login.php", "POST", loginData); //Now you are logged in and can request pages string htmlSource = client.DownloadString("index.php");/* 何问起 hwq2.com */