NET 5 使用HttpClient和HttpWebRequest

HttpWebRequest

这是.NET创建者最初开发用于使用HTTP请求的标准类。HttpWebRequest是老版本.net下常用的，较为底层且复杂，访问速度及并发也不甚理想，但是使用HttpWebRequest可以让开发者控制请求/响应流程的各个方面，如 timeouts, cookies, headers, protocols。另一个好处是HttpWebRequest类不会阻塞UI线程。例如，当您从响应很慢的API服务器下载大文件时，您的应用程序的UI不会停止响应。通常和WebResponse一起使用，一个发送请求，一个获取数据。另外HttpWebRequest库已经过时，不适合业务中直接使用，他更适用于框架内部操作。

/// <summary>
        /// HttpWebRequest请求网页示例
        /// </summary>
        /// <param name="args"></param>
        static void Main(string[] args)
        {
            HttpWebRequest httpWebRequest = null;
            HttpWebResponse httpWebResponse = null;
            Stream responseStream = null;
            string url = "https://www.cnblogs.com/";
            try
            {
                httpWebRequest = (HttpWebRequest)HttpWebRequest.Create(url);

                //cookie,cookie一般用来验证登录或是跟踪使用
                httpWebRequest.CookieContainer = new CookieContainer();
                httpWebRequest.CookieContainer.Add(new Cookie() { Name = "test", Value = "test1",Domain="www.cnblogs.com" });

                //来源页面
                httpWebRequest.Referer = url;

                //比较重要的UserAgent
                httpWebRequest.UserAgent = "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0 Gecko/20100101 Firefox/52.0";

                //请求方法，有GET,POPST,PUT等
                httpWebRequest.Method = "GET";

                //如果上传文件，是要设置 GetRequestStream
                //httpWebRequest.GetRequestStream

                try
                {
                    httpWebResponse = (HttpWebResponse)httpWebRequest.GetResponse();
                }
                catch (System.Net.WebException we)
                {
                    ///这个说明服务器返回了信息了，不过是非200,301,302这样正常的状态码
                    if (we.Response != null)
                    {
                        httpWebResponse = (HttpWebResponse)we.Response;
                    }
                }

                ///得到返回的stream,如果请求的是一个文件或图片，可以直接使用或保存
                responseStream = httpWebResponse.GetResponseStream();

                ///使用utf8方式读取数据流
                StreamReader streamReader = new StreamReader(responseStream, Encoding.UTF8);

                ///这里是一次性读取，对于超大的stream，要不断读取并保存
                string html = streamReader.ReadToEnd();
                streamReader.Close();
                responseStream.Close();
                Console.WriteLine(html.Length);
            }
            catch (Exception ex)
            {
                Console.WriteLine(ex.Message);
            }
            finally
            {
                if (httpWebRequest != null) httpWebRequest.Abort();
                if (httpWebResponse != null) httpWebResponse.Close();
                if (responseStream != null) responseStream.Close();
            }
        }

View Code

HttpClient

HttpClient提供强大的功能，提供了异步支持，可以轻松配合async await 实现异步请求，使用HttpClient，在并发量不大的情况，一般没有任何问题；但是在并发量一上去，如果使用不当，会造成很严重的堵塞的情况。

平时我们在使用HttpClient的时候，会将HttpClient包裹在using内部进行声明和初始化，

using(var httpClient = new HttpClient())
{
    //other codes
}

在高并发的情况下，连接来不及释放，socket被耗尽，耗尽之后就会出现喜闻乐见的一个错误：

Unable to connect to the remote serverSystem.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted.

那么如何处理这个问题？“复用HttpClient”即可

HttpClientFacotry很高效，可以最大程度上节省系统socket。（“JUST USE IT AND FXXK SHUT UP”:P）
Factory，顾名思义HttpClientFactory就是HttpClient的工厂，内部已经帮我们处理好了对HttpClient的管理，不需要我们人工进行对象释放，同时，支持自定义请求头，支持DNS更新等等等

从微软源码分析，HttpClient继承自HttpMessageInvoker，而HttpMessageInvoker实质就是HttpClientHandler。

HttpClientFactory 创建的HttpClient，也即是HttpClientHandler，只是这些个HttpClient被放到了“池子”中，工厂每次在create的时候会自动判断是新建还是复用。(默认生命周期为2min)

还理解不了的话，可以参考Task和Thread的关系

解决方案如下：

IHttpClientFactory

一、可以参考微软官方提供的方法：https://docs.microsoft.com/zh-cn/aspnet/core/fundamentals/http-requests?view=aspnetcore-3.1

https://docs.microsoft.com/zh-cn/aspnet/core/fundamentals/http-requests?view=aspnetcore-5.0

二、我的解决方案是根据官方提供的方法，选择一种最适合项目的写法进行改造。

1、nuget添加包Microsoft.AspNetCore.Http；

2、startup里ConfigureServices方法添加代码：

services.AddHttpClient();

public void ConfigureServices(IServiceCollection services)
        {
            //other codes
            
            services.AddHttpClient("client_1",config=>  //这里指定的name=client_1，可以方便我们后期服用该实例 比如已经填写url和header
            {
                config.BaseAddress= new Uri("http://client_1.com");
                config.DefaultRequestHeaders.Add("header_1","header_1");            });

            services.AddHttpClient();

            //other codes
            services.AddMvc().AddFluentValidation();
        }

3、可以使用依赖项注入 (DI) 来请求 IHttpClientFactory。以下代码使用 IHttpClientFactory 来创建 HttpClient 实例：（官方demo）

public class BasicUsageModel : PageModel
{
    private readonly IHttpClientFactory _clientFactory;

    public IEnumerable<GitHubBranch> Branches { get; private set; }

    public bool GetBranchesError { get; private set; }

    public BasicUsageModel(IHttpClientFactory clientFactory)
    {
        _clientFactory = clientFactory;
    }

    public async Task OnGet()
    {
        var request = new HttpRequestMessage(HttpMethod.Get,
            "https://api.github.com/repos/aspnet/AspNetCore.Docs/branches");
        request.Headers.Add("Accept", "application/vnd.github.v3+json");
        request.Headers.Add("User-Agent", "HttpClientFactory-Sample");

        var client = _clientFactory.CreateClient();

        var response = await client.SendAsync(request);

        if (response.IsSuccessStatusCode)
        {
            using var responseStream = await response.Content.ReadAsStreamAsync();
            Branches = await JsonSerializer.DeserializeAsync
                <IEnumerable<GitHubBranch>>(responseStream);
        }
        else
        {
            GetBranchesError = true;
            Branches = Array.Empty<GitHubBranch>();
        }
    }
}

在实际使用中，我们经常会用NewtonJson序列化，给一个简单的Demo:

string api_domain = _config.GetSection("OuterApi:open-api").Value;
                string api_url = $"{api_domain}/common-service/api/basic?code={code}";
                var request = new HttpRequestMessage(HttpMethod.Get, api_url);
                request.Headers.Add("Accept", "application/vnd.github.v3+json");


                var client = _clientFactory.CreateClient();

                var response = await client.SendAsync(request);

                Result<List<OpenApiDictModel>> apiRet = new Result<List<OpenApiDictModel>>();
                if (response.IsSuccessStatusCode)
                {
                    string responseStr = await response.Content.ReadAsStringAsync();
                    apiRet = JsonConvert.DeserializeObject<Result<List<OpenApiDictModel>>>(responseStr);
                }

IHttpClientFactory帮助类

using ICSharpCode.SharpZipLib.GZip;
using Jareds.Common.Logger;
using Newtonsoft.Json;
using System;
using System.Collections.Generic;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;

namespace ZYS.MessageCenter.Facade.Common
{
    /// <summary>
    /// http 请求服务
    /// </summary>
    public interface IHttpClientHelper
    {
        /// <summary>
        /// 使用post返回异步请求直接返回对象
        /// </summary>
        /// <typeparam name="T">返回对象类型</typeparam>
        /// <typeparam name="T2">请求对象类型</typeparam>
        /// <param name="url">请求链接</param>
        /// <param name="obj">请求对象数据</param>
        /// <param name="header">请求头</param>
        /// <param name="postFrom">表单提交 注* postFrom不为null 代表表单提交, 为null标识惊悚格式请求</param>
        /// <param name="gzip">是否压缩</param>
        /// <returns>请求返回的目标对象</returns>
        Task<T> PostObjectAsync<T, T2>(string url, T2 obj, Dictionary<string, string> header = null, Dictionary<string, string> postFrom = null, bool gzip = false);
        /// <summary>
        /// 使用Get返回异步请求直接返回对象
        /// </summary>
        /// <typeparam name="T">请求对象类型</typeparam>
        /// <param name="url">请求链接</param>
        /// <returns>返回请求的对象</returns>
        Task<T> GetObjectAsync<T>(string url);
    }

    /// <summary>
    /// http 请求服务
    /// </summary>
    public class HttpClientHelper : IHttpClientHelper
    {
        private readonly IHttpClientFactory _httpClientFactory;
        /// <summary>
        /// 构造函数
        /// </summary>
        /// <param name="httpClientFactory"></param>
        public HttpClientHelper(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;
        }
        #region http 请求方式
        /// <summary>
        /// 使用post方法异步请求 
        /// </summary>
        /// <param name="url">目标链接</param>
        /// <param name="posData">发送的参数JSON字符串</param>
        /// <param name="header">请求头</param>
        /// <param name="posFrom">表单提交格式</param>
        /// <param name="gzip">是否压缩</param>
        /// <returns>返回的字符串</returns>
        public async Task<string> PostAsync(string url, string posData, Dictionary<string, string> header = null, Dictionary<string, string> posFrom = null, bool gzip = false)
        {

            //从工厂获取请求对象
            var client = _httpClientFactory.CreateClient();
            //消息状态
            string responseBody = string.Empty;
            //存在则是表单提交信息
            if (posFrom != null)
            {
                var formData = new MultipartFormDataContent();
                foreach (var item in posFrom)
                {
                    formData.Add(new StringContent(item.Value), item.Key);
                }
                //提交信息
                var result = await client.PostAsync(url, formData);
                if (!result.IsSuccessStatusCode)
                {
                    Log.Error("请求出错");
                    return null;
                }
                //获取消息状态
                responseBody = await result.Content.ReadAsStringAsync();
            }
            else
            {
                HttpContent content = new StringContent(posData);
                if (header != null)
                {
                    client.DefaultRequestHeaders.Clear();
                    foreach (var item in header)
                    {
                        client.DefaultRequestHeaders.Add(item.Key, item.Value);
                    }
                }
                content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/json");
                HttpResponseMessage response = await client.PostAsync(url, content);
                if (!response.IsSuccessStatusCode)
                {
                    Log.Error("请求出错");
                    return null;
                }

                //response.EnsureSuccessStatusCode();
                if (gzip)
                {
                    GZipInputStream inputStream = new GZipInputStream(await response.Content.ReadAsStreamAsync());
                    responseBody = new StreamReader(inputStream).ReadToEnd();
                }
                else
                {
                    responseBody = await response.Content.ReadAsStringAsync();

                }
            }

            return responseBody;

        }

        /// <summary>
        /// 使用get方法异步请求
        /// </summary>
        /// <param name="url">目标链接</param>
        /// <param name="header"></param>
        /// <param name="Gzip"></param>
        /// <returns>返回的字符串</returns>
        public async Task<string> GetAsync(string url, Dictionary<string, string> header = null, bool Gzip = false)
        {
            var client = _httpClientFactory.CreateClient();
            //HttpClient client = new HttpClient(new HttpClientHandler() { UseCookies = false });
            if (header != null)
            {
                client.DefaultRequestHeaders.Clear();
                foreach (var item in header)
                {
                    client.DefaultRequestHeaders.Add(item.Key, item.Value);
                }
            }
            HttpResponseMessage response = await client.GetAsync(url);
            if (!response.IsSuccessStatusCode)
            {
                Log.Error("请求出错");
                return null;
            }

            //response.EnsureSuccessStatusCode();//用来抛异常
            string responseBody = "";
            if (Gzip)
            {
                GZipInputStream inputStream = new GZipInputStream(await response.Content.ReadAsStreamAsync());
                responseBody = new StreamReader(inputStream).ReadToEnd();
            }
            else
            {
                responseBody = await response.Content.ReadAsStringAsync();

            }
            return responseBody;
        }

        /// <summary>
        /// 使用post返回异步请求直接返回对象
        /// </summary>
        /// <typeparam name="T">返回对象类型</typeparam>
        /// <typeparam name="T2">请求对象类型</typeparam>
        /// <param name="url">请求链接</param>
        /// <param name="obj">请求对象数据</param>
        /// <param name="header">请求头</param>
        /// <param name="postFrom">表单提交 表单提交 注* postFrom不为null 代表表单提交, 为null标识惊悚格式请求</param>
        /// <param name="gzip">是否压缩</param>
        /// <returns>请求返回的目标对象</returns>
        public async Task<T> PostObjectAsync<T, T2>(string url, T2 obj, Dictionary<string, string> header = null, Dictionary<string, string> postFrom = null, bool gzip = false)
        {
            String json = JsonConvert.SerializeObject(obj);
            string responseBody = await PostAsync(url, json, header, postFrom, gzip); //请求当前账户的信息
            if (responseBody is null)
            {
                return default(T);
            }
            return JsonConvert.DeserializeObject<T>(responseBody);//把收到的字符串序列化
        }

        /// <summary>
        /// 使用Get返回异步请求直接返回对象
        /// </summary>
        /// <typeparam name="T">请求对象类型</typeparam>
        /// <param name="url">请求链接</param>
        /// <returns>返回请求的对象</returns>
        public async Task<T> GetObjectAsync<T>(string url)
        {
            string responseBody = await GetAsync(url); //请求当前账户的信息
            if (responseBody is null)
            {
                return default(T);
            }
            return JsonConvert.DeserializeObject<T>(responseBody);//把收到的字符串序列化
        }

        #endregion

    }
}

posted @ 2020-12-08 10:11 netlock 阅读(4899) 评论(1) 编辑收藏举报

会员力量，点亮园子希望

刷新页面返回顶部

netlock

代码的搬运工

NET 5 使用HttpClient和HttpWebRequest

HttpWebRequest

HttpClient

IHttpClientFactory

IHttpClientFactory帮助类

公告