Java基础教程——模拟浏览器发送请求
JAVA访问网页
分别测试使用get和post方法访问网页，可以收到服务器的请求，并写入到html文件中。
import java.io.*;
import java.net.*;
import java.util.*;
public class TestGetPostPage {
// param:请求参数，格式应该满足name1=value1&name2=value2的形式。
public static String sendGet(String url, String param) {
String result = "";
if (param != null) {
url = url + "?" + param;
}
try {
URL realUrl = new URL(url);
// 打开和URL之间的连接
HttpURLConnection conn = getHttpURLConnection(realUrl);
// 打印头信息
printHeader(conn);
// 获取响应
result = getResponse(conn);
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
// param:请求参数，格式应该满足name1=value1&name2=value2的形式。
public static String sendPost(String url, String param) {
String result = "";
try {
URL realUrl = new URL(url);
HttpURLConnection conn = getHttpURLConnection(realUrl);
// 发送POST请求必须设置如下两行
conn.setDoOutput(true);
conn.setDoInput(true);
{// Post发送参数：
// 获取HttpURLConnection 对象对应的输出流
PrintWriter out = new PrintWriter(conn.getOutputStream());
// 发送请求参数
out.print(param);
out.close();
}
// 获取响应
result = getResponse(conn);
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
// 提供主方法，测试发送GET请求和POST请求
public static void main(String args[]) {
String url = "http://www.shicimingju.com/book/xiyouji.html";
// "https://zhuanlan.zhihu.com/hulaoshi";
String param = null;
// 也可以自己写个Servlet测试是否接收到参数
// url = "http://localhost:8080/TestJavaWeb/AHServlet";
// param = "uname=tiger";
// ----------------------------
String s;
// 发送GET请求
s = TestGetPostPage.sendGet(url, param);
write("http_get.html", s);
System.out.println("-----------------------------------------------");
// 发送POST请求
s = TestGetPostPage.sendPost(url, param);
write("http_post.html", s);
}
private static HttpURLConnection getHttpURLConnection(URL realUrl) {
StringBuilder sb = new StringBuilder();
sb.append("Mozilla/5.0 (Windows NT 10.0; Win64; x64)");
sb.append(" AppleWrbKit/537.36(KHTML, like Gecko)");
sb.append(" Chrome/72.0.3626.119 Safari/537.36");
HttpURLConnection conn = null;
try {
// 打开和URL之间的连接
conn = (HttpURLConnection) realUrl.openConnection();
// 设置通用的请求属性
conn.setRequestProperty("accept", "*/*");
conn.setRequestProperty("connection", "Keep-Alive");
conn.setRequestProperty("user-agent", sb.toString());
} catch (IOException e) {
e.printStackTrace();
}
return conn;
}
private static String getResponse(HttpURLConnection conn) {
// 读取URL的响应
String result = "";
try (InputStream is = conn.getInputStream();
InputStreamReader isr = new InputStreamReader(is, "utf-8");
BufferedReader in = new BufferedReader(isr)) {
String line;
while ((line = in.readLine()) != null) {
result += "\n" + line;
}
} catch (Exception e) {
System.out.println("Err:getResponse()");
e.printStackTrace();
} finally {
conn.disconnect();
}
System.out.println("getResponse()：" + result.length());
return result;
}
private static void printHeader(HttpURLConnection conn) {
System.out.println("---↓↓↓响应头字段---");
Map<String, List<String>> map = conn.getHeaderFields();
for (String key : map.keySet()) {
System.out.println(key + "--->" + map.get(key));
}
System.out.println("---↑↑↑响应头字段---");
}
private static void write(String fileName, String text) {
File f = new File(fileName);
try (FileOutputStream fos = new FileOutputStream(f);
OutputStreamWriter osw = new OutputStreamWriter(fos, "UTF-8");
BufferedWriter bw = new BufferedWriter(osw);) {
bw.write(text);
} catch (IOException e) {
e.printStackTrace();
}
}
}
还可以自己写个Servlet测试服务器端是否接收到参数：
import java.io.*;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.*;
@WebServlet("/AHServlet")
public class AHServlet extends HttpServlet {
private static final long serialVersionUID = 1L;
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String parameter = request.getParameter("uname");
System.out.println("Get : " + parameter);
resp(response);
}
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String parameter = request.getParameter("uname");
System.out.println("Post : " + parameter);
resp(response);
}
private void resp(HttpServletResponse response) throws IOException {
response.setContentType("text/html");
PrintWriter out = response.getWriter();
out.println("https://www.bilibili.com/video/av48272174/?p=2");
out.flush();
out.close();
}
}
*JAVA多线程下载网络文件
[参见]: https://www.cnblogs.com/tigerlion/p/10661367.html
URLEncoder和URLDecoder
在使用百度搜索关键字的时候，往往会在地址栏看到如下内容：
https://www.baidu.com/s?ie=utf-8&f=8&rsv_bp=1&rsv_idx=2&tn=baiduhome_pg&wd=%E8%99%8E%E8%80%81%E7%8B%AE
有的浏览器直接看不到这段内容，地址栏中直接显示输入的关键字。这是因为浏览器对其进行了修饰，避免神秘代码对用户造成困扰，一般IE不会做这种修饰。
这里的“%E8%99%8E%E8%80%81%E7%8B%AE”就是你输入的关键字，但是进行了编码。
编码，可以使用URLDecoder.decode(...)方法
解码，可以使用URLEncoder.encode(...)方法
encoder：编码器。
decoder：解码器。URL：Uniform Resource Locator，统一资源定位符。互联网上的每个文件都有一个唯一的
URL。
普通字符串 ←→ application/x-www-form-rulencoded MIME字符串
效果在于转换汉字等。
import java.net.*;
public class _12URLDecoderTest {
public static void main(String[] args) throws Exception {
// 将application/x-www-form-urlencoded字符串
// 转换成普通字符串
String keyWord = URLDecoder.decode("%E8%99%8E%E8%80%81%E7%8B%AE", "UTF-8");
System.out.println(keyWord);
// 将普通字符串转换成
// application/x-www-form-urlencoded字符串
String urlStr = URLEncoder.encode("无字真经", "GBK");
System.out.println(urlStr);
urlStr = URLEncoder.encode("无字真经", "UTF-8");
System.out.println(urlStr);
}
}
运行结果：
虎老狮
%CE%DE%D7%D6%D5%E6%BE%AD
%E6%97%A0%E5%AD%97%E7%9C%9F%E7%BB%8F
