XSS事件（一）
前言
最近做的一个项目因为安全审计需要，需要做安全改造。其中自然就包括XSS和CSRF漏洞安全整改。关于这两个网络安全漏洞的详细说明，可以参照我本篇博客最后的参考链接。当然，我这里并不是想写一篇安全方面的专题。我要讲的是在做了XSS漏洞修复之后引发的一系列事件。
超时
本地测试的时候随便点了些页面，然后debug跟了下代码未发现任何问题。上线之后用户反馈有的页面打不开，自己去线上体验发现大部分页面正常，但是存在部分客户反馈的页面打开直接超时报错。
事件紧急处理
XSS这个漏洞修复开始不是经过我处理的，上线之后由于编码规则太严格（后面我会讲我们使用的解决方案），导致前台传入的JSON字符串中的引号全被转码，造成后台解析报错。
而我成为了那个救(bei)火(guo)英(xia)雄，需要立马解决这个问题补丁升级。我看了下以前实现的代码，有考虑到通过一个XML白名单文件，配置忽略XSS编码的请求URI。最直接的办法，是直接把我这个请求的URI加入XML白名单配置，然后补丁替换白名单文件重启服务。
但是当时我在修改的时候，考虑到可能不止这一个需要过滤的白名单，如果纯粹启动时加载的XML白名单列表。到时候还有别的URI需要忽略，那我岂不是还要再发增量补丁......
于是当时我修改的时候顺便增加了一个能力，白名单可以直接在界面配置，并且每次获取白名单列表的时候动态从数据库获取（考虑到实时请求较大，我调用的系统已有接口提供的支持缓存的查询方法）。正因为有这个“后门”我直接在线上配置了这个参数，先暂且把线上问题解决。
问题探索
解决问题第一步，自然是分析线上日志。发现线上日志的确对请求的URI中的参数做了XSS编码处理。
那问题就回到我们XSS漏洞修复的实现方式:AntiSamy(见参考链接)。其中我们的XssRequestWrapper 源码如下：
public class XssRequestWrapper extends HttpServletRequestWrapper {
private static Logger log = LoggerFactory.getLogger(XssRequestWrapper.class);
private static Policy policy = null;
static {
String path = XssRequestWrapper.class.getClassLoader().getResource("security/antisamy-tinymce.xml").getFile();
log.info("policy_filepath:" + path);
if (path.startsWith("file")) {
//以file:开头
path = path.substring(6);
}
try {
policy = Policy.getInstance(path);
} catch (PolicyException e) {
e.printStackTrace();
}
}
public XssRequestWrapper(HttpServletRequest request) {
super(request);
}
// 队请求参数进行安全转码
public String getParameter(String paramString) {
String str = super.getParameter(paramString);
if (StringUtils.isBlank(str)) {
return null;
}
return xssClean(str, paramString);
}
// 队请求头进行安全转码
public String getHeader(String paramString) {
String str = super.getHeader(paramString);
if (StringUtils.isBlank(str))
return null;
return xssClean(str, paramString);
}
@SuppressWarnings({"rawtypes","unchecked"})
public Map<String, String[]> getParameterMap() {
Map<String, String[]> request_map = super.getParameterMap();
Iterator iterator = request_map.entrySet().iterator();
log.debug("getParameterMap size:{}", request_map.size());
while (iterator.hasNext()) {
Map.Entry me = (Map.Entry) iterator.next();
String paramsKey = (String) me.getKey();
String[] values = (String[]) me.getValue();
for (int i = 0; i < values.length; i++) {
values[i] = xssClean(values[i], paramsKey);
}
}
return request_map;
}
public String[] getParameterValues(String paramString) {
String[] arrayOfString1 = super.getParameterValues(paramString);
if (arrayOfString1 == null)
return null;
int i = arrayOfString1.length;
String[] arrayOfString2 = new String[i];
for (int j = 0; j < i; j++)
arrayOfString2[j] = xssClean(arrayOfString1[j], paramString);
return arrayOfString2;
}
public final static String KEY_FILTER_STR = "'";
public final static String SUFFIX = "value";
//需要过滤的地方
private String xssClean(String value, String paramsKey) {
String keyFilterStr = KEY_FILTER_STR;
String param = paramsKey.toLowerCase();
if (param.endsWith(SUFFIX)) { // 如果参数名 name="xxxvalue"
if (value.contains(keyFilterStr)) {
value = value.replace(keyFilterStr, "‘");
}
}
AntiSamy antiSamy = new AntiSamy();
try {
final CleanResults cr = antiSamy.scan(value, policy);
// 安全的HTML输出
return cr.getCleanHTML();
} catch (ScanException e) {
log.error("antiSamy scan error", e);
} catch (PolicyException e) {
log.error("antiSamy policy error", e);
}
return value;
}
}
可以看到了我们项目组最终采用的策略配置文件是:antisamy-tinymce.xml，这种策略只允许传送纯文本到后台（这样做真的好吗？个人觉得这个规则太过严格）,并且对请求头和请求参数都做了XSS转码。请注意这里，我们相对于参考链接中源码不同的处理方式在于：我们对请求头也进行了编码处理。
那么看来问题就在于编码导致的效率低下，于是我在getHeader和getParameter方法中都打了断点。在揭晓结果之前，我说说我当时的猜测：因为当时用户反馈的有问题的页面是有很多查询条件的，我开始的猜测是应该是传入后台的参数过多导致编码影响效率。然而，现实总是无情地打脸，不管你天真不天真。
Debug发现getParameter调用的此时算正常，而getHeader处的断点没完没了的进来（最终结果证明，进来了几千次）......
原因分析
还是那句话，没有什么是源码解决不了的，如果有，那么请Debug源码。😄
我们项目是传统的SpringMVC项目，那么当然我们要从org.springframework.web.servlet.DispatcherServlet入手了。DispatcherServlet其实也是一个Servlet，他的继承关系如下:
DispatcherServlet extends FrameworkServlet
FrameworkServlet extends HttpServletBean implements ApplicationContextAware
HttpServletBean extends HttpServlet implements EnvironmentCapable, EnvironmentAware
HttpServlet extends GenericServlet
GenericServlet
implements Servlet, ServletConfig, java.io.Serializable
可以看到，实际上SpringMVC的DispatcherServlet最终也是通过doGet和doPost来对请求进行转发，而最终其实都到了DispatcherServlet的doService.该方法的源码如下:
/**
* Exposes the DispatcherServlet-specific request attributes and delegates to {@link #doDispatch}
* for the actual dispatching.
*/
@Override
protected void doService(HttpServletRequest request, HttpServletResponse response) throws Exception {
if (logger.isDebugEnabled()) {
String resumed = WebAsyncUtils.getAsyncManager(request).hasConcurrentResult() ? " resumed" : "";
logger.debug("DispatcherServlet with name '" + getServletName() + "'" + resumed +
" processing " + request.getMethod() + " request for [" + getRequestUri(request) + "]");
}
// Keep a snapshot of the request attributes in case of an include,
// to be able to restore the original attributes after the include.
Map<String, Object> attributesSnapshot = null;
if (WebUtils.isIncludeRequest(request)) {
attributesSnapshot = new HashMap<String, Object>();
Enumeration<?> attrNames = request.getAttributeNames();
while (attrNames.hasMoreElements()) {
String attrName = (String) attrNames.nextElement();
if (this.cleanupAfterInclude || attrName.startsWith("org.springframework.web.servlet")) {
attributesSnapshot.put(attrName, request.getAttribute(attrName));
}
}
}
// Make framework objects available to handlers and view objects.
request.setAttribute(WEB_APPLICATION_CONTEXT_ATTRIBUTE, getWebApplicationContext());
request.setAttribute(LOCALE_RESOLVER_ATTRIBUTE, this.localeResolver);
request.setAttribute(THEME_RESOLVER_ATTRIBUTE, this.themeResolver);
request.setAttribute(THEME_SOURCE_ATTRIBUTE, getThemeSource());
FlashMap inputFlashMap = this.flashMapManager.retrieveAndUpdate(request, response);
if (inputFlashMap != null) {
request.setAttribute(INPUT_FLASH_MAP_ATTRIBUTE, Collections.unmodifiableMap(inputFlashMap));
}
request.setAttribute(OUTPUT_FLASH_MAP_ATTRIBUTE, new FlashMap());
request.setAttribute(FLASH_MAP_MANAGER_ATTRIBUTE, this.flashMapManager);
try {
// 重点在这，都会进入doDispatch方法
doDispatch(request, response);
}
finally {
if (!WebAsyncUtils.getAsyncManager(request).isConcurrentHandlingStarted()) {
// Restore the original attribute snapshot, in case of an include.
if (attributesSnapshot != null) {
restoreAttributesAfterInclude(request, attributesSnapshot);
}
}
}
}
可以看到doService方法，最终还是会进入到doDispatch中，该方法的源码如下:
/**
* Process the actual dispatching to the handler.
* <p>The handler will be obtained by applying the servlet's HandlerMappings in order.
* The HandlerAdapter will be obtained by querying the servlet's installed HandlerAdapters
* to find the first that supports the handler class.
* <p>All HTTP methods are handled by this method. It's up to HandlerAdapters or handlers
* themselves to decide which methods are acceptable.
* @param request current HTTP request
* @param response current HTTP response
* @throws Exception in case of any kind of processing failure
*/
protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception {
HttpServletRequest processedRequest = request;
HandlerExecutionChain mappedHandler = null;
boolean multipartRequestParsed = false;
WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request);
try {
ModelAndView mv = null;
Exception dispatchException = null;
try {
processedRequest = checkMultipart(request);
multipartRequestParsed = (processedRequest != request);
// Determine handler for the current request.
// 根据请求获取处理的Handler
mappedHandler = getHandler(processedRequest);
if (mappedHandler == null || mappedHandler.getHandler() == null) {
noHandlerFound(processedRequest, response);
return;
}
// Determine handler adapter for the current request.
HandlerAdapter ha = getHandlerAdapter(mappedHandler.getHandler());
// Process last-modified header, if supported by the handler.
String method = request.getMethod();
boolean isGet = "GET".equals(method);
if (isGet || "HEAD".equals(method)) {
long lastModified = ha.getLastModified(request, mappedHandler.getHandler());
if (logger.isDebugEnabled()) {
logger.debug("Last-Modified value for [" + getRequestUri(request) + "] is: " + lastModified);
}
if (new ServletWebRequest(request, response).checkNotModified(lastModified) && isGet) {
return;
}
}
if (!mappedHandler.applyPreHandle(processedRequest, response)) {
return;
}
// Actually invoke the handler.
mv = ha.handle(processedRequest, response, mappedHandler.getHandler());
if (asyncManager.isConcurrentHandlingStarted()) {
return;
}
applyDefaultViewName(processedRequest, mv);
mappedHandler.applyPostHandle(processedRequest, response, mv);
}
catch (Exception ex) {
dispatchException = ex;
}
catch (Throwable err) {
// As of 4.3, we're processing Errors thrown from handler methods as well,
// making them available for @ExceptionHandler methods and other scenarios.
dispatchException = new NestedServletException("Handler dispatch failed", err);
}
processDispatchResult(processedRequest, response, mappedHandler, mv, dispatchException);
}
catch (Exception ex) {
triggerAfterCompletion(processedRequest, response, mappedHandler, ex);
}
catch (Throwable err) {
triggerAfterCompletion(processedRequest, response, mappedHandler,
new NestedServletException("Handler processing failed", err));
}
finally {
if (asyncManager.isConcurrentHandlingStarted()) {
// Instead of postHandle and afterCompletion
if (mappedHandler != null) {
mappedHandler.applyAfterConcurrentHandlingStarted(processedRequest, response);
}
}
else {
// Clean up any resources used by a multipart request.
if (multipartRequestParsed) {
cleanupMultipart(processedRequest);
}
}
}
}
这段处理的重点也在我中文注释的地方，我们继续跟进getHandler方法：
protected HandlerExecutionChain getHandler(HttpServletRequest request) throws Exception {
// 遍历HandlerMapping，找到请求对应的Handler具体信息
for (HandlerMapping hm : this.handlerMappings) {
if (logger.isTraceEnabled()) {
logger.trace(
"Testing handler map [" + hm + "] in DispatcherServlet with name '" + getServletName() + "'");
}
HandlerExecutionChain handler = hm.getHandler(request);
if (handler != null) {
return handler;
}
}
return null;
}
// 最终会到AbstractHandlerMethodMapping#getHandlerInternal
protected HandlerMethod getHandlerInternal(HttpServletRequest request) throws Exception {
// 分析请求URI （去掉Context）
String lookupPath = getUrlPathHelper().getLookupPathForRequest(request);
if (logger.isDebugEnabled()) {
logger.debug("Looking up handler method for path " + lookupPath);
}
this.mappingRegistry.acquireReadLock();
try {
// 重头戏： 根据请求URI去找对应的处理器 （具体到类和方法）
HandlerMethod handlerMethod = lookupHandlerMethod(lookupPath, request);
if (logger.isDebugEnabled()) {
if (handlerMethod != null) {
logger.debug("Returning handler method [" + handlerMethod + "]");
}
else {
logger.debug("Did not find handler method for [" + lookupPath + "]");
}
}
return (handlerMethod != null ? handlerMethod.createWithResolvedBean() : null);
}
finally {
this.mappingRegistry.releaseReadLock();
}
}
注意其中的中文注释，我们通过解析请求中的URI，然后根据请求URI去查找对应的处理器，也就是进行适配的关键一步：
/**
* Look up the best-matching handler method for the current request.
* If multiple matches are found, the best match is selected.
* @param lookupPath mapping lookup path within the current servlet mapping
* @param request the current request
* @return the best-matching handler method, or {@code null} if no match
* @see #handleMatch(Object, String, HttpServletRequest)
* @see #handleNoMatch(Set, String, HttpServletRequest)
*/
protected HandlerMethod lookupHandlerMethod(String lookupPath, HttpServletRequest request) throws Exception {
List<Match> matches = new ArrayList<Match>();
// 直接根据请求的URI去找对应的处理类（此时前台请求URI必须与后台注解配置的RequestMapping完全一致）
List<T> directPathMatches = this.mappingRegistry.getMappingsByUrl(lookupPath);
if (directPathMatches != null) {
addMatchingMappings(directPathMatches, matches, request);
}
if (matches.isEmpty()) {
// 注意这里： SpringMVC做了个无奈的容错处理，如果没有完全匹配的话，就遍历所有请求URI找到大致匹配的 --- 这也是本次问题出现的原因
// No choice but to go through all mappings...
addMatchingMappings(this.mappingRegistry.getMappings().keySet(), matches, request);
}
if (!matches.isEmpty()) {
Comparator<Match> comparator = new MatchComparator(getMappingComparator(request));
Collections.sort(matches, comparator);
if (logger.isTraceEnabled()) {
logger.trace("Found " + matches.size() + " matching mapping(s) for [" +
lookupPath + "] : " + matches);
}
Match bestMatch = matches.get(0);
if (matches.size() > 1) {
if (CorsUtils.isPreFlightRequest(request)) {
return PREFLIGHT_AMBIGUOUS_MATCH;
}
Match secondBestMatch = matches.get(1);
if (comparator.compare(bestMatch, secondBestMatch) == 0) {
Method m1 = bestMatch.handlerMethod.getMethod();
Method m2 = secondBestMatch.handlerMethod.getMethod();
throw new IllegalStateException("Ambiguous handler methods mapped for HTTP path '" +
request.getRequestURL() + "': {" + m1 + ", " + m2 + "}");
}
}
handleMatch(bestMatch.mapping, lookupPath, request);
return bestMatch.handlerMethod;
}
else {
return handleNoMatch(this.mappingRegistry.getMappings().keySet(), lookupPath, request);
}
}
其实到这里，我们已经找到真相了，但是为什么循环遍历请求URI会导致getHeader方法超频调用呢？我们继续跟进：
private void addMatchingMappings(Collection<T> mappings, List<Match> matches, HttpServletRequest request) {
for (T mapping : mappings) {
T match = getMatchingMapping(mapping, request);
if (match != null) {
matches.add(new Match(match, this.mappingRegistry.getMappings().get(mapping)));
}
}
}
// RequestMappingInfoHandlerMapping
/**
* Check if the given RequestMappingInfo matches the current request and
* return a (potentially new) instance with conditions that match the
* current request -- for example with a subset of URL patterns.
* @return an info in case of a match; or {@code null} otherwise.
*/
@Override
protected RequestMappingInfo getMatchingMapping(RequestMappingInfo info, HttpServletRequest request) {
return info.getMatchingCondition(request);
}
// RequestMappingInfo
/**
* Checks if all conditions in this request mapping info match the provided request and returns
* a potentially new request mapping info with conditions tailored to the current request.
* <p>For example the returned instance may contain the subset of URL patterns that match to
* the current request, sorted with best matching patterns on top.
* @return a new instance in case all conditions match; or {@code null} otherwise
*/
@Override
public RequestMappingInfo getMatchingCondition(HttpServletRequest request) {
RequestMethodsRequestCondition methods = this.methodsCondition.getMatchingCondition(request);
ParamsRequestCondition params = this.paramsCondition.getMatchingCondition(request);
// 问题就在于headersCondition的 getMatchingCondition 方法的调用
HeadersRequestCondition headers = this.headersCondition.getMatchingCondition(request);
ConsumesRequestCondition consumes = this.consumesCondition.getMatchingCondition(request);
ProducesRequestCondition produces = this.producesCondition.getMatchingCondition(request);
if (methods == null || params == null || headers == null || consumes == null || produces == null) {
return null;
}
PatternsRequestCondition patterns = this.patternsCondition.getMatchingCondition(request);
if (patterns == null) {
return null;
}
RequestConditionHolder custom = this.customConditionHolder.getMatchingCondition(request);
if (custom == null) {
return null;
}
return new RequestMappingInfo(this.name, patterns,
methods, params, headers, consumes, produces, custom.getCondition());
}
// HeadersRequestCondition
public HeadersRequestCondition getMatchingCondition(HttpServletRequest request) {
// 最终调用的是CorsUtils#isCorsRequest
if (CorsUtils.isPreFlightRequest(request)) {
return PRE_FLIGHT_MATCH;
}
for (HeaderExpression expression : expressions) {
if (!expression.match(request)) {
return null;
}
}
return this;
}
// CorsUtils
/**
* Returns {@code true} if the request is a valid CORS one.
*/
public static boolean isCorsRequest(HttpServletRequest request) {
// XSS的编码是通过Filter对请求中的Header进行编码的，所以每次遍历URI都会调用一次请求头编码
return (request.getHeader(HttpHeaders.ORIGIN) != null);
}
/**
* Returns {@code true} if the request is a valid CORS pre-flight one.
*/
public static boolean isPreFlightRequest(HttpServletRequest request) {
return (isCorsRequest(request) && HttpMethod.OPTIONS.matches(request.getMethod()) &&
request.getHeader(HttpHeaders.ACCESS_CONTROL_REQUEST_METHOD) != null);
}
好了，真相终于浮出水面了。就是因为如果前台请求URI没有完全匹配后台配置的话会导致每次请求校验都会对请求头中参数进行编码。而项目中请求的URI有几千个，假设每个请求头平均有3个参数。那么，一次请求编码可能上万次...... 你说超时不超时？
解决方案
最终和领导讨论确认去掉对header的XSS编码处理。我们业务上不存在将Header的参数入库的情况。
深思
善于思考的小伙伴一定会问了： 为什么会有请求URI不匹配呢？如果不匹配以前为什么能正常请求到呢？
哈哈，我们继续看下文：
通过最后的getMatchingCondition方法，我们可以看到要想最终能找到一个匹配的请求的URI。上面几个condition必须至少要满足一个。通过debug我发现，最终在我们项目中匹配的是patternsCondition。那么这condition的具体实现是咋样的呢？直接见源码：
/**
* Checks if any of the patterns match the given request and returns an instance
* that is guaranteed to contain matching patterns, sorted via
* {@link PathMatcher#getPatternComparator(String)}.
* <p>A matching pattern is obtained by making checks in the following order:
* <ul>
* <li>Direct match
* <li>Pattern match with ".*" appended if the pattern doesn't already contain a "."
* <li>Pattern match
* <li>Pattern match with "/" appended if the pattern doesn't already end in "/"
* </ul>
* @param request the current request
* @return the same instance if the condition contains no patterns;
* or a new condition with sorted matching patterns;
* or {@code null} if no patterns match.
*/
public PatternsRequestCondition getMatchingCondition(HttpServletRequest request) {
if (this.patterns.isEmpty()) {
return this;
}
String lookupPath = this.pathHelper.getLookupPathForRequest(request);
List<String> matches = getMatchingPatterns(lookupPath);
return matches.isEmpty() ? null :
new PatternsRequestCondition(matches, this.pathHelper, this.pathMatcher, this.useSuffixPatternMatch,
this.useTrailingSlashMatch, this.fileExtensions);
}
/**
* Find the patterns matching the given lookup path. Invoking this method should
* yield results equivalent to those of calling
* {@link #getMatchingCondition(javax.servlet.http.HttpServletRequest)}.
* This method is provided as an alternative to be used if no request is available
* (e.g. introspection, tooling, etc).
* @param lookupPath the lookup path to match to existing patterns
* @return a collection of matching patterns sorted with the closest match at the top
*/
public List<String> getMatchingPatterns(String lookupPath) {
List<String> matches = new ArrayList<String>();
for (String pattern : this.patterns) {
String match = getMatchingPattern(pattern, lookupPath);
if (match != null) {
matches.add(match);
}
}
Collections.sort(matches, this.pathMatcher.getPatternComparator(lookupPath));
return matches;
}
private String getMatchingPattern(String pattern, String lookupPath) {
if (pattern.equals(lookupPath)) {
return pattern;
}
// 如果使用后缀匹配模式 且后台配置的URI没有后缀(不包含.)，且前台请求的URI中包含. 则在后台配置的URI原来的匹配模式上加上 .* 再与前台请求URI进行匹配进行匹配
if (this.useSuffixPatternMatch) {
if (!this.fileExtensions.isEmpty() && lookupPath.indexOf('.') != -1) {
for (String extension : this.fileExtensions) {
if (this.pathMatcher.match(pattern + extension, lookupPath)) {
return pattern + extension;
}
}
}
else {
boolean hasSuffix = pattern.indexOf('.') != -1;
if (!hasSuffix && this.pathMatcher.match(pattern + ".*", lookupPath)) {
return pattern + ".*";
}
}
}
if (this.pathMatcher.match(pattern, lookupPath)) {
return pattern;
}
if (this.useTrailingSlashMatch) {
if (!pattern.endsWith("/") && this.pathMatcher.match(pattern + "/", lookupPath)) {
return pattern +"/";
}
}
return null;
}
注意中文注释的部分。我们项目中这种不匹配的情况是后台只配置了/aaa/aaaa 而前台配置的是/aaa/aaaa.json，自然符合前面的模式匹配。自然，也就不会匹配不到后台请求。
总结
每一次采坑，都能够让人前进。通过这次XSS事件，我这边还是有不少收获的:
- 多想想后门： 实现功能的时候要多考虑可能出现的意外情况，实现功能很简单，解决功能可能导致的问题却是一个值的深入思考的方向
- 源码debug的重要性： 当有一定的编程经验之后，很多问题需要耐心debug才能解决。通过这个debug的过程，不仅能够了解底层的实现流程，也熟悉了大神写的代码。 何乐而不为呢？
- SpringMVC前台请求最好与后台配置完全匹配: 通过这篇文章相信大家也看到了，不匹配的后果。SpringMVC源码中的省略号已经充分展示了他对你这种行为的无语 😢
这次的博客就到这里，一是篇幅太长了。怕吃太多消化不良，二来留点悬念，我们下回分解（虽然不知道下回是啥时候了，哈哈哈 我尽快.....）
参考链接
AntiSamy: https://blog.csdn.net/qq_35946990/article/details/74982760
XSS与CSRF: https://www.jianshu.com/p/64a413ada155
黎明前最黑暗，成功前最绝望！
