IT博客汇
  • 首页
  • 精华
  • 技术
  • 设计
  • 资讯
  • 扯淡
  • 权利声明
  • 登录 注册

    怎样写一个 RefererFilter

    TiuVe2发表于 2017-05-26 16:02:07
    love 0

    缘起

    首先,用检查Referer的方式来防披御CSRF并不是很好的方法。因项目临时有需要,所以做为过渡方案。
    为什么判断referer不是很好的办法?

    • referer 可能为空

    https跳转http没有referer
    https跳转不同的域名的https没有referer
    通过特殊构造的POST请求没有referer
    一些的proxy会把referer去掉
    用户直接在浏览器里访问(GET请求)

    • 判断的逻辑复杂(用正则匹配?)
    • 友站中招,殃及池鱼
    • 可以作为过渡方案,非长久之计

    构造空referer请求的一些参考资料

    • Stripping Referrer for fun and profit
    • Stripping the Referer in a Cross Domain POST request

    防御CSRF目前比较好的办法是CSRF Token,参考另一篇blog:Cookie & Session & CSRF。

    ##收集资料

    先搜索下前人有没有这类相关的工作。
    搜索到的关于RefererFilter的信息并不多。

    不过这里学到了一些东东:

    https://svn.apache.org/repos/asf/sling/tags/org.apache.sling.security-1.0.0/src/main/java/org/apache/sling/security/impl/ReferrerFilter.java
    • 是否允许localhost, 127.0.0.1这样referer的请求?
    • 是否允许本地的IP/host的请求?

    再搜索下java里提取request的referer的方法,还有filter里重定向请求的方法。
    再仔细看了下OWASP的文档:

    https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)_Prevention_Cheat_Sheet

    ##确定方案

    • 默认拦截“POST|PUT|DELETE|CONNECT|PATCH”的请求
    • HttpServletRequest里提取到referer
    • 用java.net.URL来提取referer里的host
    • 判断host是否符合要求,支持完全匹配的域名和子域名
    • 不符合要求的请求回应403或者重定向到指定的页面

    为什么不用正则的方式来处理referer?

    • 正则表达式通常比较慢
    • 很难判断一个复杂的正则表达式是否真的正确
    • URL是很复杂的,不要手动处理URL,参考URL的语法

    ##思考需要提供的配置项

    实际最终提供了这些配置项,考虑到像host这样的配置不是经常变动的,所以没有提供从外部配置文件加载配置的功能。

    matchMethods   即拦截的方法,默认值"POST|PUT|DELETE|CONNECT|PATCH",通常不用配置
    allowSubDomainHosts 匹配子域名,以"|"分隔,如"test.com|abc.com",
                         则http://test.com, http://xxx.test.com这样的请求都会匹配到,推荐优先使用这个配置
    completeMatchHosts 完全匹配的域名,以"|"分隔,如"test.com|abc.com",则只有http://test.com 这样的请求会匹配
                        像http://www.test.com 这样的请求不会被匹配
    
    responseError  被拦截的请求的response的返回值,默认是403
    redirectPath   被拦截的请求重定向到的url,如果配置了这个值,则会忽略responseError的配置。
                        比如可以配置重定向到自己定义的错误页: /referer_error.html
    bAllowEmptyReferer  是否允许空referer,默认是false,除非很清楚,否则不要改动这个
    bAllowLocalhost   是否允许localhost, 127.0.0.1 这样的referer的请求,默认是true,便于调试
    bAllowAllIPAndHost  是否允许本机的所有IP和host的referer请求,默认是false

    ##编码的细节

    • 重定向时,注意加上contextPath
    response.sendRedirect(request.getContextPath() + redirectPath);
    • 构造URL时,非法的URL会抛出RuntimeException,需要处理

    ##正确地处理URL

    感觉这个有必要再次说明下:

    http://docs.oracle.com/javase/tutorial/networking/urls/urlInfo.html

    用contain, indexOf, endWitch这些函数时都要小心。

    public static void main(String[] args) throws Exception {
           URL aURL = new URL("http://example.com:80/docs/books/tutorial"
                              + "/index.html?name=networking#DOWNLOADING");
           System.out.println("protocol = " + aURL.getProtocol());
           System.out.println("authority = " + aURL.getAuthority());
           System.out.println("host = " + aURL.getHost());
           System.out.println("port = " + aURL.getPort());
           System.out.println("path = " + aURL.getPath());
           System.out.println("query = " + aURL.getQuery());
           System.out.println("filename = " + aURL.getFile());
           System.out.println("ref = " + aURL.getRef());
       }

    ##用curl来测试

    最后用curl来做了一些测试:

    curl  --header "Referer:http://test.com" http://localhost:8080/filter-test/referer
    curl -X POST --header "Referer:http://test.com" http://localhost:8080/filter-test/referer
    curl -X POST --header "Referer:xxxxx" http://localhost:8080/filter-test/referer
    curl -X POST http://localhost:8080/filter-test/referer
    curl -X POST --header "Referer:http://abc.test.com" http://localhost:8080/filter-test/referer
    curl -X POST --header "Referer:http://abc.hello.com.test.com" http://localhost:8080/filter-test/referer

    ##实现的代码

    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    
    /**
     * <pre>
     * 支持的配置项:
     * matchMethods   即拦截的方法,默认值"POST|PUT|DELETE|CONNECT|PATCH",通常不用配置
     * allowSubDomainHosts 匹配子域名,以"|"分隔,如"test.com|abc.com",
     *                     则http://test.com, http://xxx.test.com这样的请求都会匹配到,推荐优先使用这个配置
     * completeMatchHosts 完全匹配的域名,以"|"分隔,如"test.com|abc.com",则只有http://test.com 这样的请求会匹配
     *                    像http://www.test.com 这样的请求不会被匹配
     *     
     * responseError  被拦截的请求的response的返回值,默认是403
     * redirectPath   被拦截的请求重定向到的url,如果配置了这个值,则会忽略responseError的配置。
     *                    比如可以配置重定向到自己定义的错误页: /referer_error.html
     * bAllowEmptyReferer  是否允许空referer,默认是false,除非很清楚,否则不要改动这个
     * bAllowLocalhost   是否允许localhost, 127.0.0.1 这样的referer的请求,默认是true,便于调试
     * bAllowAllIPAndHost  是否允许本机的所有IP和host的referer请求,默认是false
     *    
     * {@code
     * 	<filter>
     * 		<filter-name>refererFilter</filter-name>
     * 		<filter-class>com.test.RefererFilter</filter-class>
     * 		<init-param>
     * 			<param-name>completeMatchHosts</param-name>
     * 			<param-value>test.com|abc.com</param-value>
     * 		</init-param>
     * 		<init-param>
     * 			<param-name>allowSubDomainHosts</param-name>
     * 			<param-value>hello.com|xxx.yyy.com</param-value>
     * 		</init-param>
     * 	</filter>
     * 
     * 	<filter-mapping>
     * 		<filter-name>refererFilter</filter-name>
     * 		<url-pattern>/*</url-pattern>
     * 	</filter-mapping>
     * 	}
     * </pre>
     * 
     * @author hengyunabc
     *
     */
    public class RefererFilter implements Filter {
    	static final Logger logger = LoggerFactory.getLogger(RefererFilter.class);
    	public static final String DEFAULT_MATHMETHODS = "POST|PUT|DELETE|CONNECT|PATCH";
    
    	List<String> mathMethods = new ArrayList<>();
    
    	boolean bAllowEmptyReferer = false;
    
    	boolean bAllowLocalhost = true;
    	boolean bAllowAllIPAndHost = false;
    
    	/**
    	 * when bAllowSubDomain is true, allowHosts is "test.com", then
    	 * "www.test.com", "xxx.test.com" will be allow.
    	 */
    	boolean bAllowSubDomain = false;
    
    	String redirectPath = null;
    	int responseError = HttpServletResponse.SC_FORBIDDEN;
    
    	HashSet<String> completeMatchHosts = new HashSet<String>();
    
    	List<String> allowSubDomainHostList = new ArrayList<String>();
    
    	@Override
    	public void init(FilterConfig filterConfig) throws ServletException {
    		mathMethods.addAll(getSplitStringList(filterConfig, "matchMethods", "\\|", DEFAULT_MATHMETHODS));
    
    		completeMatchHosts.addAll(getSplitStringList(filterConfig, "completeMatchHosts", "\\|", ""));
    
    		List<String> allowSubDomainHosts = getSplitStringList(filterConfig, "allowSubDomainHosts", "\\|", "");
    		completeMatchHosts.addAll(allowSubDomainHosts);
    		for (String host : allowSubDomainHosts) {
    			// check the first char if is '.'
    			if (!host.isEmpty() && host.charAt(0) != '.') {
    				allowSubDomainHostList.add("." + host);
    			} else {
    				allowSubDomainHostList.add(host);
    			}
    		}
    
    		responseError = getInt(filterConfig, "responseError", responseError);
    		redirectPath = filterConfig.getInitParameter("redirectPath");
    
    		bAllowEmptyReferer = getBoolean(filterConfig, "bAllowEmptyReferer", bAllowEmptyReferer);
    
    		bAllowLocalhost = getBoolean(filterConfig, "bAllowLocalhost", bAllowLocalhost);
    		if (bAllowLocalhost) {
    			completeMatchHosts.add("localhost");
    			completeMatchHosts.add("127.0.0.1");
    			completeMatchHosts.add("[::1]");
    		}
    
    		bAllowAllIPAndHost = getBoolean(filterConfig, "bAllowAllIPAndHost", bAllowAllIPAndHost);
    		if (bAllowAllIPAndHost) {
    			completeMatchHosts.addAll(getAllIPAndHost());
    		}
    	}
    
    	@Override
    	public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException,
    			ServletException {
    		if (servletRequest instanceof HttpServletRequest && servletResponse instanceof HttpServletResponse) {
    			HttpServletRequest request = (HttpServletRequest) servletRequest;
    			HttpServletResponse response = (HttpServletResponse) servletResponse;
    
    			String method = request.getMethod();
    			/**
    			 * if method not in POST|PUT|DELETE|CONNECT|PATCH, don't check
    			 * referrer.
    			 */
    			if (!mathMethods.contains(method.trim().toUpperCase())) {
    				filterChain.doFilter(request, response);
    				return;
    			}
    
    			String referrer = request.getHeader("referer");
    
    			boolean bAllow = false;
    			if (isBlank(referrer)) {
    				bAllow = bAllowEmptyReferer;
    			} else {
    				URL url = null;
    				try {
    					url = new URL(referrer);
    					String host = url.getHost();
    					if (completeMatchHosts.contains(host)) {
    						bAllow = true;
    					} else {
    						for (String domain : allowSubDomainHostList) {
    							if (host.endsWith(domain)) {
    								bAllow = true;
    								break;
    							}
    						}
    					}
    				} catch (RuntimeException e) {
    					logger.error("illegal referrer! referrer: " + referrer, e);
    					bAllow = false;
    				}
    			}
    
    			if (bAllow) {
    				filterChain.doFilter(request, response);
    				return;
    			} else {
    				if (isBlank(redirectPath)) {
    					response.sendError(HttpServletResponse.SC_FORBIDDEN);
    				} else {
    					response.sendRedirect(request.getContextPath() + redirectPath);
    				}
    			}
    		} else {
    			filterChain.doFilter(servletRequest, servletResponse);
    		}
    	}
    
    	@Override
    	public void destroy() {
    
    	}
    
    	private static boolean isBlank(CharSequence cs) {
    		int strLen;
    		if (cs == null || (strLen = cs.length()) == 0) {
    			return true;
    		}
    		for (int i = 0; i < strLen; i++) {
    			if (Character.isWhitespace(cs.charAt(i)) == false) {
    				return false;
    			}
    		}
    		return true;
    	}
    
    	private static boolean getBoolean(FilterConfig filterConfig, String parameter, boolean defaultParameterValue) {
    		String parameterString = filterConfig.getInitParameter(parameter);
    		if (parameterString == null) {
    			return defaultParameterValue;
    		}
    		return Boolean.parseBoolean(parameterString.trim());
    	}
    
    	private static int getInt(FilterConfig filterConfig, String parameter, int defaultParameterValue) {
    		String parameterString = filterConfig.getInitParameter(parameter);
    		if (parameterString == null) {
    			return defaultParameterValue;
    		}
    		return Integer.parseInt(parameterString.trim());
    	}
    
    	/**
    	 * <pre>
    	 * getSplitStringList(filterConfig, "hosts", "\\|", "test.com|abc.com");
    	 * 
    	 * if hosts is "hello.com|google.com", will return {"hello.com", google.com"}.
    	 * if hosts is null, will return {"test.com", "abc.com"}
    	 * </pre>
    	 * 
    	 * @param filterConfig
    	 * @param parameter
    	 * @param regex
    	 * @param defaultParameterValue
    	 * @return
    	 */
    	private static List<String> getSplitStringList(FilterConfig filterConfig, String parameter, String regex, String defaultParameterValue) {
    		String parameterString = filterConfig.getInitParameter(parameter);
    		if (parameterString == null) {
    			parameterString = defaultParameterValue;
    		}
    
    		String[] split = parameterString.split("\\|");
    		if (split != null) {
    			List<String> resultList = new LinkedList<String>();
    			for (String method : split) {
    				resultList.add(method.trim());
    			}
    			return resultList;
    		}
    		return Collections.emptyList();
    	}
    
    	public static Set<String> getAllIPAndHost() {
    		HashSet<String> resultSet = new HashSet<String>();
    
    		Enumeration<NetworkInterface> interfaces;
    		try {
    			interfaces = NetworkInterface.getNetworkInterfaces();
    			while (interfaces.hasMoreElements()) {
    				NetworkInterface nic = interfaces.nextElement();
    				Enumeration<InetAddress> addresses = nic.getInetAddresses();
    				while (addresses.hasMoreElements()) {
    					InetAddress address = addresses.nextElement();
    					if (address instanceof Inet4Address) {
    						resultSet.add(address.getHostAddress());
    						resultSet.add(address.getHostName());
    					} else if (address instanceof Inet6Address) {
    						// TODO how to process Inet6Address?
    						// resultSet.add("[" + address.getHostAddress() + "]");
    						// resultSet.add(address.getHostName());
    					}
    				}
    			}
    		} catch (SocketException e) {
    			logger.error("getAllIPAndHost error!", e);
    		}
    		return resultSet;
    	}
    }

    ##其它的一些东东

    在浏览器里如何访问IPV6的地址?
    用”[]”把IPV6地址包围起来,比如localhost的:

    http://[::1]

    参考:

    • http://superuser.com/questions/367780/how-to-connect-a-website-has-only-ipv6-address-without-domain-name
    • https://msdn.microsoft.com/en-us/library/windows/desktop/ms740593(v=vs.85).aspx

    可能感兴趣的文章

    • Google I/O 2013和Android
    • HashMap和Hashtable的区别
    • 使用Spring跟踪应用异常(1)
    • 自己编写Java Web框架:Takes框架的Web App架构
    • 打造高性能JAVA应用你需要知道的
    • Java 8并行流:必备技巧
    • 诊断Java中的内存泄露
    • Java开发必会的Linux命令
    • CognitiveJ —— Java 的图像分析库
    • 一文让你明白Java字节码


沪ICP备19023445号-2号
友情链接