SQLChop、SQLWall(Druid)、PHP Syntax Parser Analysis

catalog

1. introduction
2. sqlchop sourcecode analysis
3. SQLWall(Druid)
4. PHP Syntax Parser
5. SQL Parse and Compile: Parse and compose 
6. sql-parser
7. PEAR SQL_Parser

1. introduction

SQLCHOP, This awesome new tool, sqlchop, is a new SQL injection detection engine, using a pipeline of smart recursive decoding, lexical analysis and semantic analysis. It can detect SQL injection query with extremely high accuracy and high recall with 0day SQLi detection ability, far better than nowadays' SQL injection detection tools, most of which based on regex rules. We proposed a novel algorithm to achieve both blazing fast speed and accurate detection ability using SQL syntax analysis.

0x1: Description

SQLChop is a novel SQL injection detection engine built on top of SQL tokenizing and syntax analysis. Web input (URLPath, body, cookie, etc.) will be first decoded to the raw payloads that web app accepts, then syntactical analysis will be performed on payload to classify result. The algorithm behind SQLChop is based on compiler knowledge and automata theory, and runs at a time complexity of O(N).

0x2: installation

//If using python, you need to install protobuf-python, e.g.:
1. wget https://bootstrap.pypa.io/get-pip.py
2. python get-pip.py
3. sudo pip install protobuf

//If using c++, you need to install protobuf, protobuf-compiler and protobuf-devel, e.g.:
1. sudo yum install protobuf protobuf-compiler protobuf-devel

//make
1. Download latest release at https://github.com/chaitin/sqlchop/releases
2. Make
3. Run python2 test.py or LD_LIBRARY_PATH=./ ./sqlchop_test

Relevant Link:

http://sqlchop.chaitin.com/demo
http://sqlchop.chaitin.com/
https://www.blackhat.com/us-15/arsenal.html#yusen-chen
https://pip.pypa.io/en/stable/installing.html

2. sqlchop sourcecode analysis

The SQLChop alpha testing release includes the c++ header and shared object, a python library, and also some sample usages.

0x1: c++ header

/*
 * Copyright (C) 2015 Chaitin Tech.
 *
 * Licensed under:
 *   https://github.com/chaitin/sqlchop/blob/master/LICENSE
 *
 */

#ifndef __SQLCHOP_SQLCHOP_H__
#define __SQLCHOP_SQLCHOP_H__

#define SQLCHOP_API __attribute__((visibility("default")))

#ifdef __cplusplus
extern "C" {
#endif

struct sqlchop_object_t;

enum {
  SQLCHOP_RET_SQLI = 1,
  SQLCHOP_RET_NORMAL = 0,
  SQLCHOP_ERR_SERIALIZE = -1,
  SQLCHOP_ERR_LENGTH = -2,
};

SQLCHOP_API int sqlchop_init(const char config[],
                             struct sqlchop_object_t **obj);
SQLCHOP_API float sqlchop_score_sqli(const struct sqlchop_object_t *obj,
                                     const char buf[], size_t len);
SQLCHOP_API int sqlchop_classify_request(const struct sqlchop_object_t *obj,
                                         const char request[], size_t rlen,
                                         char *payloads, size_t maxplen,
                                         size_t *plen, int detail);

SQLCHOP_API int sqlchop_release(struct sqlchop_object_t *obj);

#ifdef __cplusplus
}
#endif

#endif // __SQLCHOP_SQLCHOP_H__

Relevant Link:

https://github.com/chaitin/sqlchop/releases
https://github.com/chaitin/sqlchop

3. SQLWall(Druid)

0x1: Introduction

git clone https://github.com/alibaba/druid.git
cd druid && mvn install

Druid提供了WallFilter，它是基于SQL语义分析来实现防御SQL注入攻击的，通过将SQL语句解析为AST语法树，基于语法树规则进行恶意语义分析，得出SQL注入判断

0x2: Test Example

/*
 * Copyright 1999-2101 Alibaba Group Holding Ltd.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.alibaba.druid.test.wall;

import java.io.File;
import java.io.FileInputStream;

import junit.framework.TestCase;

import com.alibaba.druid.util.Utils;
import com.alibaba.druid.wall.Violation;
import com.alibaba.druid.wall.WallCheckResult;
import com.alibaba.druid.wall.WallProvider;
import com.alibaba.druid.wall.spi.MySqlWallProvider;

public class MySqlResourceWallTest extends TestCase {

    private String[] items;

    protected void setUp() throws Exception {
//        File file = new File("/home/wenshao/error_sql");
        File file = new File("/home/wenshao/scan_result");
        FileInputStream is = new FileInputStream(file);
        String text = Utils.read(is);
        is.close();
        items = text.split("\\|\\n\\|");
    }

    public void test_false() throws Exception {
        WallProvider provider = new MySqlWallProvider();
        
        provider.getConfig().setConditionDoubleConstAllow(true);
        
        provider.getConfig().setUseAllow(true);
        provider.getConfig().setStrictSyntaxCheck(false);
        provider.getConfig().setMultiStatementAllow(true);
        provider.getConfig().setConditionAndAlwayTrueAllow(true);
        provider.getConfig().setNoneBaseStatementAllow(true);
        provider.getConfig().setSelectUnionCheck(false);
        provider.getConfig().setSchemaCheck(true);
        provider.getConfig().setLimitZeroAllow(true);
        provider.getConfig().setCommentAllow(true);

        for (int i = 0; i < items.length; ++i) {
            String sql = items[i];
            if (sql.indexOf("''=''") != -1) {
                continue;
            }
//            if (i <= 121) {
//                continue;
//            }
            WallCheckResult result = provider.check(sql);
            if (result.getViolations().size() > 0) {
                Violation violation = result.getViolations().get(0);
                System.out.println("error (" + i + ") : " + violation.getMessage());
                System.out.println(sql);
                break;
            }
        }
        System.out.println(provider.getViolationCount());
//        String sql = "SELECT name, '******' password, createTime from user where name like 'admin' AND (CASE WHEN (7885=7885) THEN 1 ELSE 0 END)";

//        Assert.assertFalse(provider.checkValid(sql));
    }

}

Relevant Link:

https://raw.githubusercontent.com/alibaba/druid/master/src/test/java/com/alibaba/druid/test/wall/MySqlResourceWallTest.java
https://github.com/alibaba/druid/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98
https://github.com/alibaba/druid/wiki/%E9%85%8D%E7%BD%AE-wallfilter
https://github.com/alibaba/druid
http://www.cnblogs.com/LittleHann/p/3495602.html
http://www.cnblogs.com/LittleHann/p/3514532.html

4. PHP Syntax Parser

<?php 
    require_once('php-sql-parser.php');

    $sql = "select name, sum(credits) from students where name='Marcin' and lvID='42509';";
    echo $sql . "\n";
    $start = microtime(true);
    $parser = new PHPSQLParser($sql, true); 
    var_dump($parser->parsed);
    echo "parse time simplest query:" . (microtime(true) - $start) . "\n"; 
?>

Relevant Link:

http://files.cnblogs.com/LittleHann/php-sql-parser-20131130.zip

5. SQL Parse and Compile: Parse and compose

This package can be used to parse and compose SQL queries programatically.
It can take an SQL query and parse it to extract the different parts of the query like the type of command, fields, tables, conditions, etc..
It can also be used to do the opposite, i.e. compose SQL queries from values that define each part of the query.

0x1: Features

I. Parser
- insert
- replace
- update
- delete
- select
- union
- subselect
- recognizes flow control function (IF, CASE - WHEN - THEN)
- recognition of many sql functions

II. Composer (Compiler)
- insert
- replace
- update
- delete
- select
- union

III. Wrapper SQL
- object oriented writing of SQL statements from the scratch

0x2: Example

#################################################
$insertObject = new Sql();
$insertObject
->setCommand("insert")
->addTableNames("employees")
->addColumnNames(array("LastName","FirstName"))
->addValues(
array(
array("Value"=>"Davolio","Type"=>"text_val"),
array("Value"=>"Nancy","Type"=>"text_val"),
)
);
$sqlout = $insertObject->compile();
#################################################

result:
echo $sqlout;
#################################################
INSERT INTO employees (LastName, FirstName) VALUES ('Davolio', 'Nancy')
#################################################

Relevant Link:

http://www.phpclasses.org/package/5007-PHP-Parse-and-compose-SQL-queries-programatically.html

6. sql-parser

A validating SQL lexer and parser with a focus on MySQL dialect

Relevant Link:

https://github.com/dmitry-php/sql-parser
https://github.com/udan11/sql-parser/wiki/Overview
https://github.com/udan11/sql-parser/wiki/Examples

7. PEAR SQL_Parser

Relevant Link:

https://pear.php.net/package/SQL_Parser/docs/latest/elementindex_SQL_Parser.html
https://pear.php.net/package/SQL_Parser/docs/latest/__filesource/fsource_SQL_Parser__SQL_Parser-0.6.0SQLParserDialectANSI.php.html
https://pear.php.net/package/SQL_Parser/docs/latest/SQL_Parser/SQL_Parser_Compiler.html
https://pear.php.net/package/SQL_Parser/docs/latest/__filesource/fsource_SQL_Parser__SQL_Parser-0.6.0SQLParserCompiler.php.html
https://pear.php.net/package/SQL_Parser/docs/latest/__filesource/fsource_SQL_Parser__SQL_Parser-0.6.0SQLParser.php.html