Custom PMD Rules

by Tom Copeland
04/09/2003

A Review of PMD

A few weeks ago, O'Reilly Network ran an article on PMD, an open source, Java static-analysis tool sponsored under the umbrella of the Defense Advanced Research Projects Agency (DARPA) project "Cougaar." That article covered some of the basics of PMD--it's built on an Extended Backus Naur Format (EBNF) grammar, from which JavaCC generates a parser and JJTree generates an Java Abstract Syntax Tree (AST), and comes with a number of ready-to-run rules that you can run on your own source code. You can also write your own rules to enforce coding practices specific to your organization.

In this article, we'll take a closer look at the AST, how it is generated, and some of its complexities. Then we'll write a custom PMD rule to find the creation of Thread objects. We'll write this custom rule two ways, first in the form of a Java class, and then in the form of an XPath expression.

The AST

Recall from the first article that the Java AST is a tree structure that represents a chunk of Java source code. For example, here's a simple code snippet and the corresponding AST:

Source Code	Abstract Syntax Tree
Thread t = new Thread();	FieldDeclaration Type Name VariableDeclarator VariableDeclaratorId VariableInitializer Expression PrimaryExpression PrimaryPrefix AllocationExpression Name Arguments

Here we can see that the AST is a standard tree structure: a hierarchy of nodes of various types. All of the node types and their valid children are defined in the EBNF grammar file. For example, here's the definition of a FieldDeclaration:

void FieldDeclaration() :
{
}
{
  ( "public"            { ((AccessNode) jjtThis).setPublic( true ); }
  | "protected"         { ((AccessNode) jjtThis).setProtected( true ); }
  | "private"           { ((AccessNode) jjtThis).setPrivate( true ); }
  | "static"            { ((AccessNode) jjtThis).setStatic( true ); }
  | "final"             { ((AccessNode) jjtThis).setFinal( true ); }
  | "transient"         { ((AccessNode) jjtThis).setTransient( true ); }
  | "volatile"          { ((AccessNode) jjtThis).setVolatile( true ); } )*

  Type() VariableDeclarator() ( "," VariableDeclarator() )* ";"
}

A FieldDeclaration is composed of a Type followed by at least one VariableDeclarator; for example, int x,y,z = 0;. A FieldDeclaration may also be preceeded by a couple of different modifiers, that is, Java keywords like transient or private. Since these modifiers are separated by a pipe symbol and followed by an asterisk, any number can appear in any order. All of these grammar rules eventually can be traced back to the Java Language Specification (JLS) (see the References section below).

Related Reading

Java Enterprise Best Practices
By The O'Reilly Java Authors

The grammar doesn't enforce nuances like "a field can't be both public and private". That's the job of a semantic layer that would be built into a full compiler such as javacor Jikes. PMD avoids the job of validating modifiers--and the myriad other tasks a compiler must perform--by assuming the code is compilable. If it's not, PMD will report an error, skip that source file, and move on. After all, if a source file can't even be compiled, there's not much use in trying to check it for unused code.

Looking closer at the grammar snippet above, we can also see some custom actions that occur when a particular token is found. For example, when the keyword publicis found at the start of a FieldDeclaration, the parser that JavaCC generates will call the method setPublic(true) on the current node. The PMD grammar is full of this sort of thing, and new actions are continually being added. By the time a source code file makes it through the parser, a lot of work has been done that makes rule writing much easier.

A Custom Rule

Now that we've reviewed the AST a bit more, let's write a custom PMD rule. As mentioned before, we'll assume we're writing Enterprise Java Beans, so we shouldn't be using some of the standard Java library classes. We shouldn't open a FileInputStream, start a ServerSocket, or instantiate a new Thread. To make sure our code is safe for use inside of an EJB container, let's write a rule that checks for Thread creation.

Writing a Custom Rule as a Java Class

Let's start by writing a Java class that traverses the AST. From the first article, recall that JJTree generates AST classes that support the Visitor pattern. Our class will register for callbacks when it hits a certain type of AST node, then poke around the surrounding nodes to see if it's found something interesting. Here's some boilerplace code:

// Extend AbstractRule to enable the Visitor pattern 
// and get some handy utility methods
public class EmptyIfStmtRule extends AbstractRule {
}

If you look back up at the AST for that initial code snippet--Thread t = new Thread();--you will find an AST type called an AllocationExpression. Yup, that sounds like what we're looking for: allocation of newThread objects. Let's add in a hook to notify us when it hits a new [something] node:

public class EmptyIfStmtRule extends AbstractRule {
    // make sure we get a callback for any object creation expressions
    public Object visit(ASTAllocationExpression node, Object data){
       return super.visit(node, data);
    }
}

We've put a super.visit(node,data) in there so the Visitor will continue to visit children of this node. This lets us catch allocations within allocations, i.e., new Foo(new Thread()). Let's add in an if statement to exclude array allocations:

public class EmptyIfStmtRule extends AbstractRule {
    public Object visit(ASTAllocationExpression node, Object data){
    // skip allocations of arrays and primitive types:
    // new int[], new byte[], new Object[] 
        if ((node.jjtGetChild(0) instanceof ASTName) {
            return super.visit(node, data);
        }
    }
}

We're not concerned about array allocations, not even Thread-related allocations like Thread[] threads = new Thread[];. Why not? Because instantiating an array of Thread object references doesn't really create any new Thread objects. It just creates the object references. We'll focus on catching the actual creation of the Thread objects. Finally, let's add in a check for the Thread name:

public class EmptyIfStmtRule extends AbstractRule {
    public Object visit(ASTAllocationExpression node, Object data){
        if ((node.jjtGetChild(0) instanceof ASTName &&
        ((ASTName)node.jjtGetChild(0)).getImage().equals("Thread")) {
            // we've found one!  Now we'll record a RuleViolation and move on
            ctx.getReport().addRuleViolation(
                createRuleViolation(ctx, node.getBeginLine()));
        }
        return super.visit(node, data);
    }
}

That about wraps up the Java code. Back in the first article, we described a PMD ruleset and the XML rule definition. Here's a possible ruleset definition containing the rule we just wrote:

<?xml version="1.0"?>
<ruleset name="My company's EJB checker rules">
  <description>
The Design Ruleset contains a collection of rules that find questionable designs.
  </description>
  <rule name="DontCreateThreadsRule"
        message="Don't create threads, use the MyCompanyThreadService instead"
        class="org.mycompany.util.pmd.DontCreateThreadsRule">
    <description>
Don't create Threads, use the MyCompanyThreadService instead.
    </description>

    <example>
<![CDATA[
 Thread t = new Thread(); // don't do this!
]]>
    </example>
  </rule>
</ruleset>

You can put this ruleset on your CLASSPATH or refer to it directly, like this:

java net.sourceforge.pmd.PMD /path/to/src xml /path/to/ejbrules.xml

Writing a Custom Rule as an XPath Expression

Recently Daniel Sheppard enhanced PMD to allow rules to be written using XPath. We won't explain XPath completely here--it would require a large book--but generally speaking, XPath is a way of querying an XML document. You can write an XPath query to get a list of nodes that fit a certain pattern. For example, if you have an XML document with a list of departments and employees, you could write a simple XPath query that returns all the employees in a given department, and you wouldn't need to write DOM-traversal or SAX-listener code.

Related Reading XPath and XPointer Locating Content in XML Documents By John E. Simpson

That's all well and good, but how does querying XML documents relate to PMD? Daniel noticed that an AST is a tree, just like an XML document. He downloaded the Jaxen XPath engine and wrote a class called a DocumentNavigator that allows Jaxen to traverse the AST. Jaxen gets the XPath expression, evaluates it, applies it to the AST, and returns a list of matching nodes to PMD. PMD creates RuleViolation objects from the matching nodes and moves along to the next source file.

XPath is a new language, though, so why write PMD rules using XPath when you're already a whiz-bang Java programmer? The reason is that it's a whole lot easier to write simple rules using XPath. To illustrate, here's the "DontCreateThreadsRule" written as an XPath expression:

//AllocationExpression[Name/@Image='Thread'][not(ArrayDimsAndInits)]

Concise, eh? There's no Java class to track--you don't have to compile anything or put anything else on your CLASSPATH. Just add the XPath expression to your rule definition like this:

<?xml version="1.0"?>
<ruleset name="My company's EJB checker rules">
  <description>
The Design Ruleset contains a collection of rules that find questionable designs.
  </description>
  <rule name="DontCreateThreadsRule"
        message="Don't create threads, use the MyCompanyThreadService instead"
        class="org.mycompany.util.pmd.DontCreateThreadsRule">
    <description>
Don't create Threads, use the MyCompanyThreadService instead.
    </description>
    <properties>
    <property name="xpath">
        <value>
        <![CDATA[
            //AllocationExpression[Name/@Image='Thread'][not(ArrayDimsAndInits)]>
        ]]>
        </value>
    </property>
    </properties>
    <example>
<![CDATA[
 Thread t = new Thread(); // don't do this!
]]>
    </example>
  </rule>
</ruleset>

Refer to the rule as usual to run it on your source code.

You can learn a lot about XPath by looking at how the built-in PMD rules identify nodes, and you can also try out new XPath expressions using a PMD utility called the ASTViewer. Run this utility by executing theastviewer.bat or astviewer.sh scripts in the etc/ directory of the PMD distribution. It will bring up a window that looks like Figure 1. Type some code into the left-hand panel, put an XPath expression in the text field, click the "Go" button at the bottom of the window, and the other panels will be populated with the AST and the results of the XPath query.

Figure 1. Screenshot of ASTViewer

When should you use XPath to write a PMD rule? My initial thought is, "Anytime you can." I think that you'll find that many simple rules can be written using XPath, especially those that are checking for braces or a particular name. For example, almost all of the rules in the PMD basic ruleset and braces ruleset are now written as very short, concise XPath expressions. The more complicated rules--primarily those dealing with the symbol table--are probably still easiest to write in Java. We'll see, though. At some point we may even wrap the symbol table in a DocumentNavigator.

Future Plans

There's still a lot of work to do on PMD. Now that this XPath infrastructure is in place, it might be possible to write an interactive rule editor. Ideally, you could open a GUI, type in a code snippet, select certain AST nodes, and an XPath expression that finds those nodes would be generated for you. PMD can always use more rules, of course. Currently, there are over 40 feature requests on the web site just waiting for someone to implement them. Also, PMD has a pretty weak symbol table, so it occasionally picks up a false positive. There's plenty of room for contributors to jump in and improve the code.

Conclusion

This article has presented a more in-depth look at the Java AST and how it's defined. We've written a PMD rule that checks for Thread creation using two techniques--a Java class and an XPath query. Give PMD a try and see what it finds in your code today!

Credits

Thanks to the Cougaar program and DARPA for supporting PMD. Thanks to Dan Sheppard for writing the XPath integration. Thanks also to the many other contributors without whom PMD would be a much less useful utility.

References

• PMD home page
• Jaxen home page
• DARPA home page
• Cougaar home page
• Java Language Specification
• W3C XQuery/XPath home page
• JavaCC/JJTree home page

Tom Copeland started programming on a TRS-80 Model III, but demand for that skill has waned and he now programs mostly in Java and Ruby.