Regular expressions are incredibly powerful, yet most developers don't really understand when or how to use them. This tutorial will take you from regular expression noob to master. (well ok, mabye not master but you will gain quite a bit of knowledge!)

What is Regular Expression Syntax?

It's basically a small language that is used to find things in strings. Think of something complex like a the html for a web page. Suppose you wanted to get only part of the page and return it. Sound simple? Trust me it's harder than you think once you start looking at real world examples, but this is an area that regular expressions shine. (more on that later.)

Testing your Expressions

.NET for some strange reason has no regular expression test client. Since regular expression syntax can be tricky to get right you should always test your expressions before you put them in your code. As a matter of fact I prefer to incrementally test mine, I'll write part of an expression and test that part, when I'm statisified it returns what I want I add more and more until I have the whole expression, basically it's just like coding anything else, you don't write the the whole thing and press run and expect it to work do ya?

There are many applications on the web you can download that let you test regular expressions, but I prefer an extremely small test application where little can go wrong. Here is what I do.

  1. Create a new windows application.
  2. Add a TextBox to your app called "txtExpression"
  3. Add two text boxes to your form and set the Multiline property on both of them to true. Name one "txtData" and the other "txtResults"
  4. Add this line to the top of your code. 
    using System.Text.RegularExpressions;
  5. Then add a button add put this code in the button 
    txtResults.Clear();

    Regex r = new Regex(txtExpression.Text,RegexOptions.Singleline | RegexOptions.IgnoreCase);

    MatchCollection col = r.Matches(txtData.Text);

    foreach (Match m in col)
    {
        for (int i=0;i>m.Groups.Count;i++)
        {
            txtResults.Text += "(" + i + ")" + m.Groups[i].Value + "\r\n";    
        }                
    }
    .

Congrats you now have a very quick and dirty regular expression test client! Paste your data in the txtData TextBox and put your regular expression in the txtExpression TextBox and when you press the button the result of the operation will go in the txtResults TextBox. Easy isn't it?

Making your first expression

Ok lets get on with it. Suppose you have a string like this: 

The date of that transaction is 2006-12-02 MST.
And you want to get the date out of it. We can write an expression such as this: 
\d{4}-\d{2}-\d{2}

When you put that in our test client and smack the button you get a "(0)" at the front. The test client we wrote does this because you can define multiple things you want to return in a regular expression, but for now just ignore the (0) at the front I'll go into this more later.

So what is that junk? Lets take a peek. "\d" for regular expressions means digit. So it simply matches a number. The next part "{4}" tell the previous part how many to match. So it says look for 4 digits. Well that matches 2006 in our example. The next part is "-" this is just a literal and matches a "-" in the target data. You can see the digit part being used again for 2 more digits and again for another two. So this says find 4 digits a - 2 digits a - and two more digits.

Try this one against the same expression: 

The date of that transaction is 2006-12-02 MST.
The date of that transaction is 2006-12-15 MST.

You will notice that two results are returned one for each date. It's easy to use regular expressions to pull out any number of items like that from a list. Very helpful indeed.

Wildcards and other fun stuff

Ok so you got digits, what else can it do you wonder? Well lets look at a more complex example, take this html snippet for instance: 

<h1>Welcome to My Page of Goodness</h1>
Thank you for visiting my page. Pick a topic:<br>
<ul>
  <li>Goodness</li>
  <li>Extra Goodness</li>
  <li>Ultra Goodness</li>
  <li>Scooby Goodness and snacks too!</li>
</ul>

Suppose you want to get each of the items in the list, how would you do it?

Well we can use the <li> tags as a marker and get the stuff between them. To do this in an expression would look like this: 

<li>.*?</li>

The "." represents any character or number, and the "*" is a wildcard that means 0 or more of. (yeah I know could have used + instead to do 1 or more of) Now the "?" is more interesting. Try this without the "?" and then with it and see what you get. You will notice that without it matches the first <LI> tag with the last </LI> tag instead of the first </LI> it comes across. The "?" forces it to do minimal matching so that it matches with the first one it finds. Be careful, a ? without a *, + or another ? in front has a different meaning, it means zero or one matches.

Using the same example we can produce other expressions that do the exact same thing such as this one: 

<li>[^<]*</li>

Some new syntax in there, the "[]" give context to the meaning in them. The "^" is used as a NOT symbol. So this just says match 0 or more characters except for the "<" character. So it goes until it hits the next tag and stops.

Capture groups are your friends!

Suppose you have an href tag like the one shown below and you want to get both the url and the title of the link? How would you do it? 

<a href="http://goodness.is.me.com">My site of wonderful goodness</a>

Obviously you could use the techniques shown previously and capture this twice but that is very wasteful. Regular expressions have a way to do this exact thing with ease, it's called capture groups. A capture group is just a pair of rounded brackets "( )" but it allows you to pull out more than one piece of data. Lets look at an example of how you could get the url and title from the link above. 

<a href="(.*?)">(.*?)</a>

First when you run this you will notice that you have three elements being returned (thats what the (0), (1), (2) at the begining represent, the array element (or capture group) of the item being returned. You should get the following output: 

(0)<a href="http://goodness.is.me.com">My site of wonderful goodness</a>
(1)http://goodness.is.me.com
(2)My site of wonderful goodness

The syntax of that command should all be familiar as we just covered the wildcards earlier. Notice that we now attempt to match the whole expression and have brackets around parts of it? The brackets in the expression define the capture groups. The Match object returned from a call to RegEx has an array of Group. This array has each of the capture groups in it. The whole regular expression gets captured and put in the 0 element. The first capture group goes in 1 and the second capture group goes in 2. Easy and very powerful to get multiple returns from a single expression match!

Just a quick note: You can give your capture groups names if you like, then in the code you can pull them out of the group array by name instead of index. You can do names like so: 

<a href="(?<url>.*?)">(?<title>.*?)</a>

Note that each capture group now has a ?<name> in it. this defines the capture groups name. (another note, whomever created this language loves the ? character!)

Alternation

Suppose you want to match two different things such as this: 

grey
gray

Yes I know you could wildcard it like gr\wy to get any word in that spot, but lets be a bit more specific so that it only matches "e" and "a". This is where alternation works. Basically it's just like doing a logical OR statement in c#. 

gr(?:e|a)y

I know, I know wtf is the (?: stuff all about. Well the brackets define a capture group and the ?: part tells it not to capture the group, so basically the brackets work more like brackets in math, they just group parts of the expression instead of capturing them.

A bit about options

You notice that the RegEx class can accept some options on the constructor, we used these when we created our test client: 

Regex r = new Regex(txtExpression.Text,RegexOptions.Singleline | RegexOptions.IgnoreCase);

The options are very important as they make a huge difference on what gets captured and what does not. Lets look at the two we used here.

The SingleLine option is described in the help as: "Specifies single-line mode. Changes the meaning of the period character (.) so that it matches every character (instead of every character except \n)."
So in short it means that your capture can span multiple lines, else when you do a "*" it stops at the end of the current line. I'm suprised that this isn't the default, as it has bitten me more times than once, so ensure you set the value of this to the function you desire.

Next up is the IgnoreCase option. This one should be self explanitory, it doesn't care what the case of the words are and matches regardless.

The only other one I find I use often is the Compiled option. This compiles the regular expression into your assembly so it goes much faster. But remember this will only work if you know the expression beforehand, if you are planning on providing the expression from a configuration file it will not work as it cannot compile it since it does not have the expression!

A word in parting

Feel like a RegEx master now? Ya me neither, but hey your a few steps closer, hopefully this tutorial has helped you determine when a regular expression can help you get the job done faster.