test

In the beginning, there were no testers at Microsoft, no localization engineers, no program managers,

and no usability engineers. In the beginning, there were just engineers; oh, and sales and marketing.

Chapter 1, "Software Engineering at Microsoft," introduced the 10 different engineering disciplines

Microsoft uses today to organize and develop our engineering workforce. Even before we began to

break out different engineering disciplines there were a few different titles, such as product support for

shipped products. Positions like these were seen as a role for an engineer rather than a different career

path. During these early days, all engineers had the same title and one common career path. Of course,

in these early days Microsoft had less than 50 employees, software wasn't even really an industry, and

Microsoft was not yet a public company.

The move to supporting separate disciplines with distinct career paths has taken quite a while to evolve.

Program Management and Usability are two of the older disciplines at Microsoft. Around 1990, the

Usability discipline, engineers that work to make our software easy to use for real-world users, became

a formal role and eventually a recognized discipline. Usability formed largely because of such features

as the Microsoft Office Word mail merge (think snail mail and printing out form letters or stick-on

labels) that just didn't seem to work for the average person. Some customers might argue that we are

still working to make mail merge easy to use, but that story is better suited for a book on how we

design software at Microsoft.

Shortly after the creation of the software development position, the software tester position broke off as

the second distinct discipline. As the story goes, the first-ever tester at Microsoft was a young high

school intern by the name of Lloyd Frink, who started in June 1979. The Microsoft Archive team

places the hiring of the first full-time tester to be 1983 followed by a wave of testers hired in 1985, as

illustrated in Figure 2-1. Test as a separate standard title with a fully articulated career path did not

show up until the late 1980s.

 

Figure 2-1: Seattle Times 1985 advertisement for software testers.

Maybe We Should Have Someone Test the Code Before We Ship It

I had met Bill a few times before¡ªthat is how I got the job. My mom knew Bill's mom, Mary. I was

going to school at Lakeside, and at a school fund-raising auction they were talking and found out both

of their sons were interested in computers. Both Bill and I were also at the auction, so they introduced

us. I was 14 at the time, and Bill was 24. We decided to all have lunch together. So, a few weeks later,

my mom and I went over to Microsoft to have lunch with Mary, Bill, and Bill's younger sister Libby,

who was one year ahead of me at high school. I showed Bill some of the computer games I had written

and sold, and he offered me a summer internship. That's how the whole thing got started.

The first summer I was mainly testing the BASIC Compiler for Greg Whitten. We had a bunch of

BASIC programs that I would run through the compiler and see if they worked."

¡ªLloyd Frink, former Microsoft employee and co-founder of zillow.com

 

 

 

What's in a Name?

A rose by any other name may still smell as sweet, but a title can have a significant impact on the prism

through which a person, or in this case an engineer, looks out onto his world. Developers at Microsoft

have the common title of Software Development Engineer, or SDE. They develop features by writing

code. The formal title for a software tester at Microsoft is Software Development Engineer in Test, or

SDET. The similarity in the names of the two disciplines is by design because testers at Microsoft are

developers. Among other things, testers design tests, influence product design, conduct root cause

analysis, participate in code reviews, and write automation. Occasionally testers check bug fixes or

work on small features. Testing is a heavy workload, so writing features is not very common for testers.

The concept of hiring a software engineer with a passion for testing is powerful and is the biggest

differentiator between the Microsoft approach to software testing and the typical industry approach.

The most common conclusion drawn is that we hire these "coders" for test positions because we want

to automate everything and eliminate manual testing. Although we do want testers who can write

effective automation, that's only a small part of the equation. Testers who understand programming

concepts and computer architecture typically have the analysis skills that testing requires. They can also

find bugs earlier and understand the root cause so that they can quickly find other similar bugs and

implement early detection. This strong grounding in computer science¡ªthe same grounding a

developer has¡ªreinforces the tester skills and gives us a more dynamic and flexible workforce of

testers.

A common question that comes up during industry events is why Microsoft doesn't hire subject matter

experts (SMEs) as testers. For example, international accounting rules are very complex and a tester

with just a background in engineering wouldn't know all the nuances. Another example is vertical

products such as customer relationship management (CRM) solutions. The theory is that it would be

better to hire an SME and Microsoft could focus on being really good at training our testers in

computer science and engineering skills. The counterargument to this question is whether a company

would hire a CPA, train that individual to be a world-class developer, and then have that individual

write the company's accounting solution. Of course, this approach just isn't practical. Learning to be a

great developer requires passion for technology and many years of formal training.

Just about every software company starts with a developer and teaches her the problem space and

customer scenarios for the product she will be developing. This holds true whether the company

develops operating systems or software to control the flow of power across electrical grids. With

testing, we actually have two challenges. First, we must teach the engineer to be an SME for the

product area she is working on, and second, we must teach her how to test.

The rule of thumb, therefore, is to hire someone who has solid engineering skills, who can code as well

as an entry-level developer, and who has the other attributes we look for in a great tester. We call these

attributes the tester DNA, and I discuss them later in this chapter.

As with any rule of thumb, there are exceptions. The vast majority of testers at Microsoft are

developers in test, but in some areas, we do find that we need some of the team members to be SMEs.

Global accounting rules specialists or researchers in speech recognition algorithms are two examples of

areas in which we hire a lot of SMEs as testers. As we move into more consumer products, we have

hired manufacturing process experts to test our designs against the needs of high-volume costcontrolled

manufacturing. In these cases, the title of the SME test engineers is usually not SDET but

rather something more appropriate such as Linguistics Test Engineer or Manufacturing Test Engineer.

p25