摘自:http://www.ibm.com/developerworks/cn/rational/05/1108_kelly/
级别： 中级
Michael Kelly (Mike@MichaelDKelly.com), Consultant, www.MichaelDKelly.com
2006 年 10 月 27 日
这是应用 IBM Rational Functional Tester 实现测试自动化框架的三部系列的第二篇文章，重点在于创建一个数据驱动的框架。本系列的其他文章涵盖了模块化和关键字驱动的框架。
编者注: 本文是基于使用 IBM® Rational® Functional Tester for Java™ and Web 6.1 和 Windows XP Professional SP2 编写的。代码实例将使用 Java 编写的，但所有概念同样也适用于 Rational Functional Tester 的 .NET 版本。
这是应用 IBM Rational Functional Tester 实现测试自动化框架的三部系列的第二篇文章。一个测试自动化框架是为软件自动化测试提供支持的一组假设、概念和实践。在本系列中所涵盖的三种框架类型包括模块化、数据驱动和关键字驱动。本文将特别侧重在创建一个数据驱动框架的方面。
数据驱动测试是一项单个测试脚本使用不同的输入和响应数据被重复地执行的技术，这些数据来源于一个预定义的数据集。在 Rational Functional Tester 中，数据驱动测试可以通过不同的数据池来实现，数据池是相关数据记录的集合，用以在测试脚本回放时为测试脚本提供变量的值。当您使用数据池时，Rational Functional Tester 能在您每次回放脚本时将一组不同的测试数据或相同的测试数据提交到测试脚本。您使用数据池来提供真实的数据，并用不同的数据和实际的数据量来产生对应用程序压力。当您运行一个数据驱动脚本时，Rational Functional Tester 将从数据池的当前行中输入每一个值（数据池中的一行看起来就象电子数据表中的一行）到应用程序的适当字段中。在每次执行脚本时，假定数据池光标在前面（数据池光标是对当前行的指针），下一个数据行将被输入。
在这个系列的三种框架中，数据驱动是最容易在 Rational Functional Tester 中实现的。下一部分我们将着眼于 Rational Functional Tester 中的数据池，帮您了解一下这个工具的内置特性。之后，文章将对这种方法进行利弊讨论。
在 Rational Functional Tester 中使用数据池进行工作
在 Rational Functional Tester 中，一个数据池是相关数据记录的一个集合，用以在测试脚本回放时为测试脚本提供变量的值。当您在 Rational Functional Tester 里创建一个数据驱动测试时，您可以用不同的输入和响应数据来重复地使用单个测试脚本。下面介绍的是一些在 Rational Functional Tester 里有关数据池的术语，以及如何创建和使用一个数据池。
Rational Functional Tester 通过引用数据驱动一个测试脚本来使用数据池。当您在文档中看到这个术语时，它提到了构建您的测试脚本中的对象和您数据池中的数据之间的连通性的过程。要数据驱动一个脚本，您需要在被测试应用程序中选择一个对象，然后 Rational Functional Tester 将应用程序里的数据填充到一个数据池。
在 Rational Functional Tester 中使用数据池进行工作
让我们使用 Rational 的测试应用程序 ClassicsCD 来记录一个简单的数据池。
- 在 Rational Functional Tester 里开始记录一个新的测试脚本。
- 在如图1所示的 Select Script Assets 对话框中，您将在开始记录之前能够看到两个数据池选项：Test Datapool 和 Datapools Record Selection Order。
图1. Select Script Assets 对话框
Test Datapool 的默认值是 Private Test Datapool。您创建的每一个测试脚本都有一个私有的测试数据池与其相关联－－这是自动生成的。初始的私有测试数据池是一个占位符，并且直到您给它增加新数据前都是空的。您可以通过创建一个新数据池，或者通过将几个测试脚本关联到一个数据池来创建一个被称为共享数据池的数据池。
Datapool Record Selection Order 框的值是 Sequential 和 Random。Sequential 指在回放时，测试脚本访问数据池中的记录是按照它们出现在数据池中的顺序来进行。Random 是指在回放时，测试脚本每次随机地访问数据池中的每条记录。
- 这此例中，接受缺省设置并点击 Finish。
- 这样就应该打开了 Recording Monitor 窗口。
- 启动 ClassicsCD (ClassicsJavaA.java) 样例应用程序，并导航到如图2所示的 Member Login 窗口。
图2. ClassicsCD Member Login 窗口
- 在 Recording 工具栏上，点击 Insert Data Driven Commands。
- 测试脚本记录暂停下来，并且打开了如图3所示的 Insert Data Driven Actions 对话框。
在 Insert Data Driven Actions 对话框里有两个组合框：Populate then Select Test Objects 和 Data Driven Commands。
图3. Insert Data Driven Actions 对话框
在 Populate then Select Test Objects下，有两种方法: Press and drag hand to select test objects 和 Use selection wizard to select test objects。Press and drag hand to select test objects 方法选择一个对象以及所选对象的所有下级对象。这是选择一个对象的最普通和直接的方法。Use selection wizard to select test objects 方法被用来激活 Drag Hand Selection 方法及其选项，或 Test Object Browser 方法。
- 使用鼠标将 Object Finder 工具拖拽到 Member Login 窗口中的Full Name 字段。
- Rational Functional Tester用一个红色边框画出对象的轮廓。
- 释放鼠标按钮。
- 您现在应当在如图4所示的 Insert Data Driven Actions 对话框的 Data Driven Commands 区域中，看到所列出的测试对象。
图4. 带有测试对象的 Insert Data Driven Actions 对话框
- 点击 OK 关闭 Insert Data Driven Actions 对话框。
- 点击 Member Login 窗口上的 Cancel 。
- 退出 ClassicsCD 应用程序。
- 停止记录。
如果您在刚刚记录的脚本里查看代码，您应当看到类似于列表1中的一行代码。
列表1. 在 Rational Functional Tester 中记录的数据池调用
// Data Driven Code inserted on Sep 14, 2005
nameCombo().select(dpString("nameCombo"));
|
在这段代码中，您选择与命名为nameCombo的数据池列中的值相匹配的组合框值。如果您在 Script Explorer 中打开 Private Test Datapool，您应当看到在第一行中所列出的您所记录的数据，如图5所示。
图5. 在 Private Test Datapool 中记录的数据
在 Rational Functional Tester 中的一个数据池中，数据池列被称为变量，行被称为记录（不要问我为什么）。您只要在数据池表格（如图5所示）中右键点击，将显示一列数据池编辑命令，如图6所示。
图6. 在 Rational Functional Tester 里编辑一个数据池
所有这些命令正确地执行了您所期望它们做的；不要惊讶。唯一不值得的事情是，当您声明一个新的变量（或者列）时，需要为变量指定 Type。当声明类型时，您将要输入类的完全路径。例如，使用java.lang.String，而不是简单地键入String。
Rational Functional Tester 提供了一种能力，可以用一个数据池引用（一个数据池引用是对有一个关联数据池的另一种说法）来查找或替换一个测试脚本中的文字值。在工具栏或者脚本菜单中，您可以访问 Datapool Literal Substitution 对话框，如图7所示。
图7. Datapool Literal Substitution 对话框
您可以在 Datapool Literal Substitution 对话框中设置选项，用一个数据池引用来查找和替换一个脚本中的所有的、数字、字符串或布尔型文字。您也可以从一个脚本中增加一个文字到一个数据池中。如果您不使用一个已有的数据池变量，Functional Test 就在您每次运行脚本时，使用相同的文字值（Functional Test 在您记录测试脚本时捕获的值）。
您可以使用一个数据池引用来替代您正在验证点中测试的值的一个文字值。在记录时，如果您使用 Verification Point and Action 向导插入一个证实点，您可以通过点击向导的工具栏中的 Convert Value to Datapool Reference 按钮，将验证点值转换为一个数据池引用。这个按钮用红色圆圈标记，如图8所示。
图8. 将验证点转换成数据池引用
在您完成记录测试脚本之后，您也可以向数据池中添加数据。
如果您在 IBM® Rational® TestManager 中已经有数据池了，那就没问题了。您可以将它们导入到 Rational Functional Tester 中，同样您也可以使用电子数据表格应用程序创建的任何简单格式的已有.csv文件。反过来也一样。只要您将所有数据放入 Rational Functional Tester 中，您就可以将数据池导入到一个 .csv 文件中，用一个电子数据表格应用程序来编辑它。在 Rational Functional Tester 的帮助文件中有如何进行这样做的指导。
一个用 Rational Functional Tester 进行数据驱动测试的简单例子
好了，已经向您介绍了使用数据池的基础知识，现在您将看一个有关数据驱动测试更加详细的例子。这个例子将使用 http://www.google.com/作为测试。Google Web 搜索有一些您将要测试的内置特性。这些特性包括：计算器、货币兑换、电影、号码搜索，以及旅游信息（以命名包含在强大的小文本字段中的许多特性）。记录的脚本如列表2所示，启动浏览器访问 Google.com ，然后执行一系列搜索和验证点，以验证上述特性。
列表2. 记录和回放 Google Web 搜索脚本
package tests;
import resources.tests.google_search_recordHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class google_search_record extends google_search_recordHelper
{
public void testMain(Object[] args)
{
startApp("www.Google.com");
// Calculator
text_q2().click(atPoint(191,10));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputKeys("5{+}5");
button_googleSearchsubmit2().click();
calculator_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome2().click();
// Currency Conversion
text_q2().click(atPoint(192,13));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputKeys("3.5 USD in GBP");
button_googleSearchsubmit2().click();
currency_conversion_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome3().click();
// Movies
text_q2().click(atPoint(98,14));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars(
"charlie and the chocolate factory");
button_googleSearchsubmit2().click();
movies_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome6().click();
// Search by Number
text_q2().click(atPoint(177,13));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars("patent 5123123");
button_googleSearchsubmit2().click();
search_by_number_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome8().click();
// Travel Information
text_q2().click(atPoint(163,10));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars("United 134");
button_googleSearchsubmit2().click();
travel_info_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome10().click();
// Close the browser
browser_htmlBrowser(document_google2(),MAY_EXIT).close();
}
}
|
在图2所示的脚本中，测试的每个特性都记录了在搜寻文本字段中进行一下点击、输入搜索标准、点击搜索按钮、验证结果，并在结果页的左上角点击 Google 图象以为下个特性测试重新设置浏览器。在这里有许多重复的代码。列表3显示了相同的测试脚本，只是脚本现在是数据驱动的。
列表3. 数据驱动的 Google Web 搜索脚本
package tests;
import resources.tests.google_datadrivenHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class google_datadriven extends google_datadrivenHelper
{
public void testMain(Object[] args)
{
startApp("www.Google.com");
// Run test for each value in datapool
for(int i = 0; i<5; i++)
{
// Search
text_q().click(atPoint(71,11));
browser_htmlBrowser(document_google(),
DEFAULT_FLAGS).inputChars(
dpString("searchCriteria"));
button_googleSearchsubmit().click();
// Results
searchResults_textVP().performTest(2.0, 20.0);
image_goToGoogleHome().click();
// Increment the datapool
dpNext();
}
// Exit
browser_htmlBrowser(document_google2(),MAY_EXIT).close();
}
}
|
通过使用一个for循环（您也可以使用一些其它类型的循环），并且通过增量的移动数据池里的光标，您可以避免多次重复相同的代码。您所做的全部就是在 inputChars 中为搜索字段增加一个数据池调用，并向您的验证点上增加一个数据池引用。在您想要增加另一个测试时，只要向您的数据池中增加另一行，并在循环中增加计数器的值。在图9中显示的数据池显示了每个测试的搜索标准和搜索结果。
图9. 数据驱动脚本的数据池值
当您比较 Google 记录的测试与 Google 数据驱动的测试时，您将看到的最大区别是事实上什么已经被测试了的可见性。在记录和回放脚本中，您必须滚动代码以发现搜索标准。在数据驱动的测试中，您需要做的所有事情就是查看数据池中的 searchCriteria 变量（列）。此外，在记录和回放脚本中，您需要分别查找和打开每个验证点以查找搜索结果。在数据驱动的测试中，所有您需要做的就是查看数据池中的 searchResults 变量。忘记有更少的代码吧，了解数据驱动测试在测试什么将更容易。
数据选择是沿着数据驱动测试之路的第一步。您需要选择驱动您的应用程序运行的数据，或者表示将输入到您的应用程序中的数据，或者两个都选。本节将迅速查看一下选择数据进行测试的5 种不同方式：基于风险、基于需求、基于可用性、使用生产数据，或者随机使用产生的数据。
选择基于风险的数据。选择测试数据的头号标准应当是风险。当您识别风险时，您要考虑到什么会导致出错。您正在寻找可能会发生的事件，这些事件将减少您能够交付带有正确特性、所要求的质量级别和在预算之内的项目的可能性。有三种划分风险种类的方法：
- 按照风险的影响 －－ 如果风险突然出现，与计划在进度、工作量或成本上的偏离
- 按照风险发生的可能性 －－ 风险突然出现的可能性（通常用百分比表示）
- 按照风险的暴露 －－ 影响乘以发生的可能性
选择基于需求的数据。您也可以选择将使您明确地测试一个需求或一组需求的数据。查找将使您在您的应用程序中运行特性集、性能和安全特性的数据：
- 如果您的应用程序有不同的任务，执行每个任务您将需要什么数据？
- 您想要在您的测试覆盖中包括什么特性，以及您要使用它们要求什么数据？
此外，考虑在被测试的元素上不同的目标部署环境将有什么冲突。您所选择的数据列表应当包括针对被测试应用和目标环境的数据。您需要测试什么数据：
- 硬件设备
- 设备驱动程序
- 操作系统
- 网络和通信软件
- 第三方的基本软件组件（例如，e-mail 软件，互联网浏览器）
- 与所有这些元素的可能组合相关联的各种配置和设置
- 国际化
选择基于可用性的数据。您可能想要选择那些易于可用的数据，这可以包括：
- 用一种易于访问的格式生产数据（在以后的章节里详细讨论）
- 来自于过去迭代的数据
- 您的项目的手工测试人员所使用的电子数据表格
- 来自于您公司里的其他项目或团队的数据
- 来自于一些数据产生源的数据（在最后的一章里讨论）。
这里的想法是，如果数据是容易访问的，也是易用和有意义的，在您的测试中包括这些数据，可以节省时间和金钱。我强调易用性和有意义的，是因为您需要记住一件重要的事情，您不应该只是因为数据已经在那了并且是可以使用的就选择它－－因为它可能是坏数据。
使用生产数据。收集测试数据的另一个策略是使用生产数据。尽管您不应当单独地依赖这种类型的数据，它可以是自动化测试场景的最丰富的来源，因为数据是应用程序将面临的真实场景的代表，并且因为它很可能将提供大量的不同场景。在一个最近的项目中，我们大约每周从一张表格中读取一次生产数据，并且我们在没有付出多少努力的情况下得到了300到500个场景的奖赏。您可以直接将数据导入测试环境里，将它读入到数据文件中用于后面的处理，或者实时地读取它，并按照您所的需要的他们进行转换。生产数据也是并行测试的一个极好来源。如果您在正在开发的系统中使用生产数据，您就会迅速地知道是否被开发的系统能够象在生产环境一样工作。这项技术特别有助于发现使用浮点值、转换率，以及与数据类型相关长度的问题。
但是，使用生产数据有一些需要小心的地方。生产数据很可能不包含您想要测试的许多特殊情况, 并且这不是理想测试场景的一个替换。还有一些潜在的法律问题围绕着生产数据的使用。特别是如果您把一些您的测试外包给另一个公司，您将想要确保检查您的公司在生产数据使用上的公司政策；如果没有已有的正式政策，您需要询问您公司法律部门的某些人。即使如果您不能直接使用生产数据，您也可能有更改某些值（名字，社会安全号，等等)和使用其余数据的可能性。
使用随机数据生成。很多工具包括测试数据生成程序。随机数据生成程序在产生大量用户数据的过程中特别有帮助（这取决于您正在测试什么和您想在产生过程中花费多长时间）。例如，如果您需要对金融信息进行关于舍入错误的测试，您可以：
- 随机生成大量数据
- 通过您的应用程序运行数据
- 通过运行 Windows 计算器（假定 Windows 的计算器没有舍入错误）来模拟相同的计算
- 比较结果以确保他们匹配
如果您需要，例如，要生成一组500个不同的客户名称及地址，您可以有一个生成程序(TestManager，Excel，Google Sets，等等)来创建一个带有名称和地址的表。
您应当避免一次选择您所有的测试用例中的数据。相反，采取一种增量和迭代的使用测试数据的方法，将您的工作量集中在您认为最有可能产生的对于既定测试周期有用的评估信息上。这会帮助缓和在一组单个数据或者测试类型上付出太多时间的风险 －－ 疏忽其它数据或测试任务 －－并且最小化在某些今后可能并不受关注的测试想法的数据上花费工作量的风险。
在本系列所涵盖的三类框架中（模块化、数据驱动, 以及关键字驱动），数据驱动框架对于非技术人员是最容易实现的，因为几乎所有的功能都被构建在 Rational Functional Tester 中，并且支持执行记录和回放。这对于非技术人员（对于我来说没有充分的理由推荐它）来说，不但是简单的，而且是强大的。它导致了半自动文档化测试用例，减少了创建大量测试用例所要求的代码数量，并且使您可以进行一些测试类型，否则您必须进行耗时或麻烦的手工测试工作。
数据驱动框架的主要优势是它减少了为很多类型的应用程序创建测试用例的成本。例如，在金融服务或者保险应用程序中数据驱动框架是非常有价值的。在您查看一千个变量中的一个变量是如何影响一笔复杂计算的最终金额的地方，对于金融服务或保险应用程序，很多次测试用例都只有细微的差别。对于一个数据驱动的工具，您将会很难找到一个更好的市场。另一方面，我曾经参与一个涉及化学制图和化学合成分析软件有关的项目。对于这个应用程序，测试用例非常特殊和集中（并且依靠于 GUI），以至于数据驱动框架就没有意义了。
数据驱动框架最大的缺点之一是，它创建新的测试用例太容易了。因为测试用例易于创建，您能够简单地运行和维护测试，这有时会导致测试用例膨胀。这不一定是一件坏事 －－这就是您为什么想要使用类似的工具的原因之一。如果您只是因为测试用例容易创建，而不投入许多时间和金钱来维护测试集，那就要小心了。如果只是为了看到发生什么而创建一个廉价的测试并运行一次，那可能是值得的，但是一旦您开始在您的应用程序的多个版本上开始维护测试，测试的成本就开始提高了。您想要确保维护成本的合理性是足够有力的。
考虑一下能在您的自动化测试中使用的一些数据驱动测试的方式。寻找一下您发现您自己一再调用相同代码的地方 －－就有可能发现在这些地方使用数据池的机会。不要觉得好像如果您想要进行数据驱动测试，您就需要立刻转换您所有的脚本。您不用这样。只要创建所需要的数据池，并且随着时间的过去，您将会把您的测试自动化发展成一个框架，在框架中，您将数据池作为脚本生成的一个基本部分。在许多应用程序中，对已有的数据池有高级的重用。正如 DBA 花费时间组织它们的数据一样，您也要花费时间组织您的数据。
学习
- 您可以参阅本文在 developerWorks 全球网站上的 英文原文。
- Using cost-benefit analysis to compare different test structures for Rational Robot:：Michael Kelly
- Choosing a test automation framework：Michael Kelly
- Using IBM Rational Functional Tester to test financial services applications：Michael Kelly
- Totally Data-Driven Automated Testing：Keith Zambelich
- Improving the Maintainability of Automated Test Suites: Cem Kaner
- 要学习更多内容，请访问 developerWorks Rational 专区上的 Functional Tester 产品资源区域 。您将会找到技术文档、指南、教育、下载、产品信息，以及更多信息。
获得产品和技术
- 通过访问试用版本下载区域 来试用 Rational 产品。
讨论
Michael Kelly 目前是一位独立的顾问，并且提供 IBM Rational 测试工具的定制培训。他进行咨询、写书并且演讲有关软件测试方面的主题。目前他担任印第安纳波利斯质量保证协会的项目主管，并且是软件测试协会的主管。您可以通过他的电子邮件 Mike@MichaelDKelly.com 联系他。
Framework automation with IBM Rational Functional Tester: Data-driven
Level: Intermediate
Michael Kelly (mailto:Mike@MichaelDKelly.com?subject=Data-driven), Consultant, www.MichaelDKelly.com
08 Nov 2005
Updated 21 Nov 2005
This second article in a three-part series on test automation frameworks with IBM Rational Functional Tester looks at creating a data-driven framework. The other articles cover modularity and keyword driven frameworks.
Editor’s Note: This article was written using the IBM® Rational® Functional Tester for Java™ and Web 6.1 and Windows XP Professional SP2. Code examples will be in Java, but all the concepts apply to the .NET version of Rational Functional Tester as well.
This article is the second in a three-part series on test automation frameworks with IBM Rational Functional Tester. A test automation framework is a set of assumptions, concepts, and practices that provide support for automated software testing. The three types of frameworks covered in this series are modularity, data-driven, and keyword driven. This article will be looking specifically at creating a data-driven framework.
Data-driven testing is a technique where a single test script is used repeatedly with varying input and response data that comes from a predefined data set. In Rational Functional Tester, data-driven testing can be implemented via datapools, which are collections of related data records that supply values for the variables in a test script during script playback. When you use datapools, Rational Functional Tester can deliver a different set of test data or the same set of test data to a script each time you play it back. You use datapools to supply realistic data and to stress an application with a variety of data and with a realistic amount of data. When you run a data-driven script, Rational Functional Tester will enter each of the values from the current row of the datapool (a row in a datapool looks like a row in a spreadsheet) into the appropriate fields of the application. Every time the script is executed, assuming the datapool cursor is advanced (a datapool cursor is a pointer to the current row), the next row of data will be entered.
Of the three frameworks in the series, data-driven is the one most easily implemented in Rational Functional Tester. The next section will look at datapools in Rational Functional Tester, helping you get a feel for some of the built-in features of the tool. After that, the article will discuss the advantages and disadvantages to this type of approach.
Working with datapools in Rational Functional Tester
In Rational Functional Tester, a datapool is a collection of related data records, which supplies data values to the variables in a test script during test script playback. When you create a data-driven test in Rational Functional Tester, you can use a single test script repeatedly with varying input and response data. The following is a look at some of the terminology of datapools in Rational Functional Tester, along with a look at how to create and use a datapool.
Rational Functional Tester refers to using a datapool as data-driving a test script. When you see this term in documentation, it is referring to the process of building the connectivity between objects in your test script and the data in your datapool. To data-drive a script, you select an object in the application-under-test, and Rational Functional Tester then populates a datapool with data from the application.
Working with datapools in Rational Functional Tester
Let’s take a look at recording a simple datapool using the Rational test application ClassicsCD.
- Start recording a new test script in Rational Functional Tester.
- In the Select Script Assets dialog box shown in Figure 1, you will see a couple of datapool options before you even begin recording: Test Datapool and Datapools Record Selection Order.
Figure 1. Select Script Assets dialog
The default value for Test Datapool is the Private Test Datapool. Every test script that you create has a private test datapool associated with it -- this happens automatically. The initial private test datapool is a placeholder and is empty until you add new data to it. You can create what’s called a shared datapool either by creating a new datapool, or by associating a datapool with several test scripts.
Values for the Datapool Record Selection Order box are Sequential and Random. Sequential means that at playback, the test script accesses records in the datapool in the order that they appear in the datapool. Random means that at playback, the test script randomly accesses every record in the datapool once.
- In this case, accept the default settings and click Finish.
- This should open the Recording Monitor window.
- Start the ClassicsCD (ClassicsJavaA.java) sample application and navigate to the Member Login window shown in Figure 2.
Figure 2. ClassicsCD Member Login window
- On the Recording toolbar, click Insert Data Driven Commands.
- The test script recording pauses and the Insert Data Driven Actions dialog shown in Figure 3 opens.
In the Insert Data Driven Actions dialog there are two group boxes: Populate then Select Test Objects and Data Driven Commands.
Figure 3. Insert Data Driven Actions dialog
Under Populate then Select Test Objects, there are two methods: Press and drag hand to select test objects and Use selection wizard to select test objects. The Press and drag hand to select test objects method selects an object and all the descendents of the selected object. This is the most common and direct method of selecting an object. The Use selection wizard to select test objects method is used to invoke the Drag Hand Selection method with its options, or the Test Object Browser method.
- Use the mouse to drag the hand, the Object Finder tool, to the Full Name field in Member Login window.
- Rational Functional Tester outlines the object with a red border.
- Release the mouse button.
- You should now see the test object listed in the Data Driven Commands section of the Insert Data Driven Actions dialog shown in Figure 4.
Figure 4. Insert Data Driven Actions dialog with test object
- Click OK to close the Insert Data Driven Actions dialog.
- Click Cancel on the Member Login window.
- Exit the ClassicsCD application.
- Stop recording.
If you look at the code for the script you just recorded, you should see a line of code similar to that in Listing 1.
Listing 1. Recorded datapool call in Rational Functional Tester
// Data Driven Code inserted on Sep 14, 2005
nameCombo().select(dpString("nameCombo"));
|
In this code you select the combo box value that matches the value in the datapool column named nameCombo. If you open the Private Test Datapool in the Script Explorer, you should see the data you recorded listed in the first row, as shown in Figure 5.
Figure 5. Recorded data in Private Test Datapool
In a datapool in Rational Functional Tester, datapool columns are referred to as variables, and rows are referred to as records (don’t ask me why). You simply right-click in the datapool grid (shown in Figure 5) to bring up a list of datapool editing commands, as shown in Figure 6.
Figure 6. Editing a datapool in Rational Functional Tester
All these commands do exactly what you would expect them to do; no surprises. The only thing worth nothing is that when you declare a new Variable (or column) you will need to specify the Type for the variable. You will want to type the full path to the class when declaring type. For example, use java.lang.String instead of simply entering String.
Rational Functional Tester provides the ability to find or replace literal values in a test script with a datapool reference (a datapool reference is another way of saying it has an associated datapool). In the toolbar or the Script menu you can access the Datapool Literal Substitution dialog shown in Figure 7.
Figure 7. Datapool Literal Substitution dialog
You can set the options in the Datapool Literal Substitution dialog box to find and replace all, number, string, or boolean literals in a script with a datapool reference. You can also add a literal from a script to a datapool. If you do not use an existing datapool variable, Functional Test uses the same literal values (the values that Functional Test captured when you recorded the test script) each time you run the script.
Datapools as verification points
You can use a datapool reference instead of a literal value for the value you are testing in the verification point. While recording, if you insert a verification point using the Verification Point and Action wizard, you can convert the verification point value to a datapool reference by clicking the Convert Value to Datapool Reference button in the toolbar of the wizard. This button is circled in red in Figure 8.
Figure 8. Converting verification point values to datapool references
You can also add data to the datapool after you finish recording the test script.
If you already have datapools in IBM® Rational® TestManager, that’s OK. You can import them into Rational Functional Tester, along with any plain old .csv file you created using a spreadsheet application. The reverse is true as well. Once you get all your data into Rational Functional Tester, you can export datapools to a .csv file to edit in a spreadsheet application. Instructions for how to do this are in the Rational Functional Tester Help file.
A simple example of data-driven testing with Rational Functional Tester
Alright, so you’ve covered the basics of using datapools, now you’ll look at a more detailed example of data-driven testing. This example will use a test for http://www.google.com/. The Google Web Search has some built in features that you would like to test. These features include: calculator, currency conversion, movies, search by number, and travel information (to name a few of the features contained in that powerful little text field). The recorded script shown in Listing 2 starts the browser to Google.com, and then performs a series of searches and verification points to validate each of the above features.
Listing 2. Record and playback Google Web Search script
package tests;
import resources.tests.google_search_recordHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class google_search_record extends google_search_recordHelper
{
public void testMain(Object[] args)
{
startApp("www.Google.com");
// Calculator
text_q2().click(atPoint(191,10));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputKeys("5{+}5");
button_googleSearchsubmit2().click();
calculator_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome2().click();
// Currency Conversion
text_q2().click(atPoint(192,13));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputKeys("3.5 USD in GBP");
button_googleSearchsubmit2().click();
currency_conversion_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome3().click();
// Movies
text_q2().click(atPoint(98,14));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars(
"charlie and the chocolate factory");
button_googleSearchsubmit2().click();
movies_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome6().click();
// Search by Number
text_q2().click(atPoint(177,13));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars("patent 5123123");
button_googleSearchsubmit2().click();
search_by_number_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome8().click();
// Travel Information
text_q2().click(atPoint(163,10));
browser_htmlBrowser(document_google2(),
DEFAULT_FLAGS).inputChars("United 134");
button_googleSearchsubmit2().click();
travel_info_gridVP().performTest(2.0, 20.0);
image_goToGoogleHome10().click();
// Close the browser
browser_htmlBrowser(document_google2(),MAY_EXIT).close();
}
}
|
In the script shown in Figure 2, each feature tested records a click on the search text field, enters the search criteria, clicks the search button, verifies the results, and clicks the Google image in the top left of the results page to reset the browser for the next feature test. There is a lot of repeated code here. Listing 3 shows the same test script, only now it’s data-driven.
Listing 3. Data-driven Google Web Search script
package tests;
import resources.tests.google_datadrivenHelper;
import com.rational.test.ft.*;
import com.rational.test.ft.object.interfaces.*;
import com.rational.test.ft.script.*;
import com.rational.test.ft.value.*;
import com.rational.test.ft.vp.*;
public class google_datadriven extends google_datadrivenHelper
{
public void testMain(Object[] args)
{
startApp("www.Google.com");
// Run test for each value in datapool
for(int i = 0; i<5; i++)
{
// Search
text_q().click(atPoint(71,11));
browser_htmlBrowser(document_google(),
DEFAULT_FLAGS).inputChars(
dpString("searchCriteria"));
button_googleSearchsubmit().click();
// Results
searchResults_textVP().performTest(2.0, 20.0);
image_goToGoogleHome().click();
// Increment the datapool
dpNext();
}
// Exit
browser_htmlBrowser(document_google2(),MAY_EXIT).close();
}
}
|
By using a for loop (you could also use some other type of loop) and moving incrementally through the data in your datapool, you can keep from repeating the same code multiple times. All you did was add a datapool call in the inputChars for the search field and add a datapool reference to your verification point. Any time you want to add another test, just add another row to your datapool and increment the counter value in the loop. The datapool shown in Figure 9 shows both the search criteria and the search results for each test.
Figure 9. Datapool values for the data-driven script
When you compare the Google recorded test to the Google data-driven test the biggest difference you’ll see is the visibility you get into what is actually being tested. In the record and playback script, you have to scroll through the code to find the search criteria. In the data-driven test all you have to do is look at the searchCriteria variable (column) in the datapool. In addition, in the record and playback script you need to find and open each verification point individually to find the search results. In the data-driven test all you have to do is look at the searchResults variable in the datapool. Forget how much less code there is, it’s easier to tell what the data-driven test is testing.
Data selection is the first step down the road of data-driven testing. You'll need to select the data that either drives the navigation of your application, represents the data that gets entered into your application, or both. This section will quickly examine five different ways to select data for testing: based on risk, based on requirements, based on availability, using production data, or using randomly generated data.
Selecting data based on risk. The number one criterion for test data selection should be risk. When you identify risks, you consider what can go wrong. You're looking for the events that might occur that would decrease the likelihood that you'll be able to deliver the project with the right features and the requisite level of quality on time and within budget. There are three ways to categorize risks:
- By the impact of the risk -- the deviations of schedule, effort, or cost from plan if the risk materializes
- By likelihood of occurrence -- the probability that the risk will materialize (usually expressed as a percentage)
- By risk exposure -- the impact multiplied by the likelihood of occurrence
Selecting data based on requirements. You can also select data that will allow you to test a requirement or a set of requirements explicitly. Look for data that will allow you to exercise feature sets, capabilities, and security features in your application:
- If your application has different roles, what data would you need to exercise each role?
- What features do you want to include in your test coverage and what data do you require to use them?
In addition, consider what impact the various target deployment environments will have on the elements to be tested. Your list of selected data should include data for both the application under test and the target environment(s). What data will you need to test:
- Hardware devices
- Device drivers
- Operating systems
- Network and communications software
- Third-party base software components (for example, e-mail software, Internet browsers)
- Various configurations and settings related to the possible combinations of all these elements
- Internationalization
Selecting data based on availability. You may want to select data that's readily available. This could include:
- Production data (discussed in depth in the following section) that's in an easy-to-access format
- Data from past iterations
- Spreadsheets used by manual testers for your project
- Data from other projects or teams in your company
- Data from some data generation source (discussed in the final section).
The idea here is that if the data is easily accessible, as well as usable and meaningful, including this data in your testing can save time and money. I emphasize usability and meaningfulness because it's important that you don't select data just because it's there and ready to be used -- it may be bad data.
Using production data. Another strategy to gather test data is to use production data. Although you shouldn't rely solely on this type of data, it can be one of the richest sources of scenarios for automated testing, both because the data is representative of real scenarios the application will face, and because it will most likely provide a high number of different scenarios. On a recent project, we read production data from a holding table about once a week, and we were rewarded with 300 to 500 scenarios with little to no effort. You can load the data straight into the test environment, read it into data files for processing later, or read it in real time and convert it as you use it. Production data is also an excellent source for parallel testing. If you use production data in the system you're developing, you'll quickly know if that system works like the system in production. This technique can especially help in finding problems with floating-point values, conversion ratios, and lengths associated with data types.
There are some caveats about using production data, however. Production data will most likely not contain many of the special cases you'll want to test for, and it's not a replacement for well-thought-out test scenarios. There are also potentially some legal issues surrounding the use of production data. Especially if you outsource some of your testing, you'll want to be sure to check your company's policies on the use of production data; if no formal policy exists, consult someone in your legal department. Even if you can't directly use production data, odds are you'll be able to change some values (names, social security numbers, and such) and use the rest of the data.
Using random data generation. Many tools include test data generators. Random data generators can be especially helpful in generating large sets of customer data (depending on what you are testing and how much time you want to invest in the generation). For example, if you need to test for rounding errors on financial information, you can:
- Randomly generate a large set of data
- Run the data through your application
- Run it through the Windows calculator (assuming the Windows calculator has no rounding errors) simulating the same calculations
- Compare the results to make sure they match
If you need, for example, to generate a set of 500 different customer names and addresses, you can have a generator (TestManager, Excel, Google Sets, and so on) create a table of names and addresses.
You should avoid trying to select data for all of your test cases at once. Instead, take an incremental and iterative approach to working with test data, focusing your efforts on the data that you think is most likely to produce useful evaluation information for the given test cycle. This helps to mitigate the risk of devoting too much time to a single set of data or type of testing -- to the neglect of other data or testing tasks -- and minimizes the risk of expending effort on data for test ideas that may later prove of little interest.
Advantages and disadvantages of a data-driven framework
Of the three types of frameworks this series covers (modularity, data-driven, and keyword-driven), a data-driven framework is the simplest for non-technical testers to implement, as almost all of the functionality is built in to Rational Functional Tester and is supported while performing record and playback. Not only is it simple for non-technical testers (which is not reason enough for me to recommend it), but it’s powerful. It leads to semi-self-documenting test cases, reduces the amount of code required to create large numbers of test cases, and it allows you to do types of testing otherwise too time-consuming or cumbersome to do manually.
The main advantage of a data-driven framework is that it reduces the cost of creating test cases for many types of applications. In financial services or insurance applications, for example, data-driven frameworks are worth their weight in gold. Many times test cases for financial service and insurance applications are simply subtle variations where you look at how one variable in a thousand affects the end dollar amount of a complex calculation. You would be hard pressed to find a better market for a data-driven tool. On the other hand, I once worked on a project that involved software that did chemical graphing and chemical compound analysis. For this application, test cases were so specific and focused (and GUI dependent) that a data-driven framework didn’t make sense.
One of the most prominent drawbacks to a data-driven framework is that it makes creating new test cases so easy. This can sometimes lead to test case bloat, where you are running and maintaining tests simply because they were easy to create. That’s not necessarily a bad thing -- that’s one of the reasons why you want to use a tool like this. Just be careful that you aren’t investing a lot of time and money maintaining a suite of tests just because they were easy to create. It may be worth it to create an inexpensive test and run it once just to see what happens, but once you start maintaining that test over multiple releases of your application, the cost of that test starts to rise. You want to make sure it is powerful enough to justify the cost of maintenance.
Think about some of the ways you can use data-driven testing in your test automation. Look for places that you find yourself calling the same code over and over -- odds are you will find opportunities to use a datapool in these places. Don’t feel like if you want to do data-driven testing that you need to convert all of your scripts at once. You don’t. Simply create datapools as needed, and over time you will grow your test automation into a framework where you use datapools as a natural part of script generation. In many applications there can be a high level of reuse with existing datapools. Just as DBAs spend time organizing their data, you can spend some time organizing yours.
Learn
- Using cost-benefit analysis to compare different test structures for Rational Robot: by Michael Kelly
- Choosing a test automation framework : by Michael Kelly
- Using IBM Rational Functional Tester to test financial services applications: by Michael Kelly
- Totally Data-Driven Automated Testing: by Keith Zambelich
- Improving the Maintainability of Automated Test Suites: by Cem Kaner
- To learn more, visit the Functional Tester product resource area on developerWorks Rational. You'll find technical documentation, how-to articles, education, downloads, product information, and more.
Get products and technologies
- Try out Rational products by visiting the trial downloads area.
Discuss
Michael Kelly is currently an independent consultant and provides custom training in the IBM Rational testing tools. He consults, writes, and speaks on topics in software testing. He is currently serving as the Program Director for the Indianapolis Quality Assurance Association and is a Director at Large for the Association for Software Testing. He can be reached by email at Mike@MichaelDKelly.com.
