Learn different testing techniques and features of Visual Studio 2012 with detailed explanations and real-time samples
Satheesh Kumar N Subashni S
Contents
Preface 1
Chapter 1: Visual Studio 2012 Test Types 7
Software testing in Visual Studio 2012 8
Testing as part of software development life cycle 9
Types of testing 11
Unit testing 12
Manual testing 14
Exploratory testing 15
Web performance tests 16
Coded UI test 17
Load testing 18
Ordered test 20
Generic test 21
Test management in Visual Studio 2012 21
Introduction to testing tools 22
Test Explorer 25
Code coverage results 28
Microsoft Test Manager 28
Connecting to Team Project 29
Test Plans, Suites, and test cases 30
Defining test cases 31
Lab Center 32
Summary 33
Chapter 2: Test Plan, Test Suite, and Manual Testing 35
Test Plan 36
Test Suite and its types 41
Static Test Suites 42
Query-based Test Suites 44
Requirement-based Test Suites 45
Running manual tests 47
Action recording 56
Shared steps and action recording for shared steps 59
Creating shared steps 59
Action recording for shared steps 62
Adding parameters to manual tests 62
Summary 66
Chapter 3: Automated Tests 67
Coded UI tests from action recordings 68
Files generated for coded UI test 73
CodedUITest1.cs 73
UIMap.Designer.cs 74
UIMap.cs 75
UiMap.uitest 76
Data-driven coded UI test 80
Adding controls and validation to coded UI test 82
Summary 88
Chapter 4: Unit Testing 89
Creating unit tests 90
Assert statements 93
Types of Assert statements 94
Assert 94
StringAsserts 107
CollectionAssert 111
AssertFailedException 119
UnitTestAssertionException 120
ExpectedExceptionAttribute 120
Unit Tests and Generics 123
Data-driven unit testing 126
Unit Testing using Fakes 132
Stubs 132
Shims 137
Difference between Stubs and Shims 137
Code coverage unit test 138
Blocks and lines 140
Excluding elements 141
Summary 142
Chapter 5: Web Performance Test 143
Creating the web performance test 145
Recording a test 146
Adding comments 152
Cleaning the recorded tests 153
Copying the requests 153
Adding loops 153
Web performance test editor 158
Web test properties 160
Web test request properties 161
Other request properties 164
Form POST parameters 164
QueryString parameters 165
Extraction rules 166
Validation rules 171
Transactions 174
Conditional rules 176
Toolbar properties 181
Add data source 181
Setting credentials 184
Add recording 185
Parameterize web server 186
Adding a web test plugin 189
Debugging and running the web test 191
Settings in the .testsettings file 192
General 192
Roles 194
Data and Diagnostics 195
Deployment 197
Setup and Cleanup Scripts 198
Hosts 199
Test Timeouts 199
Unit test 200
Web test 201
Running the test 203
Web Browser 204
Request 204
Response 205
Context 205
Details 206
Summary 207
Chapter 6: Advanced Web Testing 209
Dynamic parameters in web testing 210
Coded web test 212
Generating code from a recorded test 213
Transactions in coded tests 218
Custom code 219
Adding a comment 219
Running the coded web test 220
Debugging coded web test 222
Custom rules 224
Extraction rules 224
Validation rules 228
Summary 232
Chapter 7: Load Testing 233
Creating a Load Test 234
Load Test Wizard 236
Specifying a scenario 239
Counter sets 248
Run settings 250
Editing Load Tests 262
Adding context parameters 267
Storing results in the central result store 268
Running the Load Test 270
Analyzing and exporting Test Results 272
Graphical view 272
Summary view 275
Tables view 277
Detail view 279
Exporting to Microsoft Excel 280
Using Test Controller and Test Agents 288
Test Controller and Test Agent configuration 289
Summary 296
Chapter 8: Ordered and Generic Tests 297
Ordered tests 298
Creating an ordered test 298
Executing an ordered test 300
Generic tests 301
Creating a generic test 302
The summary results file 304
Summary 308
Chapter 9: Managing and Configuring Tests 309
Using Test settings 310
The General option 311
The Roles option 312
Data and Diagnostics 313
The Deployment section 316
Setup and Cleanup Scripts 317
The Hosts option 318
The Test Timeouts option 319
The Unit Test option 320
Editing the Test Run configuration file 322
The Web Test option 324
Configuring unit tests using the .runsettings file 325
Summary 326
Chapter 10: The Command Line 327
VSTest.Console utility 327
Running tests using VSTest.Console 328
The /Tests option 329
The /ListTests option 329
MSTest utility 330
Running a test from the command line 332
The /testcontainer option 332
The /testmetadata option 333
The /test option 334
The /unique option 335
The /noisolation option 336
The /testsettings option 336
The /resultsfile option 337
The /noresults option 337
The /nologo option 338
The /detail option 338
Publishing Test Results 339
The /publish option 339
The /publishbuild option 339
The /flavour option 340
The /platform option 340
The /publishresultsfile option 341
TCM command line utility 344
Importing tests to a Test Plan 345
Running tests in a Test Plan 349
Summary 352
Chapter 11: Working with Test Results 353
Test Runs and Test Results 354
Test as part of the Team Foundation Server build 358
Building reports and Test Results 363
Creating a work item from the result 365
Summary 367
Chapter 12: Exploratory Testing and Reporting 369
Exploratory testing 371
Reports using Team Foundation Server 379
Bug status report 379
Test case readiness report 379
Status on all iterations 380
Other out-of-the-box reports 380
Creating a report definition using Visual Studio 2012 382
Summary 390
Chapter 13: Test and Lab Center 391
Connecting to Team Project 392
Testing Center 394
Testing Center – Plan tab 395
Testing Center – Test tab 399
Testing Center – Track tab 402
Testing Center – Organize tab 405
Lab Center 408
Environments 408
Deployed environments 410
Summary 413
Index 415
Preface
The Microsoft Visual Studio 2012 suite contains several features to support the needs of developers, testers, architects, and managers to simplify the development process. Visual Studio 2012 provides different editions of the product such as Professional, Premium, and Ultimate with different set of tools and features. Visual Studio 2012 is tightly integrated with Team Foundation Server, a central repository and configuration management system that provides version control, process guidance and templates, automated builds, automated tests, bug tracking, work item tracking, reporting, and support of the Lab Center and Test Center configurations. The Microsoft Test Manager 2012 is a standalone tool used to organize Test Plans, Manage test cases, and executing manual test cases.
Software Testing using Visual Studio 2012 helps software developers to get familiarized with the Visual Studio tools and techniques to create automated unit tests, and to use automated user interface testing, code analysis and profiling to find out more about the performance and quality of the code. Testers benefit from learning more about the usage of Testing tools, test case management techniques, working with Test Results, and using Test Center and Lab center. This book also covers different types of testing such as web performance test, load test, executing the manual test cases, recording user actions, re-running tests using recording, test case execution, capturing defects, and linking defects with requirements. Testers also get a high level overview on using Lab Center for creating virtual environments for testing multiple users and multiple location scenarios.
Visual Studio 2012 provides user interface tools such as Test Explorer, Test Results, and Test Configuration to create, execute, and maintain the tests and Test Results in integration with Team Foundation Server. This book provides detailed information on all of the tools used for testing the application during the development and testing phases of the project life cycle.
What this book covers
Chapter 1, Visual Studio 2012 Test Types, provides an overview of different types of testing which helps testing the software applications through different phases of software development. This chapter also introduces the tools and techniques in Visual Studio 2012 for different testing types, Microsoft Test Manager 2012, and its features.
Chapter 2, Test Plan, Test Suite, and Manual Testing, explains the steps involved in creating and managing the Test Plan, Test cases and Test Suite using Test Center in Test Manager. This chapter also explains how to create manual tests by recording the user actions and running the test with data inputs. Sharing the test recording across multiple tests is also covered in this chapter.
Chapter 3, Automated Tests, provides a step-by-step approach to creating Coded UI test from user action recordings. It also explains the steps to execute the coded UI test through data source and adding validation and custom rules to the test.
Chapter 4, Unit Testing, explains the detailed steps involved in creating unit test classes and methods for the code. Different type of assert methods and parameters for testing the code, passing set of data from a data source and testing the code also explained in detail. The mocking framework used for isolating the code and testing it with the help of Shims and Stubs is also explained in detail.
Chapter 5, Web Performance Test, explains the basic way of web testing by recording the user actions and creating a test out of it. Running the test using a data source, adding parameters to the web tests, adding validation and extraction rules, adding looping and branching mechanism to the recorded tests, and here configuring the settings required for the Test Runs are some of the features explained as part of this chapter.
Chapter 6, Advanced Web Testing, explains the way of generating code out of the recorded web tests explained in Chapter 5, Web Performance Test using the Generate Code option. This is very much useful for customizing the test through the code, adding additional logic to the test, adding custom validation and extraction rules.
Chapter 7, Load Testing, helps in simulating various numbers of users, network bandwidths, combination of different web browsers, and different configurations. In the case of web applications it is always necessary to test the stability and performance of the application under huge data load and concurrent users. This chapter explains the steps involved in simulating the real world scenario by using Controllers and Agents. The details of analyzing and exporting the load Test Results are also explained in this chapter.
Chapter 8, Ordered and Generic Tests, explains the way of testing the existing third party tool or service which can also be run using the command line. Visual Studio 2012 provides a feature called ordered test to group all or some of these tests and then execute the tests in the same order. The main advantage of creating the ordered test is to execute multiple tests in an order based on the dependencies. Generic tests are just like any other tests except that it is used for testing an existing third party tool or service.
Chapter 9, Managing and Configuring Tests, explains the details of the test settings file and the tools used for managing tests. The configuration includes deployment details, setup and cleaning scripts, collecting data diagnostics information, unit test and web test settings.
Chapter 10, The Command Line, explains the command line tools such as VSTest. Console, MSTest, and TCM used for running the test with different options, then collecting the output and publishing the results. Each of these commands is used for specific purposes including backwards compatibility.
Chapter 11, Working with Test Results, explains the process of running the tests and publishing the Test Results to the Team Project. Also covered in detail is to integrate the tests as part of Team Foundation Server builds, Build reports and Test Results, Creating work items from Test Results, and publishing the Test Results.
Chapter 12, Exploratory Testing and Reporting, explains the details of testing which happens without any test cases and scripts and by only exploring the application manually. This chapter also explains the details of accessing the Test Results and publishing Test Results and reporting the same in a specific format. Accessing different types of testing reports and creating new test reports are also explained in this chapter.
Chapter 13, Test and Lab Center, is useful for creating and organizing Test Plans and test cases. Test plans can be associated to the requirements using Test Center. The Lab Center helps in creating and configuring different virtual/physical environments for the Test Runs, Test Settings such as defining the roles and configuring the data and diagnostics information for the selected roles, configuring the Test Controllers required for the test, and configuring the test library to store the environment information.
What you need for this book
This book requires a basic knowledge on any of the versions of Visual Studio and Team Foundation Server. The reader must be familiar with the Visual Studio IDE and have basic knowledge of C#. The following tools are required in order to use the code samples of the chapters in this book:
• Visual Studio 2012 Ultimate
• SQL Server Express (OR) SQL Server 2008 or higher version
• Team Foundation Server 2010/2012
• SQL Server Reporting services
Who this book is for
If you are a software developer, a tester or an architect who wishes to master the amazing range of features offered by Visual Studio 2012 for testing your software applications – then this book is for you.
This book assumes that you have a basic knowledge of testing software applications and have good work experience of using Visual Studio IDE.
Visual Studio 2012 Test Types
Software testing is one of the most important phases of the software development life cycle (SDLC). Delivery of the software product is based on following good SDLC practices of analysis, design, coding, testing, and by all means meeting the customer requirements. The quality of the product is measured by verifying and validating the product based on the defined functional and non-functional requirements for product. The testing tools and techniques play an important role in simulating the real-life scenarios and the user load required for verifying the stability and reliability of the product. For example, testing a web application with 1,000 concurrent users is a very time consuming and tedious task, if we do it manually considering the required resources. But the testing tools that are part of Visual Studio 2012 can simulate such scenarios and test it with limited resources and without manual intervention during testing. Visual Studio 2012 provides tools to conduct different types of testing, such as Unit testing, Load testing, Web testing, Ordered testing, Generic testing, and Exploratory testing.
This chapter covers the following topics and provides a high-level overview of the testing tools and techniques supported by Visual Studio 2012:
• Testing as part of the software development life cycle
• Types of testing
• Test management in Visual Studio 2012
• Testing tools in Visual Studio 2012
