Friday, December 28, 2007

v model to w model

V-Model:

The V-model promotes the idea that the dynamic test stages (on the right hand side of the model) use the documentation identified on the left hand side as baselines for testing. The V-Model further promotes the notion of early test preparation.

The V-Model of testing

Early test preparation finds faults in baselines and is an effective way of detecting faults early. This approach is fine in principle and the early test preparation approach is always effective. However, there are two problems with the V-Model as normally presented.


The V-Model with early test preparation

There is rarely a perfect, one-to-one relationship between the documents on the left hand side and the test activities on the right. For example, functional specifications don’t usually provide enough information for a system test. System tests must often take account of some aspects of the business requirements as well as physical design issues for example. System testing usually draws on several sources of requirements information to be thoroughly planned.

V-Model has little to say about static testing at all. The V-Model treats testing as a back-door activity on the right hand side of the model. There is no mention of the potentially greater value and effectiveness of static tests such as reviews, inspections, static code analysis and so on. This is a major omission and the V-Model does not support the broader view of testing as a constantly prominent activity throughout the development lifecycle.

W Model:


The W-Model of testing

Paul Herzlich introduced the W-Model approach in 1993. The W-Model attempts to address shortcomings in the V-Model. Rather than focus on specific dynamic test stages, as the V-Model does, the W-Model focuses on the development products themselves. Essentially, every development activity that produces a work product is shadowed by a test activity. The purpose of the test activity specifically is to determine whether the objectives of a development activity have been met and the deliverable meets its requirements. In its most generic form, the W-Model presents a standard development lifecycle with every development stage mirrored by a test activity. On the left hand side, typically, the deliverables of a development activity (for example, write requirements) is accompanied by a test activity test the requirements and so on. If your organization has a different set of development stages, then the W-Model is easily adjusted to your situation. The important thing is this: the W-Model of testing focuses specifically on the product risks of concern at the point where testing can be most effective.


The W-Model and static test techniques.

If we focus on the static test techniques, you can see that there is a wide range of techniques available for evaluating the products of the left hand side. Inspections, reviews, walkthroughs, static analysis, requirements animation as well as early test case preparation can all be used.



The W-Model and dynamic test techniques.

If we consider the dynamic test techniques you can see that there is also a wide range of techniques available for evaluating executable software and systems. The traditional unit, integration, system and acceptance tests can make use of the functional test design and measurement techniques as well as the non-functional test techniques that are all available for use to address specific test objectives.

The W-Model removes the rather artificial constraint of having the same number of dynamic test stages as development stages. If there are five development stages concerned with the definition, design and construction of code in your project, it might be sensible to have only three stages of dynamic testing only. Component, system and acceptance testing might fit your normal way of working. The test objectives for the whole project would be distributed across three stages, not five. There may be practical reasons for doing this and the decision is based on an evaluation of product risks and how best to address them. The W-Model does not enforce a project symmetry that does not (or cannot) exist in reality. The W-model does not impose any rule that later dynamic tests must be based on documents created in specific stages (although earlier documentation products are nearly always used as baselines for dynamic testing. In projects using these methods, requirements and designs might be documented in multiple models so system testing might be based on several of these models (spread over several documents).

We use the W-Model in test strategy as follows. Having identified the specific risks of concern, we specify the products that need to be tested; we then select test techniques (static reviews or dynamic test stages) to be used on those products to address the risks; we then schedule test activities as close as practicable to the development activity that generated the products to be tested.

v model to w model

V-Model:

The V-model promotes the idea that the dynamic test stages (on the right hand side of the model) use the documentation identified on the left hand side as baselines for testing. The V-Model further promotes the notion of early test preparation.

The V-Model of testing

Early test preparation finds faults in baselines and is an effective way of detecting faults early. This approach is fine in principle and the early test preparation approach is always effective. However, there are two problems with the V-Model as normally presented.


The V-Model with early test preparation

There is rarely a perfect, one-to-one relationship between the documents on the left hand side and the test activities on the right. For example, functional specifications don’t usually provide enough information for a system test. System tests must often take account of some aspects of the business requirements as well as physical design issues for example. System testing usually draws on several sources of requirements information to be thoroughly planned.

V-Model has little to say about static testing at all. The V-Model treats testing as a back-door activity on the right hand side of the model. There is no mention of the potentially greater value and effectiveness of static tests such as reviews, inspections, static code analysis and so on. This is a major omission and the V-Model does not support the broader view of testing as a constantly prominent activity throughout the development lifecycle.

W Model:


The W-Model of testing

Paul Herzlich introduced the W-Model approach in 1993. The W-Model attempts to address shortcomings in the V-Model. Rather than focus on specific dynamic test stages, as the V-Model does, the W-Model focuses on the development products themselves. Essentially, every development activity that produces a work product is shadowed by a test activity. The purpose of the test activity specifically is to determine whether the objectives of a development activity have been met and the deliverable meets its requirements. In its most generic form, the W-Model presents a standard development lifecycle with every development stage mirrored by a test activity. On the left hand side, typically, the deliverables of a development activity (for example, write requirements) is accompanied by a test activity test the requirements and so on. If your organization has a different set of development stages, then the W-Model is easily adjusted to your situation. The important thing is this: the W-Model of testing focuses specifically on the product risks of concern at the point where testing can be most effective.


The W-Model and static test techniques.

If we focus on the static test techniques, you can see that there is a wide range of techniques available for evaluating the products of the left hand side. Inspections, reviews, walkthroughs, static analysis, requirements animation as well as early test case preparation can all be used.



The W-Model and dynamic test techniques.

If we consider the dynamic test techniques you can see that there is also a wide range of techniques available for evaluating executable software and systems. The traditional unit, integration, system and acceptance tests can make use of the functional test design and measurement techniques as well as the non-functional test techniques that are all available for use to address specific test objectives.

The W-Model removes the rather artificial constraint of having the same number of dynamic test stages as development stages. If there are five development stages concerned with the definition, design and construction of code in your project, it might be sensible to have only three stages of dynamic testing only. Component, system and acceptance testing might fit your normal way of working. The test objectives for the whole project would be distributed across three stages, not five. There may be practical reasons for doing this and the decision is based on an evaluation of product risks and how best to address them. The W-Model does not enforce a project symmetry that does not (or cannot) exist in reality. The W-model does not impose any rule that later dynamic tests must be based on documents created in specific stages (although earlier documentation products are nearly always used as baselines for dynamic testing. In projects using these methods, requirements and designs might be documented in multiple models so system testing might be based on several of these models (spread over several documents).

We use the W-Model in test strategy as follows. Having identified the specific risks of concern, we specify the products that need to be tested; we then select test techniques (static reviews or dynamic test stages) to be used on those products to address the risks; we then schedule test activities as close as practicable to the development activity that generated the products to be tested.

Equivalence partitioning

Equivalence partitioning:


Equivalence partitioning is a method for deriving test cases. In this method, classes of input conditions called equivalence classes are identified such that each member of the class causes the same kind of processing and output to occur.

In this method, the tester identifies various equivalence classes for partitioning. A class is a set of input conditions that are is likely to be handled the same way by the system. If the system were to handle one case in the class erroneously, it would handle all cases erroneously.

Equivalence partitioning drastically cuts down the number of test cases required to test a system reasonably. It is an attempt to get a good 'hit rate', to find the most errors with the smallest number of test cases.

To use equivalence partitioning, you will need to perform four steps:

  • Determining conditions to be Tested
  • Defining Tests
  • Designing test cases
  • Identifying Final set of Test Cases

Defining Tests

A number of items must be considered when determining the tests using the equivalence partitioning method, like:
  • All valid input data for a given condition are likely to go through the same process.
  • Invalid data can go through various processes and need to be evaluated more carefully. For example,
  • a blank entry may be treated differently than an incorrect entry,
  • a value that is less than a range of values may be treated differently than a value that is greater,
  • if there is more than one error condition within a particular function, one error may override the other, which means the subordinate error does not get tested unless the other value is valid.
Defining Test Cases

Create test cases that incorporate each of the tests. For valid input, include as many tests as possible in one test case. For invalid input, include only one test in a test case in order to isolate the error. Only the invalid input test condition needs to be evaluated in such tests, because the valid condition has already been tested.

EXAMPLE OF EQUIVALENCE PARTITIONING

1. Conditions to be Tested

The following input conditions will be tested:
  • For the first three digits of all social insurance (security) numbers, the minimum number is 111 and the maximum number is 222.
  • For the fourth and fifth digits of all social insurance (security) numbers, the minimum number is 11 and the maximum number is 99.
2. Defining Tests

Identify the input conditions and uniquely identify each test, keeping in mind the items to consider when defining tests for valid and invalid data.

The tests for these conditions are:
  • The first three digits of the social insurance (security) number are:
  1. = or > 111 and = or <>
  2. <>
  3. > 222, (invalid input, above the range),
  4. blank, (invalid input, below the range, but may be treated differently).
  • The fourth and fifth digits of the social insurance (security) number are:
  1. = or > 11 and = or <>
  2. <>
  3. > 99, (invalid input, above the range),
  4. blank, (invalid input, below the range, but may be treated differently).
Using equivalence partitioning, only one value that represents each of the eight equivalence classes needs to be tested.

3. Defining Test Cases

After identifying the tests, create test cases to test each equivalence class, (i.e., tests 1 through 8).

Create one test case for the valid input conditions, (i.e., tests 1 and 5), because the two conditions will not affect each other.

Identify separate test cases for each invalid input, (i.e., tests 2 through 4 and tests 6 through 8).

Both conditions specified, (i.e., condition 1 - first three digits, condition 2 - fourth and fifth digits), apply to the social insurance (security) number.

Since equivalence partitioning is a type of black-box testing, the tester does not look at the code and, therefore, the manner in which the programmer has coded the error handling for the social insurance (security) number is not known. Separate tests are used for each invalid input, to avoid masking the result in the event one error takes priority over another.

For example, if only one error message is displayed at one time, and the error message for the first three digits takes priority, then testing invalid inputs for the first three digits and the fourth and fifth digits together, does not result in an error message for the fourth and fifth digits. In tests B through G, only the results for the invalid input need to be evaluated, because the valid input was tested in test case A.

4. Suggested test cases:
  1. Test Case A - Tests 1 and 5, (both are valid, therefore there is no problem with errors),
  2. Test Case B - Tests 2 and 5, (only the first one is invalid, therefore the correct error should be produced),
  3. Test Case C - Tests 3 and 5, (only the first one is invalid, therefore the correct error should be produced),
  4. Test Case D - Tests 4 and 5, (only the first one is invalid, therefore the correct error should be produced),
  5. Test Case E - Tests 1 and 6, (only the second one is invalid, therefore the correct error should be produced),
  6. Test Case F - Tests 1 and 7, (only the second one is invalid, therefore the correct error should be produced),
  7. Test Case G - Tests 1 and 8, (only the second one is invalid, therefore the correct error should be produced).

Cyclomatic complexity

Cyclomatic complexity is a software metric (measurement). It was developed by Thomas McCabe and is used to measure the complexity of a program. It directly measures the number of linearly independent paths through a program's source code. It is computed using a graph that describes the control flow of the program. The nodes of the graph correspond to the commands of a program. A directed edge connects two nodes if the second command might be executed immediately after the first command.


Definition


M = E − N + 2P

where

M = cyclomatic complexity
E = the number of edges of the graph
N = the number of nodes of the graph
P = the number of connected components.

"M" is alternatively defined to be one larger than the number of decision points (if/case-statements, while-statements, etc) in a module (function, procedure, chart node, etc.), or more generally a system.

Separate subroutines are treated as being independent, disconnected components of the program's control flow graph.


Alternative definition


v(G) = e − n + p
G is a program's flowgraph
e is the number of edges (arcs) in the flowgraph
n is the number of nodes in the flowgraph
p is the number of connected components


Alternative way


There is another simple way to determine the cyclomatic number. This is done by counting the number of closed loops in the flow graph, and incrementing that number by one.

i.e.

M = Number of closed loops + 1

where

M = Cyclomatic number.


Implications for Software Testing:

  • M is a lower bound for the number of possible paths through the control flow graph.
  • M is an upper bound for the number of test cases that are necessary to achieve a complete branch coverage.

For example, consider a program that consists of two sequential if-then-else statements.

if (c1) {
f1();
} else {
f2();
}

if (c2) {
f3();
} else {
f4();
}

  • To achieve a complete branch coverage, two test cases are sufficient here.
  • For a complete path coverage, four test cases are necessary.
  • The cyclomatic number M is three, falling in the range between these two values, as it does for any program.

Cyclomatic complexity can be applied in several areas
, including:
  • Code development risk analysis: While code is under development, it can be measured for complexity to assess inherent risk or risk buildup.
  • Change risk analysis in maintenance: Code complexity tends to increase as it is maintained over time. By measuring the complexity before and after a proposed change, this buildup can be monitored and used to help decide how to minimize the risk of the change.
  • Test Planning: Mathematical analysis has shown that cyclomatic complexity gives the exact number of tests needed to test every decision point in a program for each outcome. Thus, the analysis can be used for test planning. An excessively complex module will require a prohibitive number of test steps; that number can be reduced to a practical size by breaking the module into smaller, less-complex sub-modules.
  • Re-engineering: Cyclomatic complexity analysis provides knowledge of the structure of the operational code of a system. The risk involved in reengineering a piece of code is related to its complexity. Therefore, cost and risk analysis can benefit from proper application of such an analysis.

test coverage matrix vs traceblity matrix

Test coverage matrix:

Test coverage matrix is a checklist which ensures that the functionality of the given screen(unit) is checked in all possible combinations (positive and negative) which have not been covered in test cases. Test coverage matrix is usually prepared for a screen having large number of controls (textboxes, dropdowns, buttons etc) usually, test coverage matrix is prepared in a spread sheet having all the controls (textboxes, dropdowns, buttons etc) in the columns and then all possible entries in those fields in the rows with an ''yes'' or ''no'' in the rows against the controls listed in the columns. For example, consider a ''login'' screen wherein we have ''username'' and ''password" textfields.

While preparing test coverage matrix, the first column will be ''s.no'' and the second will be ''username" and ''password" will be the third field followed by ''ok'' and ''cancel'' button. Then, in the first row for s.no 1, enter ''yes'' for both ''user name'' and ''password'' columns, ''yes'' implying that a value is entered in that field. In the second row, enter ''yes'' and ''no'' and in the third row, ''no'' and 'yes'' and so on.

The complexity increases with the number of controls in the screen. Each of the row is considered as one condition and executed while testing. This is how we prepare test coverage matrix. (this is a black box testing technique).

Traceability matrix:

While, Traceability matrix serves in mapping the test cases to the requirements. It serves as a checklist wherein all the requirements (of srs) are listed and the test cases covering the corresponding requirement is listed against each requirement. Every company may have their own template for rtm, but they serve the same purpose as described above.

Top 24 replies by programmers when their programs don't work:

Top 24 replies by programmers when their programs don't work:

24. "It works fine on MY computer"
23. "Who did you login as ?"
22. "It's a feature"
21. "It's WAD (Working As Designed)"
20. "That's weird..."
19. "It's never done that before."
18. "It worked yesterday."
17. "How is that possible?"
16. "It must be a hardware problem."
15. "What did you type in wrong to get it to crash?"
14. "There is something funky in your data."
13. "I haven't touched that module in weeks!"
12. "You must have the wrong version."
11. "It's just some unlucky coincidence."
10. "I can't test everything!"
9. "THIS can't be the source of THAT."
8. "It works, but it's not been tested."
7. "Somebody must have changed my code."
6. "Did you check for a virus on your system?"
5. "Even though it doesn't work, how does it feel?"
4. "You can't use that version on your system."
3. "Why do you want to do it that way?"
2. "Where were you when the program blew up?"
1. "I thought I fixed that."

Sunday, December 23, 2007