Equivalence Class Partitioning

If a tester is viewing the software-under-test as a black box with welldefined

inputs and outputs, a good approach to selecting test inputs is to

use a method called equivalence class partitioning. Equivalence class partitioning

results in a partitioning of the input domain of the softwareunder-

test. The technique can also be used to partition the output domain,

but this is not a common usage. The finite number of partitions or equivalence

classes that result allow the tester to select a given member of an

equivalence class as a representative of that class. It is assumed that all

members of an equivalence class are processed in an equivalent way by

the target software.

Using equivalence class partitioning a test value in a particular class

is equivalent to a test value of any other member of that class. Therefore,

if one test case in a particular equivalence class reveals a defect, all the

other test cases based on that class would be expected to reveal the same

defect. We can also say that if a test case in a given equivalence class did

not detect a particular type of defect, then no other test case based on

that class would detect the defect (unless a subset of the equivalence class

falls into another equivalence class, since classes may overlap in some

cases). A more formal discussion of equivalence class partitioning is given

in Beizer [5].

Based on this discussion of equivalence class partitioning we can say

that the partitioning of the input domain for the software-under-test using

this technique has the following advantages:

1. It eliminates the need for exhaustive testing, which is not feasible.

2. It guides a tester in selecting a subset of test inputs with a high probability

of detecting a defect.

3. It allows a tester to cover a larger domain of inputs/outputs with a

smaller subset selected from an equivalence class.

Most equivalence class partitioning takes place for the input domain.

How does the tester identify equivalence classes for the input domain?

One approach is to use a set of what Glen Myers calls “interesting” input

conditions [1]. The input conditions usually come from a description in

the specification of the software to be tested. The tester uses the conditions

to partition the input domain into equivalence classes and then develops

a set of tests cases to cover (include) all the classes. Given that only the

information in an input/output specification is needed, the tester can begin

to develop black box tests for software early in the software life cycle in

parallel with analysis activities (see Principle 11, Chapter 2). The tester

and the analyst interact during the analysis phase to develop (i) a set of

testable requirements, and (ii) a correct and complete input/output specification.

From these the tester develops, (i) a high-level test plan, and

(ii) a preliminary set of black box test cases for the system. Both the plan

and the test cases undergo further development in subsequent life cycle

phases. The V-Model as described in Chapter 8 supports this approach.

There are several important points related to equivalence class partitioning

that should be made to complete this discussion.

1. The tester must consider both valid and invalid equivalence classes.

Invalid classes represent erroneous or unexpected inputs.

2. Equivalence classes may also be selected for output conditions.

3. The derivation of input or outputs equivalence classes is a heuristic

process. The conditions that are described in the following paragraphs

only give the tester guidelines for identifying the partitions.

There are no hard and fast rules. Given the same set of conditions,

individual testers may make different choices of equivalence classes.

As a tester gains experience he is more able to select equivalence

classes with confidence.

4. In some cases it is difficult for the tester to identify equivalence classes.

The conditions/boundaries that help to define classes may be absent,

or obscure, or there may seem to be a very large or very small number

of equivalence classes for the problem domain. These difficulties may

arise from an ambiguous, contradictory, incorrect, or incomplete

specification and/or requirements description. It is the duty of the

tester to seek out the analysts and meet with them to clarify these

documents. Additional contact with the user/client group may be required.

A tester should also realize that for some software problem

domains defining equivalence classes is inherently difficult, for example,

software that needs to utilize the tax code.

Myers suggests the following conditions as guidelines for selecting

input equivalence classes [1]. Note that a condition is usually associated

with a particular variable. We treat each condition separately. Test cases,

when developed, may cover multiple conditions and multiple variables.

List of Conditions

1. ‘‘If an input condition for the software-under-test is specified as a

range of values, select one valid equivalence class that covers the allowed

range and two invalid equivalence classes, one outside each

end of the range.’’

For example, suppose the specification for a module says that an

input, the length of a widget in millimeters, lies in the range 1–499;

then select one valid equivalence class that includes all values from 1

to 499. Select a second equivalence class that consists of all values

less than 1, and a third equivalence class that consists of all values

greater than 499.

2. ‘‘If an input condition for the software-under-test is specified as a

number of values, then select one valid equivalence class that includes

the allowed number of values and two invalid equivalence classes that

are outside each end of the allowed number.’’

For example, if the specification for a real estate-related module

say that a house can have one to four owners, then we select one valid

equivalence class that includes all the valid number of owners, and

then two invalid equivalence classes for less than one owner and more

than four owners.

3. ‘‘If an input condition for the software-under-test is specified as a set

of valid input values, then select one valid equivalence class that contains

all the members of the set and one invalid equivalence class for

any value outside the set.’’

For example, if the specification for a paint module states that

the colors RED, BLUE, GREEN and YELLOW are allowed as inputs,

then select one valid equivalence class that includes the set RED,

BLUE, GREEN and YELLOW, and one invalid equivalence class for

all other inputs.

4. ‘‘If an input condition for the software-under-test is specified as a

“must be” condition, select one valid equivalence class to represent

the “must be” condition and one invalid class that does not include

the “must be” condition.’’

For example, if the specification for a module states that the first

character of a part identifier must be a letter, then select one valid

equivalence class where the first character is a letter, and one invalid

class where the first character is not a letter.

5. ‘‘If the input specification or any other information leads to the belief

that an element in an equivalence class is not handled in an identical

way by the software-under-test, then the class should be further partitioned

into smaller equivalence classes.’’

To show how equivalence classes can be derived from a specification,

consider an example in Figure 4.2. This is a specification for a module

that calculates a square root.

The specification describes for the tester conditions relevant to the

Function square_root

message (x:real)

when x >_0.0

reply (y:real)

where y >_0.0 & approximately (y*y,x)

otherwise reply exception imaginary_square_root

end function

FIG. 4.2

A specification of a square root

function.

input/output variables x and y. The input conditions are that the variable

x must be a real number and be equal to or greater than 0.0. The conditions

for the output variable y are that it must be a real number equal

to or greater than 0.0, whose square is approximately equal to x. If x is

not equal to or greater than 0.0, then an exception is raised. From this

information the tester can easily generate both invalid and valid equivalence

classes and boundaries. For example, input equivalence classes for

this module are the following:

EC1. The input variable x is real, valid.

EC2. The input variable x is not real, invalid.

EC3. The value of x is greater than 0.0, valid.

EC4. The value of x is less than 0.0, invalid.

Because many organizations now use some type of formal or semiformal

specifications, testers have a reliable source for applying the input/output

conditions described by Myers.

After the equivalence classes have been identified in this way, the next

step in test case design is the development of the actual test cases. A good

approach includes the following steps.

1. Each equivalence class should be assigned a unique identifier. A simple

integer is sufficient.

2. Develop test cases for all valid equivalence classes until all have been

covered by (included in) a test case. A given test case may cover more

than one equivalence class.

3. Develop test cases for all invalid equivalence classes until all have

been covered individually. This is to insure that one invalid case does

not mask the effect of another or prevent the execution of another.

An example of applying equivalence class partitioning will be shown

in the next section.

4 . 6 Boundary Value Analysis

Equivalence class partitioning gives the tester a useful tool with which

to develop black box based-test cases for the software-under-test. The

method requires that a tester has access to a specification of input/output

behavior for the target software. The test cases developed based on equivalence

class partitioning can be strengthened by use of an another technique

called boundary value analysis. With experience, testers soon realize

that many defects occur directly on, and above and below, the edges

of equivalence classes. Test cases that consider these boundaries on both

the input and output spaces as shown in Figure 4.3 are often valuable in

revealing defects.

Whereas equivalence class partitioning directs the tester to select test

cases from any element of an equivalence class, boundary value analysis

requires that the tester select elements close to the edges, so that both the

upper and lower edges of an equivalence class are covered by test cases.

As in the case of equivalence class partitioning, the ability to develop highquality

test cases with the use of boundary values requires experience.

The rules-of-thumb described below are useful for getting started with

boundary value analysis.

1. If an input condition for the software-under-test is specified as a range

of values, develop valid test cases for the ends of the range, and invalid

test cases for possibilities just above and below the ends of the

range.

For example if a specification states that an input value for a

module must lie in the range between _1.0 and _1.0, valid tests

that include values for ends of the range, as well as invalid test cases

for values just above and below the ends, should be included. This

would result in input values of _1.0, _1.1, and 1.0, 1.1.

Boundaries of an equivalence

partition.

2. If an input condition for the software-under-test is specified as a number

of values, develop valid test cases for the minimum and maximum

numbers as well as invalid test cases that include one lesser and one

greater than the maximum and minimum.

For example, for the real-estate module mentioned previously

that specified a house can have one to four owners, tests that include

0,1 owners and 4,5 owners would be developed.

The following is an example of applying boundary value analysis

to output equivalence classes. Suppose a table of 1 to 100 values is

to be produced by a module. The tester should select input data to

generate an output table of size 0,1, and 100 values, and if possible

101 values.

3. If the input or output of the software-under-test is an ordered set,

such as a table or a linear list, develop tests that focus on the first and

last elements of the set.

It is important for the tester to keep in mind that equivalence class

partitioning and boundary value analysis apply to testing both inputs and

outputs of the software-under-test, and, most importantly, conditions are

not combined for equivalence class partitioning or boundary value analysis.

Each condition is considered separately, and test cases are developed

to insure coverage of all the individual conditions

Defect Classes, the Defect Repository, and Test Design

Defects can be classified in many ways. It is important for an organization

to adapt a single classification scheme and apply it to all projects. No

matter which classification scheme is selected, some defects will fit into

more than one class or category. Because of this problem, developers,

testers, and SQA staff should try to be as consistent as possible when

recording defect data. The defect types and frequency of occurrence

should be used to guide test planning, and test design. Execution-based

testing strategies should be selected that have the strongest possibility of

detecting particular types of defects. It is important that tests for new and

modified software be designed to detect the most frequently occurring

defects. The reader should keep in mind that execution-based testing will

detect a large number of the defects that will be described; however, software

reviews as described in Chapter 10 are also an excellent testing tool

for detection of many of the defect types that will be discussed in the

following sections.

Defects, as described in this text, are assigned to four major classes

reflecting their point of origin in the software life cycle—the development

phase in which they were injected. These classes are: requirements/

specifications, design, code, and testing defects as summarized in Figure

3.2. It should be noted that these defect classes and associated subclasses

focus on defects that are the major focus of attention to execution-based

testers. The list does not include other defects types that are best found

in software reviews, for example, those defects related to conformance to

styles and standards. The review checklists in Chapter 10 focus on many

of these types of defects.

3 . 1 . 1 R e q u i r e m e n t s a n d S p e c i f i c a t i o n D e f e c t s

The beginning of the software life cycle is critical for ensuring high quality

in the software being developed. Defects injected in early phases can persist

and be very difficult to remove in later phases. Since many requirements

documents are written using a natural language representation,

there are very often occurrences of ambiguous, contradictory, unclear,

redundant, and imprecise requirements. Specifications in many organizations

are also developed using natural language representations, and

these too are subject to the same types of problems as mentioned above.

However, over the past several years many organizations have introduced

the use of formal specification languages that, when accompanied by

tools, help to prevent incorrect descriptions of system behavior. Some

specific requirements/specification defects are:

FIG. 3.2

Defect classes and the defect repository.

1 . Functional Description Defects

The overall description of what the product does, and how it should

behave (inputs/outputs), is incorrect, ambiguous, and/or incomplete.

2 . Feature Defects

Features may be described as distinguishing characteristics of a software component

or system.

           

Features refer to functional aspects of the software that map to functional

requirements as described by the users and clients. Features also map to

quality requirements such as performance and reliability. Feature defects

are due to feature descriptions that are missing, incorrect, incomplete, or

superfluous.

3 . Feature Interaction Defects

These are due to an incorrect description of how the features should interact.

For example, suppose one feature of a software system supports

adding a new customer to a customer database. This feature interacts with

another feature that categorizes the new customer. The classification feature

impacts on where the storage algorithm places the new customer in

the database, and also affects another feature that periodically supports

sending advertising information to customers in a specific category. When

testing we certainly want to focus on the interactions between these

features.

4 . Interface Description Defects

These are defects that occur in the description of how the target software

is to interface with external software, hardware, and users.

For detecting many functional description defects, black box testing

techniques, which are based on functional specifications of the software,

offer the best approach. In Chapter 4 the reader will be introduced to

several black box testing techniques such as equivalence class partitioning,

boundary value analysis, state transition testing, and cause-and-effect

graphing, which are useful for detecting functional types of detects. Random

testing and error guessing are also useful for detecting these types of

defects. The reader should note that many of these types of defects can

be detected early in the life cycle by software reviews.

Black box–based tests can be planned at the unit, integration, system,

and acceptance levels to detect requirements/specification defects. Many

feature interaction and interfaces description defects are detected using

black box–based test designs at the integration and system levels.

3 . 1 . 2 D e s i g n D e f e c t s

Design defects occur when system components, interactions between system

components, interactions between the components and outside soft3.1

ware/hardware, or users are incorrectly designed. This covers defects in

the design of algorithms, control, logic, data elements, module interface

descriptions, and external software/hardware/user interface descriptions.

When describing these defects we assume that the detailed design description

for the software modules is at the pseudo code level with processing

steps, data structures, input/output parameters, and major control structures

defined. If module design is not described in such detail then many

of the defects types described here may be moved into the coding defects

class.

1 . Algorithmic and Processing Defects

These occur when the processing steps in the algorithm as described by

the pseudo code are incorrect. For example, the pseudo code may contain

a calculation that is incorrectly specified, or the processing steps in the

algorithm written in the pseudo code language may not be in the correct

order. In the latter case a step may be missing or a step may be duplicated.

Another example of a defect in this subclass is the omission of error condition

checks such as division by zero. In the case of algorithm reuse, a

designer may have selected an inappropriate algorithm for this problem

(it may not work for all cases).

2 . Control, Logic, and Sequence Defects

Control defects occur when logic flow in the pseudo code is not correct.

For example, branching to soon, branching to late, or use of an incorrect

branching condition. Other examples in this subclass are unreachable

pseudo code elements, improper nesting, improper procedure or function

calls. Logic defects usually relate to incorrect use of logic operators, such

as less than (_), greater than (_), etc. These may be used incorrectly in

a Boolean expression controlling a branching instruction.

3 . Data Defects

These are associated with incorrect design of data structures. For example,

a record may be lacking a field, an incorrect type is assigned to a

variable or a field in a record, an array may not have the proper number

of elements assigned, or storage space may be allocated incorrectly. Soft48

| Defects, Hypotheses, and Tests

ware reviews and use of a data dictionary work well to reveal these types

of defects.

4 . Module Interface Description Defects

These are defects derived from, for example, using incorrect, and/or inconsistent

parameter types, an incorrect number of parameters, or an

incorrect ordering of parameters.

5 . Functional Description Defects

The defects in this category include incorrect, missing, and/or unclear

design elements. For example, the design may not properly describe the

correct functionality of a module. These defects are best detected during

a design review.

6 . External Interface Description Defects

These are derived from incorrect design descriptions for interfaces with

COTS components, external software systems, databases, and hardware

devices (e.g., I/O devices). Other examples are user interface description

defects where there are missing or improper commands, improper sequences

of commands, lack of proper messages, and/or lack of feedback

messages for the user.

3 . 1 . 3 C o d i n g D e f e c t s

Coding defects are derived from errors in implementing the code. Coding

defects classes are closely related to design defect classes especially if

pseudo code has been used for detailed design. Some coding defects come

from a failure to understand programming language constructs, and miscommunication

with the designers. Others may have transcription or

omission origins. At times it may be difficult to classify a defect as a design

or as a coding defect. It is best to make a choice and be consistent when

the same defect arises again.

1 . Algorithmic and Processing Defects

Adding levels of programming detail to design, code-related algorithmic

and processing defects would now include unchecked overflow and

3.1 Defect Classes, the Defect Reposi tory, and Test Design | 49

underflow conditions, comparing inappropriate data types, converting

one data type to another, incorrect ordering of arithmetic operators (perhaps

due to misunderstanding of the precedence of operators), misuse or

omission of parentheses, precision loss, and incorrect use of signs.

2 . Control, Logic and Sequence Defects

On the coding level these would include incorrect expression of case statements,

incorrect iteration of loops (loop boundary problems), and missing

paths.

3 . Typographical Defects

These are principally syntax errors, for example, incorrect spelling of a

variable name, that are usually detected by a compiler, self-reviews, or

peer reviews.

4 . I n i t i a l i z a t i o n Defects

These occur when initialization statements are omitted or are incorrect.

This may occur because of misunderstandings or lack of communication

between programmers, and/or programmers and designers, carelessness,

or misunderstanding of the programming environment.

5 . Data-Flow Defects

There are certain reasonable operational sequences that data should flow

through. For example, a variable should be initialized, before it is used

in a calculation or a condition. It should not be initialized twice before

there is an intermediate use. A variable should not be disregarded before

it is used. Occurrences of these suspicious variable uses in the code may,

or may not, cause anomalous behavior. Therefore, in the strictest sense

of the definition for the term “defect,” they may not be considered as true

instances of defects. However, their presence indicates an error has occurred

and a problem exists that needs to be addressed.

6 . Data Defects

These are indicated by incorrect implementation of data structures. For

example, the programmer may omit a field in a record, an incorrect type

or access is assigned to a file, an array may not be allocated the proper

number of elements. Other data defects include flags, indices, and constants

set incorrectly.

7 . Module Interface Defects

As in the case of module design elements, interface defects in the code

may be due to using incorrect or inconsistent parameter types, an incorrect

number of parameters, or improper ordering of the parameters. In

addition to defects due to improper design, and improper implementation

of design, programmers may implement an incorrect sequence of calls or

calls to nonexistent modules.

8 . Code Documentation Defects

When the code documentation does not reflect what the program actually

does, or is incomplete or ambiguous, this is called a code documentation

defect. Incomplete, unclear, incorrect, and out-of-date code documentation

affects testing efforts. Testers may be misled by documentation defects

and thus reuse improper tests or design new tests that are not appropriate

for the code. Code reviews are the best tools to detect these

types of defects.

9 . External Hardware, Software Interfaces Defects

These defects arise from problems related to system calls, links to databases,

input/output sequences, memory usage, resource usage, interrupts

and exception handling, data exchanges with hardware, protocols, formats,

interfaces with build files, and timing sequences (race conditions

may result).

Many initialization, data flow, control, and logic defects that occur

in design and code are best addressed by white box testing techniques

applied at the unit (single-module) level. For example, data flow testing

is useful for revealing data flow defects, branch testing is useful for detecting

control defects, and loop testing helps to reveal loop-related defects.

White box testing approaches are dependent on knowledge of the

internal structure of the software, in contrast to black box approaches,

which are only dependent on behavioral specifications. The reader will

be introduced to several white box–based techniques in Chapter 5. Many

design and coding defects are also detected by using black box testing

techniques. For example, application of decision tables is very useful for

detecting errors in Boolean expressions. Black box tests as described in

Chapter 4 applied at the integration and system levels help to reveal external

hardware and software interface defects. The author will stress

repeatedly throughout the text that a combination of both of these approaches

is needed to reveal the many types of defects that are likely to

be found in software.

3 . 1 . 4 T e s t i n g D e f e c t s

Defects are not confined to code and its related artifacts. Test plans, test

cases, test harnesses, and test procedures can also contain defects. Defects

in test plans are best detected using review techniques.

1 . Test Harness Defects

In order to test software, especially at the unit and integration levels,

auxiliary code must be developed. This is called the test harness or scaffolding

code. Chapter 6 has a more detailed discussion of the need for

this code. The test harness code should be carefully designed, implemented,

and tested since it a work product and much of this code can be

reused when new releases of the software are developed. Test harnesses

are subject to the same types of code and design defects that can be found

in all other types of software.

2 . Test Case Design and Test Procedure Defects

These would encompass incorrect, incomplete, missing, inappropriate test

cases, and test procedures. These defects are again best detected in test

plan reviews as described in Chapter 10. Sometimes the defects are revealed

during the testing process itself by means of a careful analysis of

test conditions and test results. Repairs will then have to be made.