Defect Classes, the Defect Repository, and Test Design

Defects can be classified in many ways. It is important for an organization

to adapt a single classification scheme and apply it to all projects. No

matter which classification scheme is selected, some defects will fit into

more than one class or category. Because of this problem, developers,

testers, and SQA staff should try to be as consistent as possible when

recording defect data. The defect types and frequency of occurrence

should be used to guide test planning, and test design. Execution-based

testing strategies should be selected that have the strongest possibility of

detecting particular types of defects. It is important that tests for new and

modified software be designed to detect the most frequently occurring

defects. The reader should keep in mind that execution-based testing will

detect a large number of the defects that will be described; however, software

reviews as described in Chapter 10 are also an excellent testing tool

for detection of many of the defect types that will be discussed in the

following sections.

Defects, as described in this text, are assigned to four major classes

reflecting their point of origin in the software life cycle—the development

phase in which they were injected. These classes are: requirements/

specifications, design, code, and testing defects as summarized in Figure

3.2. It should be noted that these defect classes and associated subclasses

focus on defects that are the major focus of attention to execution-based

testers. The list does not include other defects types that are best found

in software reviews, for example, those defects related to conformance to

styles and standards. The review checklists in Chapter 10 focus on many

of these types of defects.

3 . 1 . 1 R e q u i r e m e n t s a n d S p e c i f i c a t i o n D e f e c t s

The beginning of the software life cycle is critical for ensuring high quality

in the software being developed. Defects injected in early phases can persist

and be very difficult to remove in later phases. Since many requirements

documents are written using a natural language representation,

there are very often occurrences of ambiguous, contradictory, unclear,

redundant, and imprecise requirements. Specifications in many organizations

are also developed using natural language representations, and

these too are subject to the same types of problems as mentioned above.

However, over the past several years many organizations have introduced

the use of formal specification languages that, when accompanied by

tools, help to prevent incorrect descriptions of system behavior. Some

specific requirements/specification defects are:

FIG. 3.2

Defect classes and the defect repository.

1 . Functional Description Defects

The overall description of what the product does, and how it should

behave (inputs/outputs), is incorrect, ambiguous, and/or incomplete.

2 . Feature Defects

Features may be described as distinguishing characteristics of a software component

or system.

           

Features refer to functional aspects of the software that map to functional

requirements as described by the users and clients. Features also map to

quality requirements such as performance and reliability. Feature defects

are due to feature descriptions that are missing, incorrect, incomplete, or

superfluous.

3 . Feature Interaction Defects

These are due to an incorrect description of how the features should interact.

For example, suppose one feature of a software system supports

adding a new customer to a customer database. This feature interacts with

another feature that categorizes the new customer. The classification feature

impacts on where the storage algorithm places the new customer in

the database, and also affects another feature that periodically supports

sending advertising information to customers in a specific category. When

testing we certainly want to focus on the interactions between these

features.

4 . Interface Description Defects

These are defects that occur in the description of how the target software

is to interface with external software, hardware, and users.

For detecting many functional description defects, black box testing

techniques, which are based on functional specifications of the software,

offer the best approach. In Chapter 4 the reader will be introduced to

several black box testing techniques such as equivalence class partitioning,

boundary value analysis, state transition testing, and cause-and-effect

graphing, which are useful for detecting functional types of detects. Random

testing and error guessing are also useful for detecting these types of

defects. The reader should note that many of these types of defects can

be detected early in the life cycle by software reviews.

Black box–based tests can be planned at the unit, integration, system,

and acceptance levels to detect requirements/specification defects. Many

feature interaction and interfaces description defects are detected using

black box–based test designs at the integration and system levels.

3 . 1 . 2 D e s i g n D e f e c t s

Design defects occur when system components, interactions between system

components, interactions between the components and outside soft3.1

ware/hardware, or users are incorrectly designed. This covers defects in

the design of algorithms, control, logic, data elements, module interface

descriptions, and external software/hardware/user interface descriptions.

When describing these defects we assume that the detailed design description

for the software modules is at the pseudo code level with processing

steps, data structures, input/output parameters, and major control structures

defined. If module design is not described in such detail then many

of the defects types described here may be moved into the coding defects

class.

1 . Algorithmic and Processing Defects

These occur when the processing steps in the algorithm as described by

the pseudo code are incorrect. For example, the pseudo code may contain

a calculation that is incorrectly specified, or the processing steps in the

algorithm written in the pseudo code language may not be in the correct

order. In the latter case a step may be missing or a step may be duplicated.

Another example of a defect in this subclass is the omission of error condition

checks such as division by zero. In the case of algorithm reuse, a

designer may have selected an inappropriate algorithm for this problem

(it may not work for all cases).

2 . Control, Logic, and Sequence Defects

Control defects occur when logic flow in the pseudo code is not correct.

For example, branching to soon, branching to late, or use of an incorrect

branching condition. Other examples in this subclass are unreachable

pseudo code elements, improper nesting, improper procedure or function

calls. Logic defects usually relate to incorrect use of logic operators, such

as less than (_), greater than (_), etc. These may be used incorrectly in

a Boolean expression controlling a branching instruction.

3 . Data Defects

These are associated with incorrect design of data structures. For example,

a record may be lacking a field, an incorrect type is assigned to a

variable or a field in a record, an array may not have the proper number

of elements assigned, or storage space may be allocated incorrectly. Soft48

| Defects, Hypotheses, and Tests

ware reviews and use of a data dictionary work well to reveal these types

of defects.

4 . Module Interface Description Defects

These are defects derived from, for example, using incorrect, and/or inconsistent

parameter types, an incorrect number of parameters, or an

incorrect ordering of parameters.

5 . Functional Description Defects

The defects in this category include incorrect, missing, and/or unclear

design elements. For example, the design may not properly describe the

correct functionality of a module. These defects are best detected during

a design review.

6 . External Interface Description Defects

These are derived from incorrect design descriptions for interfaces with

COTS components, external software systems, databases, and hardware

devices (e.g., I/O devices). Other examples are user interface description

defects where there are missing or improper commands, improper sequences

of commands, lack of proper messages, and/or lack of feedback

messages for the user.

3 . 1 . 3 C o d i n g D e f e c t s

Coding defects are derived from errors in implementing the code. Coding

defects classes are closely related to design defect classes especially if

pseudo code has been used for detailed design. Some coding defects come

from a failure to understand programming language constructs, and miscommunication

with the designers. Others may have transcription or

omission origins. At times it may be difficult to classify a defect as a design

or as a coding defect. It is best to make a choice and be consistent when

the same defect arises again.

1 . Algorithmic and Processing Defects

Adding levels of programming detail to design, code-related algorithmic

and processing defects would now include unchecked overflow and

3.1 Defect Classes, the Defect Reposi tory, and Test Design | 49

underflow conditions, comparing inappropriate data types, converting

one data type to another, incorrect ordering of arithmetic operators (perhaps

due to misunderstanding of the precedence of operators), misuse or

omission of parentheses, precision loss, and incorrect use of signs.

2 . Control, Logic and Sequence Defects

On the coding level these would include incorrect expression of case statements,

incorrect iteration of loops (loop boundary problems), and missing

paths.

3 . Typographical Defects

These are principally syntax errors, for example, incorrect spelling of a

variable name, that are usually detected by a compiler, self-reviews, or

peer reviews.

4 . I n i t i a l i z a t i o n Defects

These occur when initialization statements are omitted or are incorrect.

This may occur because of misunderstandings or lack of communication

between programmers, and/or programmers and designers, carelessness,

or misunderstanding of the programming environment.

5 . Data-Flow Defects

There are certain reasonable operational sequences that data should flow

through. For example, a variable should be initialized, before it is used

in a calculation or a condition. It should not be initialized twice before

there is an intermediate use. A variable should not be disregarded before

it is used. Occurrences of these suspicious variable uses in the code may,

or may not, cause anomalous behavior. Therefore, in the strictest sense

of the definition for the term “defect,” they may not be considered as true

instances of defects. However, their presence indicates an error has occurred

and a problem exists that needs to be addressed.

6 . Data Defects

These are indicated by incorrect implementation of data structures. For

example, the programmer may omit a field in a record, an incorrect type

or access is assigned to a file, an array may not be allocated the proper

number of elements. Other data defects include flags, indices, and constants

set incorrectly.

7 . Module Interface Defects

As in the case of module design elements, interface defects in the code

may be due to using incorrect or inconsistent parameter types, an incorrect

number of parameters, or improper ordering of the parameters. In

addition to defects due to improper design, and improper implementation

of design, programmers may implement an incorrect sequence of calls or

calls to nonexistent modules.

8 . Code Documentation Defects

When the code documentation does not reflect what the program actually

does, or is incomplete or ambiguous, this is called a code documentation

defect. Incomplete, unclear, incorrect, and out-of-date code documentation

affects testing efforts. Testers may be misled by documentation defects

and thus reuse improper tests or design new tests that are not appropriate

for the code. Code reviews are the best tools to detect these

types of defects.

9 . External Hardware, Software Interfaces Defects

These defects arise from problems related to system calls, links to databases,

input/output sequences, memory usage, resource usage, interrupts

and exception handling, data exchanges with hardware, protocols, formats,

interfaces with build files, and timing sequences (race conditions

may result).

Many initialization, data flow, control, and logic defects that occur

in design and code are best addressed by white box testing techniques

applied at the unit (single-module) level. For example, data flow testing

is useful for revealing data flow defects, branch testing is useful for detecting

control defects, and loop testing helps to reveal loop-related defects.

White box testing approaches are dependent on knowledge of the

internal structure of the software, in contrast to black box approaches,

which are only dependent on behavioral specifications. The reader will

be introduced to several white box–based techniques in Chapter 5. Many

design and coding defects are also detected by using black box testing

techniques. For example, application of decision tables is very useful for

detecting errors in Boolean expressions. Black box tests as described in

Chapter 4 applied at the integration and system levels help to reveal external

hardware and software interface defects. The author will stress

repeatedly throughout the text that a combination of both of these approaches

is needed to reveal the many types of defects that are likely to

be found in software.

3 . 1 . 4 T e s t i n g D e f e c t s

Defects are not confined to code and its related artifacts. Test plans, test

cases, test harnesses, and test procedures can also contain defects. Defects

in test plans are best detected using review techniques.

1 . Test Harness Defects

In order to test software, especially at the unit and integration levels,

auxiliary code must be developed. This is called the test harness or scaffolding

code. Chapter 6 has a more detailed discussion of the need for

this code. The test harness code should be carefully designed, implemented,

and tested since it a work product and much of this code can be

reused when new releases of the software are developed. Test harnesses

are subject to the same types of code and design defects that can be found

in all other types of software.

2 . Test Case Design and Test Procedure Defects

These would encompass incorrect, incomplete, missing, inappropriate test

cases, and test procedures. These defects are again best detected in test

plan reviews as described in Chapter 10. Sometimes the defects are revealed

during the testing process itself by means of a careful analysis of

test conditions and test results. Repairs will then have to be made.

Software Testing Principles

Principles play an important role in all engineering disciplines and are

usually introduced as part of an educational background in each branch

of engineering. Figure 1.1 shows the role of basic principles in various

engineering disciplines. Testing principles are important to test specialists/

engineers because they provide the foundation for developing testing

knowledge and acquiring testing skills. They also provide guidance for

defining testing activities as performed in the practice of a test specialist.

A principle can be defined as:

1. a general or fundamental, law, doctrine, or assumption;

2. a rule or code of conduct;

3. the laws or facts of nature underlying the working of an artificial

device.

Extending these three definitions to the software engineering domain

we can say that software engineering principles refer to laws, rules, or

doctrines that relate to software systems, how to build them, and how

they behave. In the software domain, principles may also refer to rules or

codes of conduct relating to professionals who design, develop, test, and

maintain software systems. Testing as a component of the software engineering

discipline also has a specific set of principles that serve as guidelines

for the tester. They guide testers in defining how to test software

systems, and provide rules of conduct for testers as professionals. Glenford

Myers has outlined such a set of execution-based testing principles

in his pioneering book, The Art of Software Testing [9]. Some of these

principles are described below. Principles 1–8, and 11 are derived directly

from Myers’ original set. The author has reworded these principles, and

also has made modifications to the original set to reflect the evolution of

testing from an art, to a quality-related process within the context of an

engineering discipline. Note that the principles as stated below only relate

to execution-based testing. Principles relating to reviews, proof of correctness,

and certification as testing activities are not covered.

Principle 1. Testing is the process of exercising a software component

using a selected set of test cases, with the intent of (i) revealing

defects, and (ii) evaluating quality.

Software engineers have made great progress in developing methods to

prevent and eliminate defects. However, defects do occur, and they have

a negative impact on software quality. Testers need to detect these defects

before the software becomes operational. This principle supports testing

as an execution-based activity to detect defects. It also supports the separation

of testing from debugging since the intent of the latter is to locate

defects and repair the software. The term “software component” is used

in this context to represent any unit of software ranging in size and complexity

from an individual procedure or method, to an entire software

system. The term “defects” as used in this and in subsequent principles

represents any deviations in the software that have a negative impact on

its functionality, performance, reliability, security, and/or any other of its

specified quality attributes.

Bertolino, in the Guide to the Software Engineering Body of Knowledge,

gives a view of testing as a ‘‘dynamic process that executes a program

on valued inputs’’ [10]. This view, as well as the definition of testing

given in Chapter 1, suggest that in addition to detecting defects, testing

is also a process used to evaluate software quality. The purpose of the

former has been described in the previous paragraph. In the case of the

latter, the tester executes the software using test cases to evaluate properties

such as reliability, usability, maintainability, and level of performance.

Test results are used to compare the actual properties of the software

to those specified in the requirements document as quality goals.

Deviations or failure to achieve quality goals must be addressed.

The reader should keep in mind that testing can have a broader scope

as described in test process improvement models such as the TMM and

other quality models. Reviews and other static analysis techniques are

included under the umbrella of testing in the models. These techniques,

and how they relate to detecting defects and evaluating quality will be

described in subsequent chapters of this text.

Principle 2. When the test objective is to detect defects, then a good

test case is one that has a high probability of revealing a yetundetected

defect(s).

Principle 2 supports careful test design and provides a criterion with

which to evaluate test case design and the effectiveness of the testing effort

when the objective is to detect defects. It requires the tester to consider

the goal for each test case, that is, which specific type of defect is to be

detected by the test case. In this way the tester approaches testing in the

same way a scientist approaches an experiment. In the case of the scientist

there is a hypothesis involved that he/she wants to prove or disprove by

means of the experiment. In the case of the tester, the hypothesis is related

to the suspected occurrence of specific types of defects. The goal for the

test is to prove/disprove the hypothesis, that is, determine if the specific

defect is present/absent. Based on the hypothesis, test inputs are selected,

correct outputs are determined, and the test is run. Results are analyzed

to prove/disprove the hypothesis. The reader should realize that many

resources are invested in a test, resources for designing the test cases,

running the tests, and recording and analyzing results. A tester can justify

the expenditure of the resources by careful test design so that principle 2

is supported.

Principle 3. Test results should be inspected meticulously.

Testers need to carefully inspect and interpret test results. Several erroneous

and costly scenarios may occur if care is not taken. For example:

• A failure may be overlooked, and the test may be granted a “pass”

status when in reality the software has failed the test. Testing may

continue based on erroneous test results. The defect may be revealed

at some later stage of testing, but in that case it may be more costly

and difficult to locate and repair.

• A failure may be suspected when in reality none exists. In this case

the test may be granted a “fail” status. Much time and effort may be

spent on trying to find the defect that does not exist. A careful reexamination

of the test results could finally indicate that no failure

has occurred.

• The outcome of a quality test may be misunderstood, resulting in

unnecessary rework, or oversight of a critical problem.

Principle 4. A test case must contain the expected output or result.

It is often obvious to the novice tester that test inputs must be part of a

test case. However, the test case is of no value unless there is an explicit

statement of the expected outputs or results, for example, a specific variable

value must be observed or a certain panel button that must light up.

Expected outputs allow the tester to determine (i) whether a defect has

been revealed, and (ii) pass/fail status for the test. It is very important to

have a correct statement of the output so that needless time is not spent

due to misconceptions about the outcome of a test. The specification of

test inputs and outputs should be part of test design activities.

In the case of testing for quality evaluation, it is useful for quality

goals to be expressed in quantitative terms in the requirements document

if possible, so that testers are able to compare actual software attributes

as determined by the tests with what was specified.

Principle 5. Test cases should be developed for both valid and invalid

input conditions.

A tester must not assume that the software under test will always be

provided with valid inputs. Inputs may be incorrect for several reasons.

30 | Testing Fundamentals

For example, software users may have misunderstandings, or lack information

about the nature of the inputs. They often make typographical

errors even when complete/correct information is available. Devices may

also provide invalid inputs due to erroneous conditions and malfunctions.

Use of test cases that are based on invalid inputs is very useful for revealing

defects since they may exercise the code in unexpected ways and

identify unexpected software behavior. Invalid inputs also help developers

and testers evaluate the robustness of the software, that is, its ability to

recover when unexpected events occur (in this case an erroneous input).

Principle 5 supports the need for the independent test group called

for in Principle 7 for the following reason. The developer of a software

component may be biased in the selection of test inputs for the component

and specify only valid inputs in the test cases to demonstrate that the

software works correctly. An independent tester is more apt to select invalid

inputs as well.

Principle 6. The probability of the existence of additional defects in

a software component is proportional to the number of defects already

detected in that component.

What this principle says is that the higher the number of defects already

detected in a component, the more likely it is to have additional defects

when it undergoes further testing. For example, if there are two components

A and B, and testers have found 20 defects in A and 3 defects in B,

then the probability of the existence of additional defects in A is higher

than B. This empirical observation may be due to several causes. Defects

often occur in clusters and often in code that has a high degree of complexity

and is poorly designed. In the case of such components developers

and testers need to decide whether to disregard the current version of the

component and work on a redesign, or plan to expend additional testing

resources on this component to insure it meets its requirements. This issue

is especially important for components that implement mission or safety

critical functions.

Principle 7. Testing should be carried out by a group that is independent

of the development group.

This principle holds true for psychological as well as practical reasons. It

is difficult for a developer to admit or conceive that software he/she has

created and developed can be faulty. Testers must realize that (i) developers

have a great deal of pride in their work, and (ii) on a practical level

it may be difficult for them to conceptualize where defects could be found.

Even when tests fail, developers often have difficulty in locating the defects

since their mental model of the code may overshadow their view of

code as it exists in actuality. They may also have misconceptions or misunderstandings

concerning the requirements and specifications relating to

the software.

The requirement for an independent testing group can be interpreted

by an organization in several ways. The testing group could be implemented

as a completely separate functional entity in the organization.

Alternatively, testers could be members of a Software Quality Assurance

Group, or even be a specialized part of the development group, but in the

latter case especially, they need the capability to be objective. Reporting

to management that is separate from development can support their objectivity

and independence. As a member of any of these groups, the principal

duties and training of the testers should lie in testing rather than in

development.

Finally, independence of the testing group does not call for an adversarial

relationship between developers and testers. The testers should

not play “gotcha” games with developers. The groups need to cooperate

so that software of the highest quality is released to the customer.

Principle 8. Tests must be repeatable and reusable.

Principle 2 calls for a tester to view his/her work as similar to that of an

experimental scientist. Principle 8 calls for experiments in the testing domain

to require recording of the exact conditions of the test, any special

events that occurred, equipment used, and a careful accounting of the

results. This information is invaluable to the developers when the code is

returned for debugging so that they can duplicate test conditions. It is

also useful for tests that need to be repeated after defect repair. The repetition

and reuse of tests is also necessary during regression test (the retesting

of software that has been modified) in the case of a new release

of the software. Scientists expect experiments to be repeatable by others,

and testers should expect the same!

Principle 9. Testing should be planned.

Test plans should be developed for each level of testing, and objectives

for each level should be described in the associated plan. The objectives

should be stated as quantitatively as possible. Plans, with their precisely

specified objectives, are necessary to ensure that adequate time and resources

are allocated for testing tasks, and that testing can be monitored

and managed.

Test planning activities should be carried out throughout the software

life cycle (Principle 10). Test planning must be coordinated with project

planning. The test manager and project manager must work together to

coordinate activities. Testers cannot plan to test a component on a given

date unless the developers have it available on that date. Test risks must

be evaluated. For example, how probable are delays in delivery of software

components, which components are likely to be complex and difficult

to test, do the testers need extra training with new tools? A test plan

template must be available to the test manager to guide development of

the plan according to organizational policies and standards. Careful test

planning avoids wasteful “throwaway” tests and unproductive and unplanned

“test–patch–retest” cycles that often lead to poor-quality software

and the inability to deliver software on time and within budget.

Principle 10. Testing activities should be integrated into the software

life cycle.

It is no longer feasible to postpone testing activities until after the code

has been written. Test planning activities as supported by Principle 10,

should be integrated into the software life cycle starting as early as in the

requirements analysis phase, and continue on throughout the software

life cycle in parallel with development activities. In addition to test planning,

some other types of testing activities such as usability testing can

also be carried out early in the life cycle by using prototypes. These activities

can continue on until the software is delivered to the users. Organizations

can use process models like the V-model or any others that

support the integration of test activities into the software life cycle [11].

Principle 11. Testing is a creative and challenging task [12].

Difficulties and challenges for the tester include the following:

• A tester needs to have comprehensive knowledge of the software engineering

discipline.

• A tester needs to have knowledge from both experience and education

as to how software is specified, designed, and developed.

• A tester needs to be able to manage many details.

• A tester needs to have knowledge of fault types and where faults of

a certain type might occur in code constructs.

• A tester needs to reason like a scientist and propose hypotheses that

relate to presence of specific types of defects.

• A tester needs to have a good grasp of the problem domain of the

software that he/she is testing. Familiarly with a domain may come

from educational, training, and work-related experiences.

• A tester needs to create and document test cases. To design the test

cases the tester must select inputs often from a very wide domain.

Those selected should have the highest probability of revealing a defect

(Principle 2). Familiarly with the domain is essential.

• A tester needs to design and record test procedures for running the

tests.

• A tester needs to plan for testing and allocate the proper resources.

• A tester needs to execute the tests and is responsible for recording

results.

• A tester needs to analyze test results and decide on success or failure

for a test. This involves understanding and keeping track of an enor

mous amount of detailed information. A tester may also be required

to collect and analyze test-related measurements.

• A tester needs to learn to use tools and keep abreast of the newest

test tool advances.

• A tester needs to work and cooperate with requirements engineers,

designers, and developers, and often must establish a working relationship

with clients and users.

• A tester needs to be educated and trained in this specialized area and

often will be required to update his/her knowledge on a regular basis

due to changing technologies.

2DAY NO FIRST HOUR

HI,

TODAY NO FIST HOUR ALL ARE REQUESTED TO REACH OUR CLASS BY 7.15  ( SECOND HOUR)

"ANONYMOUS COMMENT " க்கு

  எனது அருமை "ANONYMOUS COMMENT " க்கு . கொஞ்சம் அலுவலில் ஆருவமாக  இருந்ததால் ப்ளாக் எழுத்தல. 

WAIT FOR SOME TIMES FOR THE POSTS..

nettil suttavai

பிலாசபி

வெள்ளம் அதிகமா வந்தா மீன்கள்... கரையில் இருக்கும் எறும்புகளை தின்னும்.. அதே வெள்ளம் வடிந்து தண்ணீர் வற்றினால் மீன்களை எறும்புகன் தின்னும்... சோ சான்ஸ் எல்லாருக்கும் இந்த உலகம் கொடுக்கின்றது.. உங்கள் சான்ஸ் வரும் வரை நீங்க வெயிட்டிங்ல இருக்க வேண்டும்....