Home » 7/1/07 - 8/1/07
Web Testing !!!

Web Testing:

 Testing a web application such as functionality, usability, server side interface, client side compatibility, security.

 There are generally three layers in web application. They are
1. Presentation layer
2. Business layer
3. Database

Under:
Testing Software Metrics !!!

Testing metrics:
*****************

* User participation:
To find the involvement of the tester = User participation / (test time vs total time)

 * Path tested:
Extent of testing = No. of path tested / Total no. of paths.

 * Acceptance criteria tested
Extent of testing = Acceptance criteria vs. Total acceptance criteria.

 * Test Cost:
Resources consumed in testing = Total cost vs. Total system cost.
This metric identifies the amount of resources used in the testing process.

 * Cost to locate defect:
Resources consumed in the testing = Total cost / No. of defects located in testing.
This metric shows the cost to locate a defect.

 * Detected production defect:
Effectiveness of testing = No. of defects detected in production / Application system size.

 * Test automation
Effectiveness of testing = cost of manual test / test cost.

Under: , , , ,
Types of Software Metrics !!!

Types of Metrics
******************
Process metrics and Product metrics

 Process metric:
A metric used to measure the characteristic of the methods techniques and tools employed in developing, maintaining and implementing the software system.

Product metric:
A metric used to measure the characteristic of the documentation and code.

Under: , , , ,
Software Metrics !!!

Software Metrics
*******************
A metric quantifies a characteristic of a process or product. Metrics can be directly observable quantities or can be derived from one or more directly observable quantities. Examples of raw metrics include the number of source lines of code, number of documentation pages, number of staff-hours, number of tests, number of requirements, etc. Examples of derived metrics include source lines of code per staff-hour, defects per thousand lines of code, or a cost performance index.

Software metrics are numerical data related to software development. Metrics strongly support software project management activities. They relate to the four functions of management as follows:
1. Planning - Metrics serve as a basis of cost estimating, training planning, resource planning, scheduling, and budgeting.
2. Organizing - Size and schedule metrics influence a project's organization.
3. Controlling - Metrics are used to status and track software development activities for compliance to plans.
4. Improving - Metrics are used as a tool for process improvement and to identify where improvement efforts should be concentrated and measure the effects of process improvement efforts.
The term indicator is used to denote a representation of metric data that provides insight into an ongoing software development project or process improvement activity. Indicators are metrics in a form suitable for assessing project behavior or process improvement. For example, an indicator may be the behavior of a metric over time or the ratio of two metrics. Indicators may include the comparison of actual values versus the plan, project stability metrics, or quality metrics. Examples of indicators used on a project include actual versus planned task completions, actual versus planned staffing, number of trouble reports written and resolved over time, and number of requirements changes over time.

Indicators are used in conjunction with one another to provide a more complete picture of project or organization behavior. For example, a progress indicator is related to requirements and size indicators. All three indicators should be used and interpreted together.

Under: , , , ,
Test Plan !!!

The Following are the test plan outlines:

 1. BACKGROUND
2. INTRODUCTION
3. ASSUMPTIONS
4. TEST ITEMS - List each of the items (programs) to be tested.
5. FEATURES TO BE TESTED - List each of the features (functions or requirements) which will be tested or demonstrated by the test.
6. FEATURES NOT TO BE TESTED
Explicitly lists each feature, function, or requirement which won't be tested and why not.
7. APPROACH
Describe the data flows and test philosophy.
Simulation or Live execution, Etc.
8. ITEM PASS/FAIL CRITERIA Blanket statement
Itemized list of expected output and tolerances
9. SUSPENSION/RESUMPTION CRITERIA
Must the test run from start to completion?
Under what circumstances may it be resumed in the middle?
Establish check-points in long tests.
10.TEST DELIVERABLE
What, besides software, will be delivered?

 Test report
Test software
11. TESTING TASKS Functional tasks (e.g., equipment set up)
Administrative tasks
12. ENVIRONMENTAL NEEDS
Security clearance
Office space & equipment
Hardware/software requirements
13. RESPONSIBILITIES
Who does the tasks in Section 10?
What does the user do?
14. STAFFING & TRAINING
15. SCHEDULE
16. RESOURCES
17. RISKS & CONTINGENCIES
18. APPROVALS

Under: , ,
Test Plan Preparation !!!

The following concepts are present in the Test plan preparation:

 Test Objective
A Test Objective is simply a testing "goal". It is a statement of what a tester is expected to perform during testing activity.

 Test Strategy
It contains the following datas
1.Technology that are going to be used during the testing.
2.What types of tests must be conducted.
3.what stage of testing are required. (eg. Unit Testing)

 Test Risk analysis
A risk is a condition that can result in a loss. The risk that may occur during analysis of the testing. The probability that the negative event will occur.

 Risk identify and mitigation
This plan contains the unavailability of resources (eg. hardware and software ). And what are the ways to overcome.

Scheduled and resources
Roles and Responsibilities
Test Manager - Prepares the test plan
Test Leader - Review the test case, take decision.
Test engineer - Prepare the test cases.

 Communication approach
The communication during the project will be informed through this communication approach.

 Test Environment
What are the environment that the person needed to develop and testing.
(eg. Tools that are needed and Hardware & software requirements)

 Automated Test tools:
Identify which tool is used for each individual test and should know the test level criteria.

Under: ,
Testing Life Cycle (TLC) !!!

Testing Life Cycle:

Steps involved in testing an application:

 Test Plan preparation
It is a document that defines over all testing approach.

 Test case design
Documents that defines what is selected to test and describes its expected results

 Test Execution
Executing the test cases.

 Test Log Preparation
Pass and fail information will be stored in test log.

 Defect cracking
All the fail items will come under defect tracking.

 Test reports
Reports are generated during the final test summary report.

Under: ,
Verification and Validation !!!

Verification:
It is the process of confirming that software meets its specifications.It is used to identify the defects in the product early in the life cycle.

 Classification:
1. Based on time
2.Based on classes.

 1. Based on Time:
• In process review
* During the specific period of the development cycle. Ex : Design phase
* Used to find the defects in the work product and the work process.
* Catches defects early where they are less costly to correct.
* It may occur at any time.

• Phase end/ decision making review/ milestone review
* Review of product or process near the completion of each phase of development.
* Decisions for proceeding with development or based on the cost, schedule, risk, progress, readiness for the next phase.
* Also referred to a milestone review

• Post implementation/ post Morten review
* Review or evaluation of the product that includes planned Vs actual development results and compliance with requirements.
* Used for process improvement of software development.
* Can be held up to 3 to 6 months after implementation.

 2. Based on classes:

• Informal (peer reviews)
* Generally one to one meeting.
* No agenda
* Results are not formally reported
* Occur as needed throughout each phase.

• Semi formal
* Facilitated by author(tester).
* Reports are distributed to the participants.
* Possible solutions for defects not discussed.
* Occurs one or more times during a phase.

 • Formal

 * Facilitated by moderator
* Assisted by recorder
* Meeting are planned in advance.
* Directly dependent on the preparation of participants.
* Held at any time.

Validation:
Its is the process confirming that it meets the users requirement.

Under: ,
Thread Testing !!!

 Thread Testing

 1. Its a technique that often used during in the early integration testing.
2. Demonstrates key functional capabilities by testing a string of units that accomplishes a specific function in the application.

Under: ,
Incremental Testing !!!

 Incremental Testing
A disciple method of testing the interfaces between unit tested programs as well as between system components.

 Types:
1. Top down Testing
2. Bottom up testing
1. Top Down
Begin testing from the top of the module hieracrchy and works down to the bottom using interior stubs to stimulate the lower interfacing modules or program.

 It tests the program of LLD. If LLD is not ready means it uses a dummy module called STUB.
LLD - Low Level Design

 2. Bottom up
* Begin testing from the bottom of the hierarchy and works up to the top.
* It requires the development of driver module which provide the test input call the module or program being testing and display the test output.

 It tests the program of HLD. If it is not ready means it uses a dummy module called DRIVER.
HLD - High Level Design

 If both dummy variables STUB and DRIVER are presented in the testing process then it is known as Sandwich Incremental Testing.

Under: , , , , , ,
Black Box Testing !!!

Black Box Testing:

Testing of a function without knowing internal structure of the program.Black box testing that is also known as functional or behavioral testing. This is testing a pre- release or release version of the program and trying various inputs looking for incorrect outputs or program crashes. It is one of the best ways to diagnose bugs before users discover them. Given the complexity of many software packages, much thought must be devoted to optimizing the testing process. Ideally only one test will be performed for each possible set of software conditions. Of all possible sets of software conditions there are groupings of conditions that if all tested would be testing the same code and reveal the same bugs (or hopefully the lack of bugs). These groups of conditions are considered an equivalent class or each is called an equivalent class partition.

Equivalence Partitioning:
A subset of data that is representation of a large class.
Ex: A program which exits credit limits with in a given range. (10000 – 15000)
Would have three equivalent classes.
• Less than 10000 (invalid)
• Between 10000 and 15000 ( valid)
• Above 15000. ( invalid)

Boundary Analysis:
Technique that consists of developing test cases and data that focus on the input and output boundaries of the given function.
• Low Boundary plus or minus one ( 9999 or 10001)
• On the boundary ( 10000 and 15000)
• Upper boundary plus or minus (14999 and 15001)

Error guessing
Based on the theory the test cases developed based on the experience of the Test engineer.
Ex : If one of the input is date, a test engineer must try Feb 29,2000 or 9/9/99.

Advantages of Black Box Testing
- Tester can be non-technical.
- This testing is most likely to find those bugs as the user would find.
- Testing helps to identify the vagueness and contradiction in functional specifications.
- Test cases can be designed as soon as the functional specifications are complete

Disadvantages of Black Box Testing
- Chances of having repetition of tests that are already done by programmer.
- The test inputs needs to be from large sample space.
- It is difficult to identify all possible inputs in limited testing time. So writing test cases is slow and difficult
Chances of having unidentified paths during this testing

Under: , , , , ,
Tools for Black box testing !!!

Many tool vendors have been producing tools for automated black box and automated white box testing for several years. The basic functional or regression testing tools capture the results of black box tests in a script format. Once captured, these scripts can be executed against future builds of an application to verify that new functionality hasn't disabled previous functionality.

Automation Testing tools used for Black box testing such as

* Rational Robot
* Silk Test
* OpenSTA
* QTP
* WinRunner
* LoadRunner

Under:
White Box Testing !!!

Testing of a function with knowing internal structure of the program.

White box testing involves looking at the structure of the code. When you know the internal structure of a product, tests can be conducted to ensure that the internal operations performed according to the specification. And all internal components have been adequately exercised. In other word WBT tends to involve the coverage of the specification in the code.

Code coverage is defined in six types as listed below.

• Segment coverage – Each segment of code b/w control structure is executed at least once.
• Branch Coverage or Node Testing – Each branch in the code is taken in each possible direction at least once.
• Compound Condition Coverage – When there are multiple conditions, you must test not only each direction but also each possible combinations of conditions, which is usually done by using a ‘Truth Table’
• Basis Path Testing – Each independent path through the code is taken in a pre-determined order. This point will further be discussed in other section.
• Data Flow Testing (DFT) – In this approach you track the specific variables through each possible calculation, thus defining the set of intermediate paths through the code i.e., those based on each piece of code chosen to be tracked. Even though the paths are considered independent, dependencies across multiple paths are not really tested for by this approach. DFT tends to reflect dependencies but it is mainly through sequences of data manipulation. This approach tends to uncover bugs like variables used but not initialize, or declared but not used, and so on.
• Path Testing – Path testing is where all possible paths through the code are defined and covered. This testing is extremely laborious and time consuming.
• Loop Testing – In addition top above measures, there are testing strategies based on loop testing. These strategies relate to testing single loops, concatenated loops, and nested loops. Loops are fairly simple to test unless dependencies exist among the loop or b/w a loop and the code it contains.

In WBT, we use the control structure of the procedural design to derive test cases. Using WBT methods a tester can derive the test cases that
• Guarantee that all independent paths within a module have been exercised at least once.
• Exercise all logical decisions on their true and false values.
• Execute all loops at their boundaries and within their operational bounds
• Exercise internal data structures to ensure their validity.

White box testing (WBT) is also called Structural or Glass box testing.
We do WBT because Black box testing is unlikely to uncover numerous sorts of defects in the program. These defects can be of the following nature:

 • Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed. Error tend to creep into our work when we design and implement functions, conditions or controls that are out of the program
• The logical flow of the program is sometimes counterintuitive, meaning that our unconscious assumptions about flow of control and data may lead to design errors that are uncovered only when path testing starts.
• Typographical errors are random, some of which will be uncovered by syntax checking mechanisms but others will go undetected until testing begins.

Talking theoretically, all we need to do in WBT is to define all logical paths, develop test cases to exercise them and evaluate results i.e. generate test cases to exercise the program logic exhaustively.

For this we need to know the program well i.e. We should know the specification and the code to be tested; related documents should be available too us .We must be able to tell the expected status of the program versus the actual status found at any point during the testing process.

Under: , , , ,
Tools for white box testing !!!

Few Test automation tool vendors offer white box testing tools which:

1) Provide run-time error and memory leak detection;
2) Record the exact amount of time the application spends in any given block of code for the purpose of finding inefficient code bottlenecks; and
3) Pinpoint areas of the application that have and have not been executed.

Under:
Testing Techniques !!!

Testing Techniques

 There are four types of testing techniques. They are
• White Box Testing
• Black Box Testing
• Incremental Testing
• Thread Testing

Comparison between Testing Levels and techniques:
 

Under: , , , ,
V Model !!!

CHAPTER 9

 Diagrammatic Representation of V Model:
In Brief:

 There are two phases in the V model development. They are
1. Verification
2. Validation.

 Verification Phases

 Requirements analysis

 In this phase, the requirements of the proposed system are collected by analyzing the needs of the user(s). This phase is concerned about establishing what the ideal system has to perform. However, it does not determine how the software will be designed or built. Usually, the users are interviewed and a document called the user requirements document is generated. The user requirements document will typically describe the system’s functional, physical, interface,
performance, data, security requirements etc as expected by the user.The users carefully review this document as this document would serve as the guideline for the system designers in the system design phase. The user acceptance tests(UAT) are designed in this phase.

 System Design

 System engineers analyze and understand the business of the proposed system by studying the user requirements document. They figure out possibilities and techniques by which the user requirements can be implemented. If any of the requirements is not feasible, the user is informed of the issue. A resolution is found and the user requirement document is edited accordingly.The software specification document which serves as a blueprint for the development phase is generated. This document contains the general system organization, menu structures, data structures etc. It may also hold examples business scenarios, sample windows, reports for the better understanding. Other technical documentation like entity diagrams, data dictionary will also be produced in this phase. The documents for system testing is prepared in this phase.

 Architecture Design

 This phase can also be called as high-level design (HLD). The baseline in selecting the architecture is that it should realize all the requirements within the given time, cost and resources. Software architecture is commonly represented as two-tier, three-tier or multi-tier models which typically comprises of the database layer, user-interface layer and the application layer. The modules and components representing each layer, their inter-relationships, subsystems, operating environment and interfaces are laid out in detail.The output of this phase is the high-level design document which typically consists of the list of modules, brief functionality of each module, their interface relationships, dependencies, database tables, architecture diagrams, technology details etc. The integration testing design is carried
out in this phase.

 Module Design

 This phase can also be called as low-level design (LLD). The designed system is broken up in to smaller units or modules and each of them is explained so that the programmer can start coding directly. The low level design document or program specifications will contain a detailed functional logic of the module, in pseudocode - database tables, with all elements, including their type and size - all interface details with complete API references- all dependency issues- error
message listings- complete input and outputs for a module. The unit test design is developed in this stage.

 Coding

 It is the process of writing, testing, and maintaining the source code of computer programs. The source code is written in a programming language.

 Validation Phases

 Unit Testing

 In the V-model of software development, unit testing implies the first stage of dynamic testing process.It involves analysis of the written code with the intention of eliminating errors. It also verifies that the codes are efficient and adheres to the adopted coding standards. Testing is usually white box. It is done using the Unit test design prepared during the module design phase. This may be carried out by software testers, software developers or both.

 Integration Testing

 In integration testing the separate modules will be tested together expose faults in the interfaces and in the interaction between integrated components. Testing is usually black box as the code is not directly checked for errors. It is done using the integration test design prepared during the architecture design phase. Integration testing is generally conducted by software testers.

 System Testing

 System testing will compare the system specifications against the actual system. The system test design derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. once all the modules are integrated several erros may rise.Testing done at this stage is called system test.

 User Acceptance Testing

 Acceptance Testing checks the system against the requirements of the user. It uses black box testing using real data, real people and real documents to ensure ease of use and functionality of systems. Users who understand the business functions run the tests as given in the acceptance test plans, including installation and Online help. Hardcopies of user documentation are also being reviewed for usability and accuracy. The testers formally document the results of each test, and provide error reports, correction requests to the developers.

Under: , , , , ,
User Acceptance Testing !!!

 User Acceptance Testing (UAT)

 A range of simple buildings should be simulated with the building simulation program undergoing testing automatically prior to all other tests. These should be considered an acceptance test intended to weed out unstable versions of the software that would not be fruitful to use for further debugging. Acceptance tests are often automated and may be provided to the programmers as a way to reduce the number of versions submitted for testing.

 1. Building the confidence of the client and the users is the role of acceptance test
Phase.
2. It is depend on the Business Scenario.

Under: , ,
System Testing !!!

 System Testing: (For SRS)

 * A system is the big component.
* System testing is aimed at revealing bugs that cannot be attributed to a component as such, to inconsistencies between components or planned interactions between components.
* Concern: issues, behaviors that can only be exposed by testing the entire integrated system (e.g., performance, security, recovery).

 1. Testing the entire application.
2. It’s a Black box testing.

 SRS - Software Requirement Specifications

Under: , ,
Integration testing !!!

Integration testing is a systematic technique for constructing the program structure while at the same time conducting tests to uncover errors associated with interfacing. The objective is to take unit tested components and build a program structure that has been dictated by design.

Usually, the following methods of Integration testing are followed:
1. Top-down Integration approach.
2. Bottom-up Integration approach.

 1. Top-Down Integration
Top-down integration testing is an incremental approach to construction of program structure. Modules are integrated by moving downward through the control hierarchy, beginning with the main control module. Modules subordinate to the main control module are incorporated into the structure in either a depth-first or breadth-first manner.

1. The Integration process is performed in a series of five steps:
2. The main control module is used as a test driver and stubs are substituted for all components directly subordinate to the main control module.
3. Depending on the integration approach selected subordinate stubs are replaced one at a time with actual components.
4. Tests are conducted as each component is integrated.
5. On completion of each set of tests, stub is replaced with the real component.
6. Regression testing may be conducted to ensure that new errors have not been introduced.

 2.Bottom-Up Integration
Bottom-up integration testing begins construction and testing with atomic modules (i.e. components at the lowest levels in the program structure). Because components are integrated from the bottom up, processing required for components subordinate to a given level is always available and the need for stubs is eliminated.

1. A Bottom-up integration strategy may be implemented with the following steps:
2. Low level components are combined into clusters that perform a specific software sub function.
3. A driver is written to coordinate test case input and output.
4. The cluster is tested.
Drivers are removed and clusters are combined moving upward in the program structure.

Under: , , ,
Unit Testing !!!

This is a typical scenario of Manual Unit Testing activity-
A Unit is allocated to a Programmer for programming. Programmer has to use ‘Functional Specifications’ document as input for his work. Programmer prepares ‘Program Specifications’ for his Unit from the Functional Specifications. Program Specifications describe the programming approach, coding tips for the Unit’s coding.
Using these ‘Program specifications’ as input, Programmer prepares ‘Unit Test Cases’ document for that particular Unit. A ‘Unit Test Cases Checklist’ may be used to check the completeness of Unit Test Cases document.
‘Program Specifications’ and ‘Unit Test Cases’ are reviewed and approved by Quality Assurance Analyst or by peer programmer.

 The programmer implements some functionality for the system to be developed. The same is tested by referring the unit test cases. While testing that functionality if any defects have been found, they are recorded using the defect logging tool whichever is applicable. The programmer fixes the bugs found and tests the same for any errors.

 Stubs and Drivers

 A software application is made up of a number of ‘Units’, where output of one ‘Unit’ goes as an ‘Input’ of another Unit. e.g. A ‘Sales Order Printing’ program takes a ‘Sales Order’ as an input, which is actually an output of ‘Sales Order Creation’ program.
Due to such interfaces, independent testing of a Unit becomes impossible. But that is what we want to do; we want to test a Unit in isolation! So here we use ‘Stub’ and ‘Driver.

 A ‘Driver’ is a piece of software that drives (invokes) the Unit being tested. A driver creates necessary ‘Inputs’ required for the Unit and then invokes the Unit.

 A Unit may reference another Unit in its logic. A ‘Stub’ takes place of such subordinate unit during the Unit Testing. A ‘Stub’ is a piece of software that works similar to a unit which is referenced by the Unit being tested, but it is much simpler that the actual unit. A Stub works as a ‘Stand-in’ for the subordinate unit and provides the minimum required behavior for that unit.
Programmer needs to create such ‘Drivers’ and ‘Stubs’ for carrying out Unit Testing.
Both the Driver and the Stub are kept at a minimum level of complexity, so that they do not induce any errors while testing the Unit in question.

 Example - For Unit Testing of ‘Sales Order Printing’ program, a ‘Driver’ program will have the code which will create Sales Order records using hard coded data and then call ‘Sales Order Printing’ program. Suppose this printing program uses another unit which calculates Sales discounts by some complex calculations. Then call to this unit will be replaced by a ‘Stub’, which will simply return fix discount data.

Under: , , , ,
Testing Levels !!!

Testing Levels:

 • Unit Testing.
• Integration Testing
• System Testing
• User Acceptance Testing


Under: , , , , ,
Management in Testing !!!

There are various Management types in testing. They are

Project Management
**********************
It is nothing but organizing, planning, scheduling, costing, monitoring review, report writing and presentation of the software project.
• Manual – Contains records of Name, requirements, schedule, cost etc.
• Automation – MPP – Microsoft Project Planner.

 Requirement Management
****************************
It is managing changes in evolving software in a cost effective manner.

 Manual tools

 SRS HLD LLD Code Testing
S1 HL1 LL1 C1
S2 * * C2 (if changes)

 SRS - Software Requirements Specification
HLD - High Level Design
LLD - Low Level Design

 Automation tools

 Requistic pro – Rational operation product
TBI Caliber – Technology Builder Incorporation - Organis
QSS Doors – Quality System Software - Actions

 Configuration Management
*****************************
CM is standards and procedures for managing changes in an evolving software product.
• Version control
• Changes in Progress.

 Manual Tools
CR ------------> CCB ---------> CI =======>
CR – Change request CCB – Change Control Board CI – Change Information
CM System
Version 1.0
Version 2.0
Shared
Version 3.0
Version 4.0
Draft
Managed File

Automation

 VSS – Visual Source Safe (Microsoft Product)
RCC – Rational Clear Case (Rational corporation product)
CVS – Concurrent Vision System.

Under: , , ,