Manual & Automation Testing

Software Testing

Career Profile 

Overview - Software Test Engineer:

        A Software Test Engineer is involved in the testing of software and is generally a member of the software quality assurance team. Software QA Engineers may test software, but are not always expert Test Engineers.

            There are two main types of testing, which are white box testing and black box testing. The differences between the two are based on the assumptions and the view points taken by the testers while performing the tests. White box testing assumes an internal view of the software and black box testing basically assumes an external view of the software.

                    Testing is done at different phases of the software development life cycle (SDLC) and includes unit testing, integration testing, functional testing, system testing, acceptance testing (often performed by the internal or external client) and regression testing (which is often automated).(Click here to view jobs)

Becoming a Software Test Engineer

                Analytical thinking skills are a key to becoming a successful Test Engineer. Communication and documentation skills are also important. There are often intern-ships and entry level positions available in larger companies. Check out the entry level job boards for the best leads.

Software Test Engineer Career Path

               There are different level of Test Engineers. At an entry level, a testing engineer may set up and document simple test cases. As they become more experienced, Software Test Engineers may set up very complex test case scenarios, write test scripts and perform white box

CONTENTS:

• Introduction
• Testing Fundamentals
• Testing:

Introduction

            Testing is a process used to help identify the correctness, completeness and quality of developed computer software. With that in mind, testing can never completely establish the correctness of computer software.

              There are many approaches to software testing, but effective testing of complex products is essentially a process of investigation, not merely a matter of creating and following rote procedure. One definition of testing is "the process of questioning a product in order to evaluate it", where the "questions" are things the tester tries to do with the product, and the product answers with its behavior in reaction to the probing of the tester. Although most of the intellectual processes of testing are nearly identical to that of review or inspection, the word testing is connoted to mean the dynamic analysis of the product�putting the product through its paces.

                The quality of the application can and normally does vary widely from system to system but some of the common quality attributes include reliability, stability, portability, maintainability and usability. Refer to the ISO standard ISO 9126 for a more complete list of attributes and criteria.

             Testing helps is Verifying and Validating if the Software is working as it is intended to be working. Thins involves using Static and Dynamic methodologies to Test the application.

Because of the fallibility of its human designers and its own abstract, complex nature, software development must be accompanied by quality assurance activities. It is not unusual for developers to spend 40% of the total project time on testing. For life-critical software (e.g. flight control, reactor monitoring), testing can cost 3 to 5 times as much as all other activities combined. The destructive nature of testing requires that the developer discard preconceived notions of the correctness of his/her developed software.

• Software Testing Fundamentals:

Testing objectives include

1. Testing is a process of executing a program with the intent of finding an error.

2. A good test case is one that has a high probability of finding an as yet undiscovered error.

3. A successful test is one that uncovers an as yet undiscovered error.

              Testing should systematically uncover different classes of errors in a minimum amount of time and with a minimum amount of effort. A secondary benefit of testing is that it demonstrates that the software appears to be working as stated in the specifications. The data collected through testing can also provide an indication of the software's reliability and quality. But, testing cannot show the absence of defect -- it can only show that software defects are present.

   

Test analysis:

           This is primarily an exercise to study the software requirements and understand the scenarios that need to be tested. Study the requirements, talk to various stake holders in case of queries you get from studying the software requirements and come up with high level test requirements / test scenarios. This requires to understand the external and internal impact a certain requirement would have on the customer or other integrated products / modules.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

• Testing:

MANUAL & AUTOMATION TESTING

• Manual testing is the process of manually testing for defects. It requires a tester to play the role of an end user, and use most of all features of the application to ensure correct behavior. To ensure completeness of testing, the tester often follows a written testplan that leads them through a set of important testcase.
• Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process.

Software testing - what, why and how

Broadly, once the requirements are clear you can start testing. Depending on the size and complexity of the project or company, various methodologies, processes and strategies will be adopted. Fundamentally, software should be tested against well defined requirements.

Software Development Models:-

Introduction

There are various software development approaches defined and designed which are used/employed during development process of software, these approaches are also referred as software development process Models".Each process model follows a particular lifecycle in order to ensure success in process of software development.

• Waterfall Model

Waterfall approach was first Process Model to be introduced and followed widely in Software Engineering to ensure success of the project. In "The Waterfall" approach, the whole process of software development is divided into separate process phases.The phases in Waterfall model are: Requirement Specifications phase, Software Design, Implementation and Testing & Maintenance. All these phases are cascaded to each other so that second phase is started as and when defined set of goals are achieved for first phase and it is signed off, so the name "Waterfall Model". All the methods and processes undertaken in Waterfall Model are more visible.

• Iterative Model

An iterative lifecycle model does not attempt to start with a full specification of requirements. Instead, development begins by specifying and implementing just part of the software , which can then be reviewed in order to identify further requirements. This process is then repeated, producing a new version of the software for each cycle of the model. Consider an iterative lifecycle model which consists of repeating the following four phases in sequence: 

A Requirements phase, in which the requirements for the software are gathered and analyzed. Iteration should eventually result in a requirements phase that produces a complete and final specification of requirements. - A Design

phase, in which a software solution to meet the requirements is designed. This may be a new design, or an extension of an earlier design.

- An Implementation and test phase, when the software is coded, integrated and tested.

- A Review phase, in which the software is evaluated, the current requirements are reviewed, and changes and additions to requirements proposed.

For each cycle of the model, a decision has to be made as to whether the softwareproduced by the cycle will be discarded, or kept as a starting point for the next cycle (sometimes referred to as incremental prototyping). Eventually a point will be reached where the requirements are complete and the software can be delivered, or it becomes impossible to enhance the software as required, and a fresh start has to be made.

The iterative lifecycle model can be likened to producing software by successive approximation. Drawing an analogy with mathematical methods that use successive approximation to arrive at a final solution, the benefit of such methods depends on how rapidly they converge on a solution.

The key to successful use of an iterative software development lifecycle is rigorous validation of requirements, and verification (including testing) of each version of the software against those requirements within each cycle of the model. The first three phases of the example iterative model is in fact an abbreviated form of a sequential V or waterfall lifecycle model. Each cycle of the model produces software that requires testing at the unit level, for software integration, for system integration and for acceptance. As the software evolves through successive cycles, tests have to be repeated and extended to verify each version of the software.

• V-Model

The V-model is a software development which can be presumed to be the extension of the waterfall model. Instead of moving down in a linear way, the process steps are bent upwards after the coding phase, to form the typical V shape. The V-Model demonstrates the relationships between each phase of the development life cycle and its associated phase of testing.

 

Verification Phases

1. Requirements analysis:
   In this phase, the requirements of the proposed system are collected by analyzing the needs of the user(s). This phase is concerned about establishing what the ideal system has to perform. However, it does not determine how the software will be designed or built. Usually, the users are interviewed and a document called the user requirement documents is generated. The userrequirements document will typically describe the system’s functional, physical, interface, performance, data, security requirements etc as expected by the user. It is one which the business analysts use to communicate their understanding of the system back to the users. The users carefully review this document as this document would serve as the guideline for the system designers in the system design phase. The user acceptance tests are designed in this phase.

System Design:

System engineers analyze and understand the business of the proposed system by studying the user requirements document. They figure out possibilities and techniques by which the user requirements can be implemented. If any of the requirements are not feasible, the user is informed of the issue. A resolution is found and the user requirement document is edited accordingly.

The software specification document which serves as a blueprint for the development phase is generated. This document contains the general system organization, menu structures, data structures etc. It may also hold example business scenarios, sample windows, reports for the better understanding. Other technical documentation like entity diagrams, data dictionary will also be produced in this phase. The documents for system testing is prepared in this phase.

Architecture Design:This phase can also be called as high-level design. The baseline in selecting the architecture is that it should realize all which typically consists of the list of modules, brief functionality of each module, their interface relationships, dependencies, database tables, architecture diagrams, technology details etc. The integration testing design is carried out in this phase.

Module Design:This phase can also be called as low-level design. The designed system is broken up in to smaller units or modules and each of them is explained so that the programmer can start coding directly  The low level design document or program specifications will contain a detailed functional logic of the module, in pseudo code - database tables, with all elements, including their type and size - all interface details with complete API references- all dependency issues- error message listings- complete input and outputs for a module. The unit test design is developed in this stage.

• Spiral Model

History

The spiral model was defined by Barry Boehm in his 1988 article A Spiral Model of software development,and Enhancement. This model was not the first model to discuss iterative development, but it was the first model to explain why the iteration matters. As originally envisioned, the iterations were typically 6 months to 2 years long. Each phase starts with a design goal and ends with the client (who may be internal) reviewing the progress thus far. Analysis and engineering efforts are applied at each phase of the project, with an eye toward the end goal of the project.

The Spiral Model

The spiral model, also known as the spiral lifecycle model, is a systems developmentmethod (SDM) used in information technology (IT). This model of developmentcombines the features of the prototyping model and the waterfall model. The spiral model is intended for large, expensive, and complicated projects.

The steps in the spiral model can be generalized as follows:

1. The new system requirements are defined in as much detail as possible. This usually involves interviewing a number of users representing all the external or internal users and other aspects of the existing system.
2. A preliminary design is created for the new system.
3. A first prototype of the new system is constructed from the preliminary design. This is usually a scaled-down system, and represents an approximation of the characteristics of the final product.
4. A second prototype is evolved by a fourfold procedure: (1) evaluating the firstprototype in terms of its strengths, weaknesses, and risks; (2) defining the requirements of the second prototype; (3) planning and designing the second prototype; (4) constructing and testing the second prototype.
5. At the customer's option, the entire project can be aborted if the risk is deemed too great. Risk factors might involve development cost overruns, operating-cost miscalculation, or any other factor that could, in the customer's judgment, result in a less-than-satisfactory final product.
6. The existing prototype is evaluated in the same manner as was the previous prototype, and, if necessary, another prototype is developed from it according to the fourfold procedure outlined above.
7. The preceding steps are iterated until the customer is satisfied that the refined prototype represents the final product desired.
8. The final system is constructed, based on the refined prototype.
9. The final system is thoroughly evaluated and tested. Routine maintenance is carried out on a continuing basis to prevent large-scale failures and to minimize downtime.

Applications

For a typical shrink-wrap application, the spiral model might mean that you have a rough-cut of user elements (without the polished / pretty graphics) as an operable application, add features in phases, and, at some point, add the final graphics. The spiral model is used most often in large projects. For smaller projects, the concept of agile software development is becoming a viable alternative. The US military has adopted the spiral model for its Future Combat Systems program.

• The Big Bang Model

Let's review the observational evidence:

• Distance/velocity relationship: distant galaxies are moving away from us, with speeds which increase linearly with distance
• Chemistry: the universe is almost entirely hydrogen and helium, in a mixture of roughly 12 H atoms to 1 He atom

• Cosmic microwave Background: no matter where we look in the universe, we see radio waves which look like those radiated by a blackbody at about 2.7 degrees above absolute zero. There are tiny (one part in 10,000) variations in the brightness of this radiation on scales of a degree or so

Is there any way to tie all these pieces of data together? Yes! One model which can explain them all is called the Big Bang model. The name was coined by a scientist who didn't like the theory and tried to make it sound silly.

• RAD Model

 

   

 Introduction

RAD is a linear sequential software development process that emphasis an extremely short development cycle using a component based construction approach. If the requirements are well understood and defines, and the project scope is constraint, the RAD process enables a development team to create a fully functional system with in very short time period.

System Development Life Cycle Phases 

SDLC Models (ppt)             

     Software testing - that's the buzz word in the IT industry today. I come across so many youngsters who are excited about career prospects in this field. They are excited about the number of job opportunities for skilled testers and more importantly, the high salaries. Many are willing to quit jobs in other fields

Software testing training institutes are flourishing in cities like Bangalore and Pune in India. They promise to teach Winrunner, Quicktest Pro (QTP) and Loadrunner in a month. These tools appeal to the youngsters as they are in demand and people with these skills are well paid.

What they fail to realise is that there is more to testing than the tools.

So, here's an attempt to break the hype and discuss real time situations and problems in software testing.

What is software testing

It means exactly that - testing the software before making it available for use. Compare that with what you would do before you buy a car. Do you test drive it? Do you go ahead and buy it if you think there is a problem?

No. You expect it to work to your satisfaction before you decide to buy. Same goes with software.

Why should software be tested

Software should be thoroughly tested because in some cases if the software malfunctions, it can be life threatening. Plane crashes to financial losses in millions of dollars - they have happenned when software misbehaved.

Check out the Software Hall of Shame to realise the importance of software testing.

How can software be tested

Just like when you go looking to buy a new car, you know exactly what you want which means you have made a list of requirements that the car you decide to buy should satisfy.

Similarly, the most important thing in software testing is the requirements that the software is expected to satisfy. A calculator application is expected to provide accurate calculations when you add, subtract, multiply or divide numbers.

 
• Planning

An important task in creating a software program is extracting the requirements or requirements analysis ^[1]. Customers typically have an abstract idea of what they want as an end result, but not what software should do. Skilled and experienced software engineers recognize incomplete, ambiguous, or even contradictory requirements at this point. Frequently demonstrating live code may help reduce the risk that the requirements are incorrect.

Once the general requirements are gathered from the client, an analysis of the scope of the development should be determined and clearly stated. This is often called a scope document.

Certain functionality may be out of scope of the project as a function of cost or as a result of unclear requirements at the start of development. If the development is done externally, this document can be considered a legal document so that if there are ever disputes, any ambiguity of what was promised to the client can be clarified.

• Requirement Analysis

In this software requirements analysis, firstly analysis the requirement for the proposed system. To understand the nature of the program to built, the system engineer must understand the information domain for the software, as well as required functions, performance and the interfacing. From the available information the system engineer develops a list of the actors use cases and system level requirement for the project. With the help of key user the list of use case and requirement is reviewed. Refined and updated in an iterative fashion until the user is satisfied that it represents the essence of the proposed system

• Design phase

The primary objective of the design phase is to create a design that satisfies the agreed application requirements. In the design phase the SDLC process continues to move from the "what" questions of the analysis phase to the "how" questions. 

The requirements prototype that was developed earlier during the analysis phase is gradually improved and extended to include all the specified functions of the application. 

Also, the planning of the system documentation process should be started.

For description of the analysis phase activities, please click on the top of the bar

• Integration & Testing

Objective of Integration Testing

The objective of Integration Test is to ensure that various system parts of related components and programs work properly together.

It includes:

• Integrates Components.
• Components with Components Tests
• Hardware Interfaces Tests
• External Software Interfaces Tests
• Functional Requirements Tests
• Sub-Assembly Tests

• System Testing & Implementation :

Testing Implementation

1.Testing Client/Server Systems

Concerns:

• Organizational readiness.

• Client installation.

• Security.

• Client data.

• Client/server standards

2.Testing Client/Server Systems 

• Assess readiness.

◦ Motivation.

◦ Investment.

◦ Client/server skills.

◦ User education.

◦ Culture.

◦ Client/server support staff.

◦ Client/server aids/tools.

◦ Software development process maturity.

• Assess key components.

◦ Client installations are done correctly.

◦ Adequate security is provided for the client/server system.

◦ Client data is adequately protected.

◦ Client/server standards are in place and working.

• Test the system.

1.Testing Rapid Application Development

Concerns:

• Test information from iteration.

• Test information about components.

• Test information about performance.

• Record test information.

2.Testing Rapid Application Development 

• Test planning iterations.

• Test subsequent planning iterations.

• Test final planning iteration.

1.Testing Web-based Systems 

Concerns:

• Browser compatibility.

• Functional correctness.

• Integration.

• Usability.

• Security.

• Performance.

• Verification of code.

2.Testing Web-based Systems 

• Select web-based risks to include in the test plan.

Key areas of concern:

◦ Security risk.

◦ Performance.

◦ Correctness.

◦ Compatibility.

◦ Reliability.

◦ Data integrity.

◦ Usability.

◦ Recoverability.

• Select web-based tests.

• Select web-based test tools.

• Test web-based systems.

 

3.Testing Web-based Systems 

Security risk:

• External intrusion.

• Protection of secured transactions.

• Viruses.

• Access control.

• Authorization levels

4.Testing Web-based Systems 

Performance:

• Concurrency.

• Stress.

• Throughput.

5.Testing Web-based Systems 

Correctness:

• Functionality.

• Calculations.

• Navigation.

6.Testing Web-based Systems 

Compatibility:

• Operating systems/platforms.

• Browser

7.Testing Web-based Systems 

Reliability:

• Server and system availability.

• Consistently correct results.

8.Testing Web-based Systems 

Data integrity:

• Ensuring only correct data is accepted.

• Ensuring data stays in a correct state.

9.Testing Web-based Systems 

Usability:

• Ensuring the application is easy to use and understand.

• Ensuring that users know how to interpret and use the information delivered from the application.

• Ensuring that navigation is clear and correct.

10.Testing Web-based Systems 

Recoverability:

• Lost connections.

◦ Timeouts.

◦ Dropped lines.

• Client system crashes.

• Server system crashes or other application problems

11.Testing Web-based Systems

Web-based tests:

• Unit or component test.

• Integration test.

• System test.

• User acceptance test (business process validation).

• Performance test.

• Load/stres test.

• Regression test.

• Usability test.

• Compatibility test.

1.Testing Off-the-shelf Software 

Concerns:

• Task/item missing.

• Software fails to perform.

• Extra features.

• Does not need business needs.

• Does not meet operational needs.

• Does not meet people needs.

2.Testing Off-the-shelf Software 

• Test business fit.

◦ Completeness of needs specification.

◦ Critical success factor test.

• Test operational fit.

◦ Compatibility with your hardware, operating system, and other software packages.

◦ Integrating the software into your business system work flow.

◦ Demonstrating the software in operation.

• Test people fit.

• Acceptance test software processing.

◦ Create functional test conditions.

◦ Create structural test conditions.

1.Testing in a Multi-platform Environment 

Concerns:

• The platforms in the test lab will not be representative of the platform in the real world.

• The software will be expected to work on platforms not included in the test labs.

• The supporting software on various platforms is not comprehensive.

2.Testing in a Multi-platform Environment 

• Design platform configuration concerns.

• List needed platform configurations.

• Assess test room configuration.

• List software structure platform effects.

• List interfaces platform effects.

• Execute tests.

1.Testing Security

• Identify potential perpetrators.

• Identify potential points of penetration.

• Create a penetration point matrix.

• Identify high risks points of penetration.

• Execute security risks.

◦ Evaluate the adequacy of security controls at identified points.

◦ Determine if penetration can occur at identified point(s).

◦ Determine if penetration has actually occurred at this point.

2.Testing Security : Functional vulnerabilities

• Poor controls over manual handling of input/output data.

• Weak or nonexistent physical access controls.

• Computer and terminal operational procedures.

• Weaknesses in the business test process.

• Weaknesses in the control of computer programs.

• Weaknesses in operating system access and integrity.

• Poor controls over access through impersonation.

• Weaknesses in media control.

3.Testing Security : Location of vulnerabilities

• Computer data and report preparation facilities.

• Computer operations.

• Non-IT areas.

• Central processors.

• Programming offices.

• Magnetic media storage facilities.

• On-line terminal systems.

• On-line data preparation and output report handling areas.

• On-line operations.

SOFTWARE TESTING LIFE CYCLE