Mostly Asked Software Testing Methodology Interview Preparation Guide

Download PDF

Usage " To ensure that system performs correctly
" To ensure that correctness can be sustained for a considerable period of time.
" System can be tested for correctness through all phases of SDLC but incase of reliability the programs should be in place to make system operational.
Objectives " Successfully implementation of user requirements
" Correctness maintained over considerable period of time
" Processing of the application complies with the organization's policies and procedures.
" Secondary users needs are fulfilled -
Security officer
DBA
Internal auditors
Record retention
Comptroller
How to Use " Test conditions created
These test conditions are generalized ones, which becomes test cases as the SDLC progresses until system is fully operational.
Test conditions are more effective when created from user's requirements.
Test conditions if created from documents then if there are any error in the documents those will get incorporated in Test conditions and testing would not be able to find those errors.
Test conditions if created from other sources (other than documents) error trapping is effective.
" Functional Checklist created.
When to use " Every application should be Requirement tested
" Should start at Requirements phase and should progress till operations and maintenance phase.
" The method used to carry requirement testing and the extent of it is important.
Examples " Creating test matrix to prove that system requirements as documented are the requirements desired by the user.
" Creating checklist to verify that application complies to the organizational policies and procedures.

Web-Enabled application go further in these areas:

* Web-enabled application are meant to be stateless. HTTP was designed to be stateless. Each request from a Web-enabled application is meant to be atomic and not rely on any previouse requests. This has huge advantages for system architecture and datacenter provisioning. When requests are stateless, then any sserver can respond to the request and any request handler on any server may service the request.
* Web-enabled application are platform independent. The client application may be written for Windows, Macintosh, Linux, and any other platform that is capable of implementing the command protocol and network connection to the server.
* Web-enabled application expect the client application to provide presentation rendering and simple scripting capabilities. The client application is usually a browser, however, it may also be a dedication client application such as a retail cash register, a Windows-based data analysis tool, ot an electronic address book in your mobile phone.

The missing context in a Web-enabled application test automation means that software developers and QA technicians must manually script tests for each Web-enalbled application. Plus, they need to maintain the test scriots as the application changes. Web-enabled application test automation tools focus on making the scriot writing and maintenance tasks easier. The test automation tool offer these features:

* A friendly, graphical user interface to integrate the record, edit, and run-time script functions.
* A recorder that watches how an application is used and writes a test script for you.
* A playback utility that drives a Web-enalbed application by processing the test script and logging. The playback utility also provides the facility to play back several concurrently running copies of the same script to check the system for scalability and load testing.
* A report utility to show how the playback differed from the original recording. The differences may be slower or faster performance times, errors, and incomplete transactions.

The original intent for client/server applications was to separete presentation logic from business logic. In an ideal system design, the client was reponsible for presenting the user interface, command elements (drop-down menus, buttons, controls), displayed results information in a set of windows, charts, and dials. The client connected to a server to process functions and the server responded with data.

In a client/server environment the protocols are cleanly defined so that all the clients use the same protocols to communicate with the server.

The client-side frameworks to provide the same functionality of desktop application frameworks plus most of the needed communication code to issue commands to the server and the code needed to automatically update a client with new functions received from the server.The server-side frameworks provide code needed to received and handle requests from multiple clients, and code to connect to database for data persistence and remote information providers. Additionally, these framworks need to handle stateful transations and intermittent network connections. Stateful transactions require multiple steps to accomplish a task.

Client/server applications are normally transactional in nature and usually several interactions with the user are needed to finish a single request. For example, in a stock trading application the user begins a transaction by identifying themselves to the server, looking up an order code, and then submitting a request to the server, and receives and presents the results to the user. The client-side application normally knows something about the transaction - for example, the client-side application will normally store the user identification and a session code such as cookie value across the user's interaction with the server-based application. Users like it better when the client-side application knows about the transaction because each step in a request can be optimized in the client application. For example. in the stock trading example the client application could calculate a stock trade commission locally without having to communication with server.

Client/server application test automation provides the functionality of desktop application test automation plus these:

* Client/server applications operate in a network environment. The tests need to not only check for the function of an application, they need to test how the application handles slow or intermittent network performance.
* Automated test are ideal to determine the number of client applications a server is able to efficiently handle at any given time.
* The server is usually a middle tier between the client application and several data sources. Automated tests need to check the server for correct functionality while it communicates with the data source.

* If the database is linked to other database, are the links secure and working?
* If the database publishes to the Internet, is the data correct?
* When data is deployed, is it still accurate?
* Do the queries give accurate information to the reports?
* If thedatabase performs calculations, are the calculatons accurate?

* If the Web site publishes from inside the SQL Server straight to a Web page, is the data accurate and of the correct data type?
* If the SQL Server reads from a stored procedure to produce a Web page or if the stored procedure is changed, does the data on the page change?
* If you are using FrontPage or interDev is the data connection to your pages secure?
* Does the database have scheduled maintenance with a log so testers can set changes or errors?
* Can the tester check to see how back ups are being handled?
* Is the database secure?

The most common data errors are due to incorrect data entry, called data validity errors.

The turnaround time for responding to queries in a database must be short; therefor, query response time is essential for online transactions. The results from this test will help to identify problems, such as possible bottlenecks in the network, sspecific queries, the database structure, or the hardware.

Did a new build break an existing function? Repeat testing after changes for managing risk relate to product enhancement.

A regression test is performded when the tester wishes to see the progress of the testing processs by performing identical tests before and after a bug has been fixed. A regression test allows the tester to compare expeted test results with the actual results.

Regression testing's primary objective is to ensure that all bugfree features stay that way. In addition, bugs which have been fixed once should not turn up again in subsequent program versions.

Regression testing: After every software modification or before next release, we repeat all test cases to check if fixed bugs are not show up again and new and existing functions are all working correctly.

Regression testing is used to confirm that fixed bugs have, in fact, been fixed and that new bugs have not been introduced in the process, and that festures that were proven correctly functional are intact. Depending on the size of a project, cycles of regression testing may be perform once per milestone or once per build. Some bug regression testing may also be performed during each accceptance test cycle, forcusing on only the most important bugs. Regression tests can be automated.

CONDITIONS DURING WHICH REGRESSION TESTS MAY BE RUN Issu fixing cycle. Once the development team has fixed issues, a regression test can be run t ovalidate the fixes. Tests are based on the step-by-step test casess that were originally reported:

* If an issue is confirmeded as fixed, then the issue report status should be changed to Closed.
* If an issue is confirmed as fixed, but with side effects, then the issue report status should be changed to Closed. However, a new issue should be filed to report the side effect.
* If an issue is only partially fixed, then the issue report resolution should be changed back to Unfixed, along with comments outlining the oustanding problems

Open-status regression cycle. Periodic regression tests may be run on all open issue in the issue-tracking database. During this cycle, issue status is confirmed either the report is reproducible as is with no modification, the report is reproducible with additional comments or modifications, or the report is no longer reproducible

Closed-fixed regression cycle. In the final phase of testing, a full-regression test cycle should be run to confirm the status of all fixed-closed issues.

Feature regression cycle. Each time a new build is cut or is in the final phase of testing depending on the organizational procedure, a full-regression test cycle should be run to confirm that the proven correctly functional features are still working as expected.

Boundary tests are designed to check a program's response to extreme input values. Extreme output values are generated by the input values. It is important to check that a program handles input values and output results correctly at the lower and upper boundaries. Keep in mind that you can create extreme boundary results from non-extreme input values. It is essential to analyze how to generate extremes of both types. In addition. sometime you know that there is an intermediate variable involved in processing. If so, it is useful to determine how to drive that one through the extremes and special conditions such as zero or overflow condition.

What happens when the everyday user tries to access a control that is authorized only for adminstrators?

When you begin your stress testing, you will want to increase your capacity testing to make sure you are able to handle the increased load of data such as ASP pages and graphics. When you test the ASP pages, you may want to create a page similar to the original page that will simulate the same items on the ASP page and have it send the information to a test bed with a process that completes just a small data output. By doing this, you will have your processor still stressing the system but not taking up the bandwidth by sending the HTML code along the full path. This will not stress the entire code but will give you a basis from which to work. Dividing the requests per second by the total number of user or threads will determine the number of transactions per second. It will tell you at what point the server will start becoming less efficient at handling the load. Let's look at an example. Let's say your test with 50 users shows your server can handle 5 requests per seconf, with 100 users it is 10 requests per second, with 200 users it is 15 requests per second, and eventually with 300 users it is 20 requests per second. Your requests per second are continually climbing, so it seems that you are obtaining steadily improving performance. Let's look at the ratios:

05/50 = 0.1
10/100 = 0.1
15/200 = 0.075
20/300 = 0.073

From this example you can see that the performance of the server is becoming less and less efficient as the load grows. This in itself is not necessarily bad (as long as your pages are still returning within your target time frame). However, it can be a useful indicator during your optimization process and does give you some indication of how much leeway you have to handle expected peaks.

The primary goal of performance-testing is to develop effective enhancement strategies for maintaining acceptable system performance. Performance testing is a capacity analysis and planning process in which measurement data are used to predict when load levels will exhaust system resources.

The idea of stress testing is to find the breaking point in order to find bugs that will make that break potentially harmful. Load testing is merely testing at the highest transaction arrival rate in performance testing to see the resource contention, database locks etc..

Background " Pre determination of Error handling features is the basic difference between Automated and manual systems.
" Manual System: can deal with problems as they occur.
" Automated Systems: Must pre program error handling.
Usage " It determines the ability of applications system to process the incorrect transactions properly
" Errors encompass all unexpected conditions.
" In some system approx. 50% of programming effort will be devoted to handling error condition.
Objectives " Determine:
" Application system recognizes all expected error conditions.
" Accountability of processing errors has been assigned and procedures provide a high probability that errors will be properly corrected.
" During correction process reasonable control is maintained over errors.
How to Use " A group of knowledgeable people is required to anticipate what can go wrong in the application system.
" It is needed that all the application knowledgeable people assemble to integrate their knowledge of user area, auditing and error tracking.
" Then logical test error conditions should be created based on this assimilated information.
" The error handling testing technique should test -
Error
Processing of error
Control condition
Reentry of condition is proper or not.
" The iterative process should be used where first the errors in the system trapped should be corrected and then the corrected system should be re-tested to check the authenticity of application operation and to complete the error handling testing cycle.
" Tester should think negatively to trap errors.
" Testers should determine how the system should fail so that they can test to determine if the software can properly process the erroneous data.
When to use " Throughout SDLC
" Impact from errors should be identified and should be corrected to reduce the errors to acceptable level.
" Used to assist in error management process of system development and maintenance.
Examples " Create a set of erroneous transactions and enter them into the application system then find out whether the system is able to identify the problems.
" Using iterative testing enters transactions and trap errors. Correct them. Then enter transactions with errors, which were not present in the system earlier.

Background " One half of total system development effort is directly attributable to controls.
" Controls include:
Data validation
File integrity
Audit trail
Back up and recovery
Documentation.
Other aspects of system related to integrity
" Control is system within a system.
" Control looks at the totality of the system.
Usage " Control is a management tool to ensure that processing is performed in accordance to what management desire or intents of management.
Objectives " Accurate and complete data
" Authorized transactions
" Maintenance of adequate audit trail of information.
" Efficient, effective and economical process.
" Process meeting the needs of the user.
How to Use " To test controls risks must be identified.
" Develop risk matrix, which identifies the risks, controls; segment within application system in which control resides.
" Testers should have negative approach i.e. should determine or anticipate what can go wrong in the application system.
When to use " Should be tested with other system tests.
Examples " file reconciliation procedures work
" Manual controls in place.

Overwhelm the product for performance, reliability, and efficiency assessment; To find the breakpoint when system is failure; to increase load regressively to gather information for finding out maximum concurrent users.

Stress tests force programs to operate under limited resource conditions. The goal is to push the upper functional limits of a program to ensure that it can function correctly and handle error conditions gracefully. Examples of resources that may be artificially manipulated to create stressful conditions include memory, disk space, and network bandwidth. If other memory-oriented tests are also planned, they should be performed here as part of the stress test suite. Stress tests can be automated.

Breakpoint:

the capabilites and weakness of the product:

* High volunmes of data
* Device connections
* Long transation chains

Stress Test Environment:

As you set up your testing environment for a stress test, you need to make sure you can answer the following questions:

* Will my test be able to support all the users and still maintain performance?
* Will my test be able to simulate the number of transactions that pass through in a matter of hours?
* Will my test be able to uncover whether the system will break?
* Will my server crash if the load continues over and over?


The test should be set up so that you can simulate the load; for example:

* If you have a remote Web site you should be able to monitor up to four Web sites or URLs.
* There should be a way to monitor the load intervals.
* The load test should be able to simulate the SSL (Secure Server)
* The test should be able to simulate when a user submits the Form Data (GET method)
* The test should be set up to simulate and authentical the keyword verification.
* The test should be able to simulate up to six email or pager mail addresses and an alert should occur when there is a failure.

It is important to remember when stressing your Web site to give a certain number of users a page to stress test and give them a certain amount of time in which to run the test.

Some of the key data features that can help you measure this type of stress test, determine the load, and uncover bottlenecks in the system are:

* Amount of memory available and used
* The processor time used
* The number of requests per second
* The amount of time it takes ASP pages to be set up.
* Server timing errors.

Black Box Testing is testing without knowledge of the internal workings of the item being tested. The Outside world comes into contact with the test items, --only through the application interface ,,, an internal module interface, or the INPUT/OUTPUT description of a batch process. They check whether interface definitions are adhered to in all situation and whether the product conform to all fixed requirements. Test cases are created based on the task descriptions.

Black Box Testing assumes that the tester does not know anything about the application that is going to be tested. The tester needs to understand what the program should do, and this is achieved through the business requirements and meeting and talking with users.

Funcional tests: This type of tests will evaluate a specific operating condition using inputs and validating results. Functional tests are designed to test boundaries. A combination of correst and incorrect data should be used in this type of test.

HTML content checking tests makes a request to a Web page, parses the response for HTTP hyperlinks, requests hyperlinks from their associated host, and if the links returned successful or exceptional conditions. The downside is that the hyperlinks in a Web-enalbled application are dynamic and can change, depending on the user's actions. There is little way to know the context of the hyperlinks in a Web-enabled application. Just checking the links' validity is meaningless if not misleading. These tests were meant to test static Web sites, not Web-enabled application

Click stream Testing is to show which URLs the user clicked, The Web site's user activity by time period during the day, and other data otherwise found in the Web server logs. Popular choice for Click-Stream Testing statisticss include KeyNote Systems Internet weather report , WebTrends log analysis utility, and the NetMechanic monitoring service.

Disadvantage: Click-Stream Testing statistics reveal almost nothing about the user's ability to achieve their goals using the Web site. For example, a Web site may show a million page views, but 35% of the page views may simply e pages with the message "Found no search results," With Click-Stream Testing, there's no way to tell when user reach their goals.

Security measures protect Web systems from both internal and external threats. E-commerce concerns and the growing popularity of Web-based applications have made security testing increasingly relevant. Security tests determine whether a company's security policies have been properly implemented; they evaluate the functionality of existing systems, not whether the security policies that have been implemented are appropriate.

PRIMARY COMPONENTS REQUIRING SECURITY TESTING

* Application software
* Database
* Servers
* Client workstations
* Networks

Easy-of-use UI testing evaluates how intuitive a system is. Issues pertaining to navigation, usablility, commands, and accessibility are considered. User interface functionality testing examines how well a UI operates to specifications.

AREAS COVERED IN UI TESTING

* Usability
* Look and feel
* Navigation controls/navigation bar
* Instructional and technical information style
* Images
* Tables
* Navigation branching
* Accessibility

Online help tests check the accuracy of help contents, correctness of features in the help system, and functionality of the help system.

Compatibility and configuration testng is performanced to check that an application functions properly across various hardware and software environments. Often, the stragegy is to run the functional acceptance simple tests or a subset of the task-oriented functional tests on a range of software and hardware configurations. Sometimes, another strategy is to create a specific test that takes into account the error risks associated with configuration differences. For example, you might design an extensive series of tests to check for browser compatibility issues.
Software compatibility configurations include variances in OS versions, input/output (I/O) devices, extension, network software, concurrent applications, online services and firewalls. Hardwere configurations include variances in manufacturers, CPU types, RAM, graphic display cards, video capture cards, sound cards, monitors, network cards, and connection types(e.g. T1, DSL, modem, etc..).

These tests simulate the actions customers may take with a program. Real-World user-level testing often detects errors that are otherwise missed by formal test types.

The task-oriented functional test (TOFT) consists of positive test cases that are designed to verify program features by checking the task that each feature performs against specifications, user guides, requirements, and design documents. Usually, features are organized into list or test matrix format. Each feature is tested for:

* The validity of the task it performs with supported data conditions under supported operating conditions.
* The integrity od the task's end result
* The feature's integrity when used in conjunction with related features