Product Quality Measures Interview Preparation Guide

Download PDF

(Quality ultimately is measured in terms of customer satisfaction.)
Surveyed before product delivery and after product delivery (and on-going on a periodic basis, using standard questionnaires)
Number of system enhancement requests per year
Number of maintenance fix requests per year
User friendliness: call volume to customer service hotline
User friendliness: training time per new user
Number of product recalls or fix releases (software vendors)
Number of production re-runs (in-house information systems groups)

Turnaround time for defect fixes, by level of severity
Time for minor vs. major enhancements; actual vs. planned elapsed time

Defects found after product delivery per function point
Defects found after product delivery per LOC
Pre-delivery defects: annual post-delivery defects
Defects per function point of the system modifications

McCabe's cyclomatic complexity counts across the system
Halstead’s measure
Card's design complexity measures
Predicted defects and maintenance costs, based on complexity measures

Business losses per defect that occurs during operation
Business interruption costs; costs of work-arounds
Lost sales and lost goodwill
Litigation costs resulting from defects
Annual maintenance cost (per function point)
Annual operating cost (per function point)
Measurable damage to your boss's career

Re-work effort (hours, as a percentage of the original coding hours)
Re-worked LOC (source lines of code, as a percentage of the total delivered LOC)
Re-worked software components (as a percentage of the total delivered components)

Normalized per function point (or per LOC)
At product delivery (first 3 months or first year of operation)
Ongoing (per year of operation)
By level of severity
By category or cause, e.g.: requirements defect, design defect, code defect, documentation/on-line help defect, defect introduced by fixes, etc.

Ratio of maintenance fixes (to repair the system and bring it into compliance with specifications), vs. enhancement requests (requests by users to enhance or change functionality)

Number of post-release defects (found by clients in field operation), categorized by level of severity
Ratio of defects found internally prior to release (via inspections and testing), as a percentage of all defects
All defects include defects found internally plus externally (by customers) in the first year after product delivery

Breadth of functional coverage
Percentage of paths, branches or conditions that were actually tested
Percentage by criticality level: perceived level of risk of paths
The ratio of the number of detected faults to the number of predicted faults.

Costs of reviews, inspections and preventive measures
Costs of test planning and preparation
Costs of test execution, defect tracking, version and change control
Costs of diagnostics, debugging and fixing
Costs of tools and tool support
Costs of test case library maintenance
Costs of testing & QA education associated with the product
Costs of monitoring and oversight by the QA organization (if separate from the development and test organizations)

Availability (percentage of time a system is available, versus the time the system is needed to be available)
Mean time between failure (MTBF)
Mean time to repair (MTTR)
Reliability ratio (MTBF / MTTR)
Number of product recalls or fix releases
Number of production re-runs as a ratio of production runs

QA refers to the planned and systematic way of monitoring the quality of process which is followed to produce a quality product. QA tracks the outcomes and adjusts the process to meet the expectation.

Concern with the quality of the product. QC finds the defects and suggests improvements. The process set by QA is implemented by QC. The QC is the responsibility of the tester.

Software Testing is the process of ensuring that product which is developed by the developer meets the user requirement. The motive to perform testing is to find the bugs and make sure that they get fixed.

Good time to start the QA is from the beginning of the project startup. This will lead to plan the process which will make sure that product coming out meets the customer quality expectation. QA also plays a major role in the communication between teams. It gives time to step up the testing environment. The testing phase starts after the test plans are written, reviewed and approved.

Destructive testing includes methods where material is broken down to evaluate the mechanical properties, such as strength, toughness and hardness.
For example, finding the quality of a weld is good enough to withstand extreme pressure and also to verify the properties of a material.

Benefits of Destructive Testing (DT):
✰ Verifies properties of a material
✰ Determines quality of welds
✰ Helps you to reduce failures, accidents and costs
✰ Ensures compliance with regulations

✰ The subset of software which helps in performing the testing of application.
✰ Testware is term given to combination of all utilities and application software that required for testing a software package.
✰ Testware are required to plan, design, and execute tests. It contains documents, scripts, inputs, expected results, set-up and additional software or utilities used in testing.

Testware is special because it has:
1) Different purpose
2) Different metrics for quality and
3) Different users

✰ Retesting is done to verify defects fixes where as regression is perform to check if the defect fix have not impacted other functionality that was working fine before doing changes in the code.
✰ Retesting is planned testing based on the defect fixes listed where as regression is not be always specific to any defect fix. Also regression can be executed for some modules or all modules.
✰ Retesting concern with executing those test cases that are failed earlier whereas regression concern with executing test cases that was passed in earlier builds.
✰ Retesting has higher priority over regression, but in some case retesting and regression testing are carried out in parallel.

✰ Verification is Static Testing where as Validations is Dynamic Testing.
✰ Verification takes place before validation.
✰ Verification evaluates plans, documents, requirements and specifications, where as Validation evaluates product.
✰ Verification inputs are checklist, issues list, walk-through and inspection, where as in Validation testing of actual product.
✰ Verification output is set of documents, plans, specifications and requirement documents where as in Validation actual product is output.

✰ Sanity testing is performed when new build is released after fixing bugs where as smoke testing is performed to check the major functionalities of the application.
✰ Sanity is performed by the tester or the developer but smoke testing can be performed by the tester or developer.
✰ Smoke testing is performed earlier where as sanity is performed after the smoke testing.
✰ Sanity testing is narrow and deep approach of testing and smoke testing is focused testing based on major functionalities.

✰ Purchasing the license of tool (QTP, selenium, QC, LR).
✰ Lack of skilled Tester to run the tool.
✰ Expectation that automated tests will find a lot of new defects.
✰ Maintenance of automated tests.
✰ Technical problems of tools.

✰ QA team is responsible for monitoring the process to be carried out for development.
✰ Responsibilities of QA team are planning testing execution process.
✰ QA Lead creates the time tables and agrees on a Quality Assurance plan for the product.
✰ QA team communicated QA process to the team members.
✰ QA team ensures traceability of test cases to requirements.