This chapter provides an introduction for software quality management. Software quality assurance (SQA) is the concern of every software engineer to reduce cost and improve product time-to-market. A Software Quality Assurance Plan is not merely another name for a test plan, though test plans are included in an SQA plan. SQA activities are performed on every software project. Use of metrics is an important part of developing a strategy to improve the quality of both software processes and work products.
Quality Concepts
Variation control is the heart of quality control (software engineers strive to control the process applied, resources expended, and end product quality attributes).
Quality of design - refers to characteristics designers specify for the end product to be constructed
Quality of conformance - degree to which design specifications are followed in manufacturing the product
Quality control - series of inspections, reviews, and tests used to ensure conformance of a work product to its specifications
Quality assurance - consists of the auditing and reporting procedures used to provide management with data needed to make proactive decisions
Cost of Quality
Prevention costs - quality planning, formal technical reviews, test equipment, training
Appraisal costs - in-process and inter-process inspection, equipment calibration and maintenance, testing
Using formal technical reviews (walkthroughs or inspections) is an effective means for improving software quality.
Software Defects
Industry studies suggest that design activities introduce 50-65% of all defects or errors during the software process
Review techniques have been shown to be up to 75% effective in uncovering design flaws which ultimately reduces the cost of subsequent activities in the software process
Defect amplification models can be used to show that the benefits of detecting and removing defects from activities that occur early in the software process
Formal Technical Reviews
Involves 3 to 5 people (including reviewers)
Advance preparation (no more than 2 hours per person) required
Duration of review meeting should be less than 2 hours
Focus of review is on a discrete work product
Review leader organizes the review meeting at the producer's request
Reviewers ask questions that enable the producer to discover his or her own error (the product is under review not the producer)
Producer of the work product walks the reviewers through the product (alternative: in inspections a "reader" who is not the producer presents the work product)
Recorder writes down any significant issues raised during the review
Reviewers decide to accept or reject the work product and whether to require additional reviews of product or not
Formal Technical Review Guidelines
Review the product not the producer.
Seat an agenda and maintain it.
Limit rebuttal and debate.
Enunciate problem area, but don't attempt to solve every problem noted.
Take written notes.
Limit number of participants and insist on advance preparation.
Develop a checklist for each product that is likely to be reviewed.
Allocate resources and schedule time for all reviewers.
Conduct meaningful training for all reviewers.
Review your early reviews,
Sample Driven Reviews
Samples of all software engineering work products are reviewed to determine the most error-prone
Full FTR resources are focused on the likely error-prone work products based on sampling results
To be effective the sample driven review process must be driven by quantitative measures of the work products
Inspect a representative fraction of the content of each software work product (i) and record the number of faults (fi) found within (ai)
Develop a gross estimate of the number of faults within work product i by multiplying fi by 1/ai
Sort work products in descending order according to the gross estimate of the number of faults in each
Focus on available review resources on those work products with the highest estimated number of faults
Statistical Quality Assurance
Information about software defects is collected and categorized
Each defect is traced back to its cause
Using the Pareto principle (80% of the defects can be traced to 20% of the causes) isolate the "vital few" defect causes
Move to correct the problems that caused the defects in the "vital few"
Six Sigma Software Engineering
Define customer requirements, deliverables, and project goals via well-defined methods of customer communication.
Measure each existing process and its output to determine current quality performance (e.g., compute defect metrics)
Analyze defect metrics and determine viral few causes.
For an existing process that needs improvement
Improve process by eliminating the root causes for defects
Control future work to ensure that future work does not reintroduce causes of defects
If new processes are being developed
Design each new process to avoid root causes of defects and to meet customer requirements
Verify that the process model will avoid defects and meet customer requirements
Software Reliability
Defined as the probability of failure free operation of a computer program in a specified environment for a specified time period
Can be measured directly and estimated using historical and developmental data (unlike many other software quality factors)
Software reliability problems can usually be traced back to errors in design or implementation.
Measures of Reliability
Mean time between failure (MTBF) = MTTF + MTTR
MTTF = mean time to failure
MTTR = mean time to repair
Availability = [MTTF / (MTTF + MTTR)] x 100%
Software Safety
Defined as a software quality assurance activity that focuses on identifying potential hazards that may cause a software system to fail.
Early identification of software hazards allows developers to specify design features can eliminate or at least control the impact of potential hazards.
Software reliability involves determining the likelihood that a failure will occur, while software safety examines the ways in which failures may result in conditions that can lead to a mishap.
Mistake-Proofing Software
Poka-yoke devices are mechanisms that lead to the prevention of a potential quality problem before it occurs or to the rapid detection of a quality problem if one is introduced
Poka-yoke devices are simple, cheap, part of the engineering process, and are located near the process task where the mistakes occur
ISO 9000 Quality Standards
Quality assurance systems are defined as the organizational structure, responsibilities, procedures, processes, and resources for implementing quality management.
ISO 9000 describes the quality elements that must be present for a quality assurance system to be compliant with the standard, but it does not describe how an organization should implement these elements.
ISO 9001:2000 is the quality standard that contains 20 requirements that must be present in an effective software quality assurance system.
SQA Plan
Management section - describes the place of SQA in the structure of the organization
Documentation section - describes each work product produced as part of the software process
Standards, practices, and conventions section - lists all applicable standards/practices applied during the software process and any metrics to be collected as part of the software engineering work
Reviews and audits section - provides an overview of the approach used in the reviews and audits to be conducted during the project
Test section - references the test plan and procedure document and defines test record keeping requirements
Problem reporting and corrective action section - defines procedures for reporting, tracking, and resolving errors or defects, identifies organizational responsibilities for these activities
Other - tools, SQA methods, change control, record keeping, training, and risk management
