QUALITY PROCESS ASSESSMENTS
Consulting Assignment Objectives
(Printable Page)

The primary objective of the Quality Process Assessment (QPA) is to assist the client in identifying the areas of opportunity for reducing time to deliver, reducing cost to deliver, and improving quality at delivery for their software initiatives. The emphasis for the Quality Process Assessment report is on the solution set and how to implement it in a timely manner.

The focus of the effort will be on the processes for creating and validating the detailed requirements and on creating a sufficient test infrastructure – test methods, processes, tools, staffing – to ensure high quality systems are delivered in a timely manner.
The level of testing should meet or exceed industry requirements, as appropriate, and demonstrate a degree of due diligence appropriate to the criticality of the functions and features. This level of testing should be quantifiably verified.


The recommendations should leverage off of the existing infrastructure to the degree possible.


The rollout plan should reflect and be sensitive to the fact that key projects may already be well under way. The recommendations must factor in the current status of the various projects a nd provide for the implementation with a minimal amount of disruption of on-going activities. However, at no time should the necessary level of quality be compromised.

Topics To Be Addressed

A typical Quality Process Assessment Report will contain the following sections:

1. Introduction

2. Executive Summary

3. The Business Case For Quality
3.1 Time to Market / Cost to Market
3.2 Litigation Exposure
3.3 Quality Moves Market Share
3.4 Production Defect Costs
3.5 Complying With Regulatory Requirements
[Note: there are over a dozen major areas considered as applicable to the specific client]

4. Technology Based Recommendations
4.1 Requirements Quality Process Overview
4.2 Creating Complete, Consistent. Correct, Unambiguous Specifications
4.2.1 Creating and Testing Specifications For Completeness
4.2.2 Process Modeling
4.2.3 Writing Unambiguous Specifications
4.2.4 Customer/User Reviews of the Specifications
4.2.5 Using Test Cases to Verify Requirements
4.2.6 Test considerations and the Sequence of Writing Requirments
4.2.7 Requirements Management Automated Support
4.3 Test Process Overview
4.3.1 Rigorous Black Box / Specification Based Test Case Design Criteria
4.3.2 Rigorous White Box / Code Based Test Case Design Criteria
4.4 Test Case Build and Execution
4.4.1 Building Test Data Bases
4.4.2 Automating the Test Scripts
4.5 Test Results Validation
4.6 Cover Monitors
4.6.1 Functional Coverage Monitors
4.6.2 Code Coverage Monitors
4.7 Maintaining Regression Test Libraries
4.8 Test Case Management
4.9 Test Status Tracking
4.9.1 Tracking Ambiguities
4.9.2 Tracking Defects
4.9.3 Defect Removal Efficiency
4.9.4 Residual Defect Rate
4.9.5 Product Confidence / Component Confidence
4.10 Testing User Manuals, Training Materials, and Marketing Materials
4.11 Testing By Project Type
4.11.1 Testing Third Party Packages
4.11.2 Testing Technology Conversions
4.11.3 Testing Maintenance Changes
4.11.4 Testing Rewrites, Re-Engineering, ERP
4.12 Levels of Testing and Continuous Integration
4.12.1 Unit Test
4.12.2 Integration Test / Component Test
4.12.3 System Test / Product Test
4.12.4 Customer Acceptance Test / Beta Test
4.12.5 Inter-Product Test
4.12.6 Overlapping Entry/Exit Criteria
4.12.7 Continuous Integration
4.13 Testing and Agile Methodologies
4.14 Testing and Web Based Applications
[Note: other technology based and/or application specific considerations might be addressed here - e.g., testing complex models in super computers.]

5. Management Based Quality Considerations
5.1 The Impact of Planning on Quality and Vice Versa
5.1.1 The Five Variables of Planning
5.1.2 Linear Slip
5.1.3 Requirements Management Tools and Planning Support
5.2 Iterative Life Cycle Versus Waterfall Approach
5.3 Quantitative Test Completion Criteria and Go / No-Go Decisions
5.4 Testing and CMM/CMMi
5.5 Quality and Contract Management / Outsourcing
5.6 Quality and Technology Transfer
5.7 The Quality Process and Staffing
5.7.1 Testers As Peers
5.7.2 Testers' Skills Profiles
5.7.3 Sizing the Testing Resources
5.7.4 Organizing the Testers

6. Quality Strategy Rollout
6.1 Managing the Cultural Impact
6.2 Training
6.3 Tools Rollout
6.4 Mentoring
6.5 Project Priorities

7. Summary of Recommendations

8. Appendices
[Note: this section would contain various white papers which would go into more detail on some of the specific topics from the report.]

Performing a Quality Process Assessment generally takes from four to six weeks elapsed time and four to eight person weeks depending on the scope of the effort. In general the activities break out as follows:

Activity One - Understand the Client's Objectives and the Status of any Quality Improvement Initiatives (8 to 16 hours)

Step 1 -
Obtain a General Understanding of the Client's Specific Objectives for This Effort and a High Level Understanding of the Current Status

Determining the optimal path forward requires an understanding of the current perceived status of the current infrastructure and the projects currently underway.

Step 2 -
Review Existing Assessments

Using the contextual understanding developed from Step 1, review any recent assessment reports done by the client and/or third parties, for example, CMM assessments.

Discuss with the staff their current perceptions as to the strengths and weaknesses of these assessments given the experience in the interim since the reports were written.

Activity Two - Interview Management and Staff (40 hours)

Step 1 -
Identify the People to Interview and Schedule Their Time

Step 2 -
Conduct the Interviews and Gather Relevant Documentation

Generally the interviews are done one on one and last for one to one and a half hours.

During the interviews various documents are identified such as descriptions of existing methods and procedures or examples of specifications, etc., that will prove useful in understanding the current status.

Activity Three - Review Gathered Documents (40 hours)

Read all of the documents identifying opportunites to build on what already exists.

Activity Four - Write the Assessment Report (40 to 60 hours)

Identify the topics to be addressed in the report.
Review the suggested topical outline with the client.
Write the report.
The overall recommendations should have a number of key characteristics:

- They should reduce the time to delivery by supporting a strong degree of concurrency between the development and test activities and between the component level testing and system level testing.

- The approach should minimize scrap and rework by providing timely feedback/testing. This should also result in defect avoidance, not just defect detection.

- There should be a strong emphasis on verifying the quality of the requirements since studies consistently show that the majority of defects have their root cause in poor requirements. This also supports the fact that the earlier a defect is detected the cheaper it is to resolve.

- The emphasis must be on practical solutions whose deployment should "pay as you go".

Activity Five - Apply Portions of the Recommended Processes to Some Existing Specifications (40 hours)

To further quantify the impact of the recommendations it might be appropriate to select some sample functions and apply some the recommended processes to them. For example, one process that we have found to be universally effective is our ambiguity review process. The effectiveness of this could be demonstrated by applying it to some sample specifications. This has the added value of providing concrete examples from the project to include in the recommendations report.

Once the identified ambiguities have been resolved then tests could be developed from the specification using the recommended test design approaches. These tests could then be compared to the tests previously developed by the client for those functions.

Activity Six - Present Recommendations to Management and Staff (24 to 32 hours)

Step 1-
Create a Power Point presentation of the QPA report.

Step 2 -
Present the assessment and recommendations in one or more sessions as appropriate.

Activity Seven - Revise The Report As Necessary (8 to 16 hours)

Based on the feedback, update and finalize the recommendations report.

Activity Eight - Provide On Going Support And Mentoring As Needed

Assist in the implementation of the recommendations by providing training, mentoring, and consulting as needed.

Consulting Rates:

Richard Bender - $250/hour plus travel and living expenses.

Other Staff As Needed - $150 to $175/hour. Specific people will be identified for approval by the client as the needs, if any, are identified.

(Bender-QPA-services.pdf)

HOME/ RETURN TO MENU SELECTION