Causal analysis and resolution is one of the center pillars of Software Process Improvement. Without tracing defects to their root cause there is no opportunity
to reduce (or eliminate) those defects. Key to Causal Analysis is Product Quality Assurance (Software Quality Assurance) which establishes process descriptions, standards, and procedures
and then checks for noncompliance issues against those documented standards and procedures. Identification of defects provides initial input to Casual analysis as does an ongoing Software Metrics (measurements) program, implemented as Measurement and Analysis (MA) under CMMi.
Under CMMi for Software Development the following 3 Process Areas form the backbone for a Software Process Improvement framework:-
Process and Product Quality Assurance (PPQA)
Measurement and Analysis (MA)
Causal Analysis and Resolution (CAR)
Causal analysis is, however, a science and an art. There is an old saying that is applicable to Casual Analysis and it goes:-
It ain't what you don't know that hurts you, it is what you know that ain't so.
Basically Causal Analysis is the result of deep analytical (and possibly experimental) research which is often initiated by hunches or experience.
Casual Analysis can be broken into two phases:-
Macro problem identification.
Micro problem identification.
Macro problem identification uses tools such as Cause-and-effect (fishbone) diagrams, this tool will scope the cause to a number of possibilities.
This tool, and other similar tools, are only identifying possibilities and typically look at:-
The 4 M's - Methods, Machines, Materials, Manpower.
The 4 P's - Place, Procedure, People, Policies .
The 4 S's - Surroundings, Suppliers, Systems, Skills.
Once possibilities have been identified (and this is usually based on experience and hunches) then the Micro problem identification can begin.
Micro problem identification attempts to pin point the cause, or at least one cause, to a level that can be addressed.
For this stage of problem identification a Hypothesis or assumption is formed and then tested.
For example Software Supportability can be measured by mean time to repair MTTR, one cause on high MTTR would be code that is difficult to read. Since code complexity itself
can be measured (see Cyclomatic complexity) it is possible to test the assumption (lower code complexity will yield lower MTTR) of the cause for high Support costs.
In this way coding standards could be introduced and their effects measured to verify a cause for high MTTR.
In the above example experiments could also be conducted using sample code (containing defects). In this way two sets of programmers would be shown either structured (less complicated) code or
unstructured (more complicated) code containing the same defect. Then the time taken to find the defect could be determined. This information could prove the assumption being tested, to identify a cause of the problem.
Such experiments may or may not be practical in all situations but they underline the points of:-
Proving an assumption via experimentation, to pin point a cause of a given defect.
Identifying the correct metric (measurement) that quantifies the presence of the defect.
Only examining one variable, on the cause of the defect, at a time.
It is noted that all CMMI Process Areas are subjected to Root Cause Analysis, in order to achieve capability rating of level 5.
Also Root Cause Analysis is the foundation of any Software Process Improvement framework (not just CMMi).
|
Causal Analysis and Resolution (CAR)
A Support Process Area at Maturity Level 5
Purpose
The purpose of Causal Analysis and Resolution (CAR) is to identify causes of defects and other problems and take action to prevent them from occurring in the future.
Introductory Notes
The Causal Analysis and Resolution process area involves the following:
Identifying and analyzing causes of defects and other problems
Taking specific actions to remove the causes and prevent the occurrence of those types of defects and problems in the future
Causal analysis and resolution improves quality and productivity by preventing the introduction of defects into a product. Reliance on detecting defects after they have been introduced is not cost effective. It is more effective to prevent defects from being introduced by integrating causal analysis and resolution activities into each phase of the project.
Since defects and problems may have been previously encountered on other projects or in earlier phases or tasks of the current project, causal analysis and resolution activities are a mechanism for communicating lessons learned among projects.
The types of defects and other problems encountered are analyzed to identify any trends. Based on an understanding of the defined process and how it is implemented, the root causes of the defects and the future implications of the defects are determined.
Causal analysis may also be performed on problems unrelated to defects. For example, causal analysis may be used to improve quality attributes such as cycle time. Improvement proposals, simulations, dynamic systems models, engineering analyses, new business directives, or other items may initiate such analysis.
When it is impractical to perform causal analysis on all defects, defect targets are selected by tradeoffs on estimated investments and estimated returns of quality, productivity and cycle time.
A measurement process should already be in place. The defined measures can be used, though in some instances new measures may be needed to analyze the effects of the process change.
Refer to the Measurement and Analysis process area for more information about establishing objectives for measurement and analysis, specifying the measures and analyses to be performed, obtaining and analyzing measures, and reporting results.
Causal Analysis and Resolution activities provide a mechanism for projects to evaluate their processes at the local level and look for improvements that can be implemented.
When improvements are judged to be effective, the information is extended to the organizational level.
Refer to the Organizational Innovation and Deployment process area for more information about improving organizational level processes through proposed improvements and action proposals.
The informative material in this process area is written with the assumption that the specific practices are applied to a quantitatively managed process. The specific practices of this process area may be applicable, but with reduced value, if this assumption is not met.
See the definitions of “stable process” and “common cause of process variation” in the glossary.
Related Process Areas.
Refer to the Quantitative Project Management process area for more information about the analysis of process performance and the creation of process capability measures for selected project processes.
Refer to the Organizational Innovation and Deployment process area for more information about the selection and deployment of improvements to organizational processes and technologies.
Refer to the Measurement and Analysis process area for more information about establishing objectives for measurement and analysis, specifying the measures and analyses to be performed, obtaining and analyzing measures, and reporting results.
Specific Practices by Goal
SG 1 Determine Causes of Defects
Root causes of defects and other problems are systematically determined.
A root cause is a source of a defect such that, if it is removed, the defect is decreased or removed.
SP 1.1 Select Defect Data for Analysis
Select the defects and other problems for analysis.
Typical Work Products
Defect and problem data selected for further analysis
Subpractice 1: Gather relevant defect or problem data.
Examples of relevant defect data may include the following:
Defects reported by the customer
Defects found in peer reviews
Defects found in testing
Examples of relevant problem data may include the following:
Project management problem reports requiring corrective action
Process capability problems
Process duration measurements
Earned value measurements by process (e.g., cost performance index)
Resource throughput, utilization, or response time measurements
Refer to the Verification process area for more information about work product verification.
Refer to the Quantitative Project Management process area for more information about statistical management.
Subpractice 2: Determine which defects and other problems will be analyzed further.
When determining which defects to analyze further, consider the impact of the defects, their frequency of occurrence, the similarity between defects, the cost of analysis, the time and resources needed, the safety considerations, etc.
Examples of methods for selecting defects and other problems include the following:
Pareto analysis
Histograms
Process capability analysis
SP 1.2 Analyze Causes
Perform causal analysis of selected defects and other problems and propose actions to address them.
The purpose of this analysis is to develop solutions to the identified problems by analyzing the relevant data and producing action proposals for implementation.
Typical Work Products
Action proposal
Subpractice 1: Conduct causal analysis with the people who are responsible for performing the task.
Causal analysis is performed, typically in meetings, with those people who have an understanding of the selected defect or problem under study. The people who have the best understanding of the selected defect are typically those responsible for performing the task.
Examples of when to perform causal analysis include the following:
When a stable process does not meet its specified quality and process-performance objectives
During the task, if and when problems warrant a causal analysis meeting
When a work product exhibits an unexpected deviation from its requirements
Refer to the Quantitative Project Management process area for more information about achieving the project’s quality and process-performance objectives.
Subpractice 2: Analyze selected defects and other problems to determine their root causes.
Depending on the type and number of defects, it may make sense to first group the defects before identifying their root causes.
Examples of methods to determine root causes include the following:
Cause-and-effect (fishbone) diagrams
Check sheets
Subpractice 3: Group the selected defects and other problems based on their root causes.
Examples of cause groups, or categories, include the following:
Inadequate training
Breakdown of communications
Not accounting for all details of a task
Making mistakes in manual procedures (e.g., typing)
Process deficiency
Subpractice 4: Propose and document actions that need to be taken to prevent the future occurrence of similar defects or other problems.
Examples of proposed actions include changes to the following:
The process in question
Training
Tools
Methods
Communications
Work products
Examples of specific actions include the following:
Providing training in common problems and techniques for preventing them
Changing a process so that error-prone steps do not occur
Automating all or part of a process
Reordering process activities
Adding process steps to prevent defects, such as task kickoff meetings to review common defects and actions to prevent them
An action proposal usually documents the following:
Originator of the action proposal
Description of the problem
Description of the defect cause
Defect cause category
Phase when the problem was introduced
Phase when the defect was identified
Description of the action proposal
Action proposal category
SG 2 Address Causes of Defects
Root causes of defects and other problems are systematically addressed to prevent their future occurrence.
Projects operating according to a well-defined process will systematically analyze the operation where problems still occur and implement process changes to eliminate root causes of selected problems.
SP 2.1 Implement the Action Proposals
Implement the selected action proposals that were developed in causal analysis.
Action proposals describe the tasks necessary to remove the root causes of the analyzed defects or problems and avoid their reoccurrence.
Only changes that prove to be of value should be considered for broad implementation.
Typical Work Products
Action proposals selected for implementation
Improvement proposals
Subpractice 1: Analyze the action proposals and determine their priorities.
Criteria for prioritizing action proposals include the following:
Implications of not addressing the defects
Cost to implement process improvements to prevent the defects
Expected impact on quality
Subpractice 2: Select the action proposals that will be implemented.
Subpractice 3: Create action items for implementing the action proposals.
Examples of information provided in an action item include the following:
Person responsible for implementing it
Description of the areas affected by it
People who are to be kept informed of its status
Next date that status will be reviewed
Rationale for key decisions
Description of implementation actions
Time and cost for identifying the defect and correcting it
Estimated cost of not fixing the problem
To implement the action proposals, the following tasks must be done:
Make assignments
Coordinate the persons doing the work
Review the results
Track the action items to closure
Experiments may be conducted for particularly complex changes.
Examples of experiments include the following:
Using a temporarily modified process
Using a new tool
Action items may be assigned to members of the causal analysis team, members of the project team, or other members of the organization.
Subpractice 4: Identify and remove similar defects that may exist in other processes and work products.
Subpractice 5: Identify and document improvement proposals for the organization’s set of standard processes.
Refer to the Organizational Innovation and Deployment process area for more information about the selection and deployment of improvement proposals for the organization’s set of standard processes.
SP 2.2 Evaluate the Effect of Changes
Evaluate the effect of changes on process performance.
Refer to the Quantitative Project Management process area for more information about analyzing process performance and creating process capability measures for selected processes.
Once the changed process is deployed across the project, the effect of the changes must be checked to gather evidence that the process change has corrected the problem and improved performance.
Typical Work Products
Measures of performance and performance change
Subpractices 1: Measure the change in the performance of the project's defined process as appropriate.
This subpractice determines whether the selected change has positively influenced the process performance and by how much.
An example of a change in the performance of the project’s defined design process would be the change in the defect density of the design documentation, as statistically measured through peer reviews before and after the improvement has been made. On a statistical process control chart, this would be represented by a change in the mean.
Subpractices 2: Measure the capability of the project's defined process as appropriate.
This subpractice determines whether the selected change has positively influenced the ability of the process to meet its quality and process-performance objectives, as determined by relevant stakeholders.
An example of a change in the capability of the project’s defined design process would be a change in the ability of the process to stay within its process-specification boundaries. This can be statistically measured by calculating the range of the defect density of design documentation, as collected in peer reviews before and after the improvement has been made. On a statistical process control chart, this would be represented by lowered control limits.
SP 2.3 Record Data
Record causal analysis and resolution data for use across the project and organization.
Data are recorded so that other projects and organizations can make appropriate process changes and achieve similar results.
Record the following:
Data on defects and other problems that were analyzed
Rationale for decisions
Action proposals from causal analysis meetings
Action items resulting from action proposals
Cost of the analysis and resolution activities
Measures of changes to the performance of the defined process resulting from resolutions
Typical Work Products
Causal analysis and resolution records
Generic Practices by Goal
GG 1 Achieve Specific Goals
The process supports and enables achievement of the specific goals of the process area by transforming identifiable input work products to produce identifiable output work products.
GP 1.1 Perform Specific Practices
Perform the specific practices of the causal analysis and resolution process to develop work products and provide services to achieve the specific goals of the process area.
GG 2 Institutionalize a Managed Process
The process is institutionalized as a managed process.
GP 2.1 Establish an Organizational Policy
Establish and maintain an organizational policy for planning and performing the causal analysis and resolution process.
Elaboration:
This policy establishes organizational expectations for identifying and systematically addressing root causes of defects and other problems.
GP 2.2 Plan the Process
Establish and maintain the plan for performing the causal analysis and resolution process.
Elaboration:
This plan for performing the causal analysis and resolution process can be included in (or referenced by) the project plan, which is described in the Project Planning process area. This plan differs from the action proposals and associated action items described in several specific practices in this process area. The plan called for in this generic practice would address the project’s overall causal analysis and resolution process (perhaps tailored from a standard process maintained by the organization). In contrast, the process action proposals and associated action items address the activities needed to remove a specific root cause under study.
GP 2.3 Provide Resources
Provide adequate resources for performing the causal analysis and resolution process, developing the work products, and providing the services of the process.
Elaboration:
Examples of resources provided include the following tools:
Database systems
Process modeling tools
Statistical analysis packages
Tools, methods, and analysis techniques (e.g., Ishikawa or fishbone diagram, Pareto analysis, histograms, process capability studies, or control charts)
GP 2.4 Assign Responsibility
Assign responsibility and authority for performing the process, developing the work products, and providing the services of the causal analysis and resolution process.
GP 2.5 Train People
Train the people performing or supporting the causal analysis and resolution process as needed.
Elaboration:
Examples of training topics include the following:
Quality management methods (e.g., root cause analysis)
GP 2.6 Manage Configurations
Place designated work products of the causal analysis and resolution process under appropriate levels of control.
Elaboration:
Examples of work products placed under control include the following:
Action proposals
Action proposals selected for implementation
Causal analysis and resolution records
GP 2.7 Identify and Involve Relevant Stakeholders
Identify and involve the relevant stakeholders of the causal analysis and resolution process as planned.
Elaboration:
Examples of activities for stakeholder involvement include the following:
Conducting causal analysis
Assessing the action proposals
GP 2.8 Monitor and Control the Process
Monitor and control the causal analysis and resolution process against the plan for performing the process and take appropriate corrective action.
Elaboration:
Examples of measures and work products used in monitoring and controlling include the following:
Number of root causes removed
Change in quality or process performance per instance of the causal analysis and resolution process
Schedule of activities for implementing a selected action proposal
GP 2.9 Objectively Evaluate Adherence
Objectively evaluate adherence of the causal analysis and resolution process against its process description, standards, and procedures, and address noncompliance.
Elaboration:
Examples of activities reviewed include the following:
Determining causes of defects
Addressing causes of defects
Examples of work products reviewed include the following:
Action proposals selected for implementation
Causal analysis and resolution records
GP 2.10 Review Status with Higher Level Management
Review the activities, status, and results of the causal analysis and resolution process with higher level management and resolve issues.
GG 3 Institutionalize a Defined Process
The process is institutionalized as a defined process.
This generic goal's appearance here reflects its location in the continuous representation.
GP 3.1 Establish a Defined Process
Establish and maintain the description of a defined causal analysis and resolution process.
GP 3.2 Collect Improvement Information
Collect work products, measures, measurement results, and improvement information derived from planning and performing the causal analysis and resolution process to support the future use and improvement of the organization’s processes and process assets.
Elaboration:
Examples of work products, measures, measurement results, and improvement information include the following:
Action proposals
Number of action proposals that are open and for how long
Action proposal status reports
GG 4 Institutionalize a Quantitatively Managed Process
The process is institutionalized as a quantitatively managed process.
GP 4.1 Establish Quantitative Objectives for the Process
Establish and maintain quantitative objectives for the causal analysis and resolution process, which address quality and process performance, based on customer needs and business objectives.
GP 4.2 Stabilize Subprocess Performance
Stabilize the performance of one or more subprocesses to determine the ability of the causal analysis and resolution process to achieve the established quantitative quality and process-performance objectives.
GG 5 Institutionalize an Optimizing Process
The process is institutionalized as an optimizing process.
GP 5.1 Ensure Continuous Process Improvement
Ensure continuous improvement of the causal analysis and resolution process in fulfilling the relevant business objectives of the organization.
GP 5.2 Correct Root Causes of Problems
Identify and correct the root causes of defects and other problems in the causal analysis and resolution process.
|
|