Software Risk Management

Chapter 04: Risk Identification

Course Instructor: Amir Imam

History-Based Methods (Caper Jones’s list)
• Jones present software metrics as a major risk in software projects.
• Many things can go wrong in metrics.
• We measure the wrong factors, and miss the right ones.
• When we decide to trust an ill-designed metrics system, we run risks.
• This caution is age-old. Deming had said: “Do not manage a company by visible
numbers alone.” But Jones puts it as a risk.
• A typical risk identifier may not connect risks to metrics, but after Jones’s proposal,
a new perspective has arisen.
• Cross-checking your project against this risk list could be a “life saver.”
• Part of Jones’ risk list is given in Appendix A.
History-Based Methods (Rex Black’s risk list)
• That “risk lists” can have multiple uses is beautifully demonstrated in Rex Black’s
“Critical Testing Processes.”
1. Functionality
2. Load, capacity, and volume
3. Reliability/stability
4. Stress, error handling, and recovery
5. Date and time handling
6. Operations and maintenance
7. Data quality
8. Performance
9. Localization
10. Compatibility
11. Security/privacy
12. Installation/migration
13. Documentation
14. Interfaces
• These are also quality attributes of the product, and play a role in managing
product quality from a standpoint not related to risk thinking.
• These are also “failure modes” of the product according to FMEA (failure modes
effects analysis) practice and become the anchor points for reliability analysis and
preventive actions.
History-Based Methods (SEI’s & Top N risk list)
• The risk list published by SEI, under the name of risk taxonomy, is a very useful
tool for risk identification.
• It covers three categories of risk: product engineering risks, development
environment risks, and program constraints risk.
• Risks identified by three researchers Brian A. Will, Will captures risks that seem to
be present in all development projects.
• From requirements to office space, some well-known, oft-repeated risks have
been identified by Will.
• Barry Boehm’s top ten risk list can help in quick risk identification.
• His “gold plating” risk has gained great popularity.
• Chester Summer’s top ten list also meets the same need.
• We find “communications” and “concurrent engineering” in the list.
• Such top ten risk lists define what we already know, but help us to remember
them during risk identification.
 Risk Checklist, Taxonomy-Based
Questionnaire
• Experience makes preparation of risk checklists possible. The checklist is
constructed with tell-tale symptoms, clues, or simply names of known risks.
• The organization’s collective experience in risk identification is used to design risk
checklists. We can have special risk check lists for each phase, and for each
process.
• SEI proposes a (TBQ) as a formal and structured way of identifying risks.
• Risks are viewed through windows of known risk types, making identification of
risks faster and more economical.
• Then TBQ method uses searching questions for each attribute. Say “Stability”

Requirement phase risk attributes Taxonomy-Based Questionnaire

a. Stability SEI TBQ for Requirement Stability
b. Completeness [1] Are the requirements stable? (No) (1a) What is the effect on the system?
c. Clarity Quality, Functionality, Schedule, Integration, Design, Testing
d. Validity [2] Are the external interfaces changing?
e. Feasibility SEI TBQ for Requirement Completeness
[3] Are there any TBDs in the specifications? [4] Are there requirements you
f. Precedent
know should be in the specification but are not present?
g. Scale
(Yes) (4a) Will you be able to get these requirements into the system?
[5] Does the customer have unwritten requirements/expectations?
(Yes) (5a) Is there a way to capture these requirements?
[6] Are the external interfaces completely defined?
Type II: Project-Specific Risk Identification
Phase I: Context Setting
• Project-specific risk identification is a context-based identification process.
• By setting a context, we might make the mistake of excluding some risks
that lie outside the set context, risks which might be significant.
• We rely on Type I risk identification to make a context-free, general scan
of risks.
• Project team may set themes for various risk identification exercises.
▫ Product risk identification
▫ Design risk identification
▫ Project risk identification
▫ Business risk identification
▫ Testing risk identification
▫ Bug fixing risk identification
• The scope of risk identification will be defined in such cases before the
meeting takes place.
 Type II: Project-Specific Risk Identification
Phase II: Data Gathering
• Risk identifier is aware of risks whose symptoms existed but were ignored.
• After the bad experience he becomes wiser and is willing to look for symptoms -
the risk indicators.
• Firstly he records the presence of risk indicators known to him which are a
fraction of the full list.
• Only the familiar are illuminated as his vision is narrow, limited and selective.
• The risk identifier looks for methods that will capture risks which are unfamiliar
and escaped earlier
• Then he realizes that what is unknown to him may be known to others. Each
person has a unique and different history.
• In a project environment, the identifier calls for a brainstorming session.
• The participants are the stakeholders and are motivated to look at risks.
• The session evokes multiple perspectives and illuminates hitherto untraced
areas. More risks are identified as a result.
 Type II: Project-Specific Risk Identification
• To improve the yield of the risk identification process, the team must go through
a preparatory phase before the actual meeting.
• The biggest input to risk identification is pertinent information. Here is an
example of a list that shows the range of inputs which may be of use in risk
identification.
1. Corporate goals 11. Management reviews 24. Competitor analysis
2. Project objectives 12. Inspection and test reports 25. Threat modeling
3. Assumptions 13. Risk checklist 26. Business intelligence
4. Constraints 14. List of known risks 27. Metrics data mining
5. Customer requirements 15. Risk taxonomy 28. Internal quality audit reports
6. Customer feedback 16. Risk classification system 29. Certification body audit reports
7. Benchmark studies 17. Risk attributes 30. Finance audit reports
8. Metrics data 18. Risk history 31. Management review findings
9. Process capability baselines 19. Risk database
10. Internal quality audit findings 20. Growth plans
21. Investor’s expectations
22. Market research
23. Customer behavior analysis
 Type II: Project-Specific Risk Identification
Phase III: Risk Discovery
• To succeed in risk discovery, the organization must become risk sensitive.
• Risk discovery needs the right environment. It requires vision and
empowerment.
• Uninterested and lethargic People without vision cannot discover risks.
• Without a risk management policy in place, there is no motivation to discover
risks.
• It encompasses the whole organization, from the individual level to the
corporate level.
• The manager supports risk identification; to detect vulnerabilities in the project
and set up defenses, and to ensure the longevity of project plans, the manager
initiates mitigation and contingency plans to manage risks.
• The mapping of indictor to risk events.
• Use of Risk Survey, Models, Metrics Data, Scenarios, etc. ..
 Type II: Project-Specific Risk Identification
Phase IV: Assigning Attributes
• Once the risk identifier has zeroed in on the risk event, he has to describe the
event in a concise manner.
• He also creates a unique identity and reference for the risk.

Referential Data Primary Evaluation Data Secondary data and risk attributes
1. Strategic business unit (SBU) name 8. Risk consequence description 12. Origin (internal or external)
2. Project name 9. Risk probability (p) (scale: 0 to 10) 13. Type (business or technical)
3. Risk identifier team members 10. Risk impact (i) (scale: 0 to 10) 14. Most affected process (Requirements, Data
4. Risk identification date 11. Risk exposure (p) x (i) Encryption Standard,
5. Risk ID Coding, Testing, Training, Quality Management,
6. Risk name Project Management,
7. Risk event description Financial Management)
15. Most affected result (EFF, SCH, Q, PERF)
16. Risk trigger
17. Expected time of occurrence (existing, next M,
Q, Yr)
18. Risk visibility (VL, L, N, H, VH)
19. Risk nature (hazard, constraint, nominal, trivial)
20. Risk owner
 Type II: Project-Specific Risk Identification
Phase V: Validation
• Validation of risks means removing the following defects from the risk list:
▫ Wrong description
▫ Wrong name
▫ Wrong classification
▫ Irrelevance (not relevant to the project)
▫ Ambiguity
▫ Repetition
▫ Blurred differences (lack of uniqueness)
• Validation should be done by the same team that identified risks. Others may not
be able to get the right context from wrong statements to cross-check the risk
statements.
• Burying Trivial Differences - The Funnel Model, risks be reduced
• Naming the Risks
• Risk Statements Are Problem Definitions
 Type II: Project-Specific Risk Identification
Phase V: Validation - The Trouble with Validation
• Validating risk statements calls for great self-discipline and motivation.
• It is not a “favorite” activity.
• Those who created risk statements do not like to see them perish.
• In some cases, once risks are entered into a risk-tracker tool, quick changes are
not possible and the tool ensures that risk statements survive for a long time.
• The statements are “records” that cannot be tampered with.
• Identification is seen as a value-adding process, whereas validation is seen as a
fault-finding process that diminishes hard-earned satisfaction.
Phase VI: List
• The output of risk identification is a list of validated risks.
• The risks will be tabulated along with identified attributes. This list is the basis for
further analysis.
 Levels in Identification
• Process-Level Risk Identification
 List all the requirements (or features or functions) and identify risks you may meet while
realizing the requirements as software products.
 Evaluate the risk exposure number for each requirement.
 To enhance your perception, estimate the FP count corresponding to each requirement.
• Project-Level Risk Identification
▫ Organization goals associated risks
▫ Performance targets associated risks
▫ Project objectives associated risks
• Enterprise-Level Risk Identification
▫ List your organizations strengths, weaknesses, opportunities, and threats.
▫ Define your growth goals and marketing strategies. Then fill in this data:
 SBU name
 Growth goal
 Marketing strategy
 Strengths
 Weaknesses
 Opportunities
 Threats
 Implementing Risk Identification Processes
• Risk identification is a complex process. We can search the entire risk
environment or look at chosen sections.
• The span and depth can vary between risk identification exercises.
• The dilemma faced by every project team is to choose between general risks
(most of which anyway will be escalated) and project-specific risks.
• However, a quick scan at all levels is recommended, followed by a detailed study
of the project-specific environment.
• The team must do type I risk identification first, rapidly, and then go in for type II
risk identification.
• The risk list must contain risks identified by both methods.