Citation
Notes
Their definition of risk (because I like collecting definitions of risk):
Risks are factors that can, when present, adversely affect a project, unless project managers take appropriate countermeasures.
The framework is based on Keil et al. (1998) which proposes two dimensions (four quadrants) for the categorization of risk. They present Keil thusly:
Customer mandate (quadrant 1, or Q1, in Figure 1) focuses on risk factors relating to customers and users, including lack of top management commitment and inadequate user involvement; though important to the success of a project, such factors are often beyond the project manager’s control. Scope and requirements (Q2) focuses on risk factors associated with a project manager’s inability to judge a system’s scope. It also includes the risks associated with required functionality. Project managers should be able to control many of the risks associated with Q2. Execution (Q3) focuses on such risk factors as inadequate project staffing, inappropriate development methodology, failure to define roles and responsibilities, and poor project planning and control. […] Finally, environment (Q4) focuses on risk factors in both internal and external environments, including changes in organizational management, that might affect a project.
Their analysis regards the ways in which risks in each quadrant effect processes and products in projects. Process outcome is defined as “whether a project is completed on schedule and within budget.” Product outcome is “the success of the application produced through the development effort”. It is further explained as, “Producing a project on schedule and within budget is of little use if the resulting product lacks the features and functions users thought they were paying for.” Process outcome seems easily measured in relationship to original budget and schedule projections. Product outcome could be harder to express. It could be expressed as a relationship between features planned and features present, especially in terms of work estimates for those features. However, not all features are equal in most software development projects. Even more difficult is when completing specific features aren’t the measure of product success, but rather market performance of a shipped consumer IT product. Evaluating project performance remains a problem to investigate.
Their results:
For process outcome, we found only scope/requirements risk (Q2) and execution risk (Q3) were significant and no interactions among quadrants (p. 70). […] customer mandate (Q1), scope and requirements (Q2), and execution risks (Q3) have a significant relationship with product outcome. Environment risks (Q4) did not significantly affect product outcome (p. 71).
Risks associated with customer mandate (Q1) and environment (Q4) do not significantly affect process outcome. In other words, it may be possible to deliver software on schedule and within budget without much regard for risks associated with customer buy-in or the organizational environment. However, these risks may still affect project outcome (p. 71).
Which of the risks I’m investigating are in which quadrant?
Q1 includes interactions with the user. This could be significant to my work if we have to start considering the ways in which identifying with the product makes developers consider themselves to be users and behave like users.
Q2 includes feature creep, risks that affect feature creep. It may also include the game-related problem of “knowing when a game is fun enough to ship” in factors such as “undefined project success criteria” and “unclear system requirements.”
- Undefined project success criteria
- Conflicting system requirements
- Continually changing system requirements
- Continually changing project scope/objectives
- System requirements not adequately identified
- Unclear system requirements
- Incorrect system requirements
- Ill-defined project goals
- Users lack understanding of system capabilities and limitations
- Difficulty in defining the inputs and outputs of the system
Q3 includes appropriateness of staff to project, some of which may be attributable to identity, but mostly include skills. A partial list of Q3:
- Inadequately trained development team members
- Lack of commitment to the project among development team members
- Inexperienced team members
- Frequent conflicts among development team members
- Team members not familiar with the task(s) being automated
- Negative attitudes by development team
- Team members lack specialized skills required by the project