Schmidt et al. 2001: Identifying Software Project Risks

Citation

Notes

Keil et al. (1998) and Schmidt et al. (2001) involve the same group of four researchers working with data from the same underlying international Delphi study, the former paper targeted at practitioners while the latter at researchers. Schmidt et al (2001) hoped “to develop an authoritative list of risk factors” (p. 10). After asking experienced, successful project managers from the United States, Finland, and Hong Kong to identify and define risk factors, a combined list was further modified and validated by that panel. The regional sub-panels then selected and ranked the factors they decided were most important. While this allowed for regional comparison, the larger list was shared: 53 factors divided among fourteen categories. Keil et al. (1998) reported only a subset of 11 risk factors held in common across the international participants, but also proposed a four quadrant categorization of risks. This framework included axes of “perceived relative importance of risk” and “perceived level of control”. The resulting quadrants were identified as Customer Mandate, Scope and Requirements, Environment, and Execution. 

This was the process:

I am interested in the findings on multiple levels: I’m obviously interested in the specific risk factors identified as they apply to my focused area of research, but I’m also interested in this as an exploration of cultural differences in project management practices and perceptions.

The Risks

The authors divided their lists of risks into categories. The following partial list contains material applicable to my work.

1. Corporate Environment

1.1 A climate of change in the business and organizational environment that creates instability in the project.
1.4 Unstable corporate environment: Competitive pressures radically alter user requirements, sometimes making the entire project obsolete.

3. Relationship Management

3.4 Failure to identify all stakeholders: Tunnel vision leads project management to ignore some key stakeholders in the project, affecting requirements definition, implementation, etc.
3.7 Lack of appropriate experience of the user representatives: Users assigned who lack necessary knowledge of the application or the organization

4. Project Management

4.1 Not managing change properly: Each project needs a process to manage change so that scope and budget are controlled. Scope creep is a function of ineffective change management and of not clearly identifying what equals success.

5. Scope

5.2 Changing scope/objectives: Business changes or reorganizes part way through the project
5.3 Scope creep: Not thoroughly defining the scope of the new system and the requirements before starting, consequently not understanding the true work effort, skill sets and technology required to complete the project.
5.4 Project not based on sound business case: Users and developers ignore business requirements, develop system for the sake of technology.

6. Requirements

6.1 Lack of frozen requirements. Because the needs of the users change, the requirements change. Consequently the system will never be moved into production because none of the requirements are ever completed. Alternatively, freezing a subset of the functionality and delivering allows for the completion of the system and update releases as required.

10. Personnel

10.1 Lack of required knowledge/skills in the project personnel: For example, technology, business knowledge, and experience.

11. Staffing

11.1 Insufficient/inappropriate staffing: Not enough people or people with wrong skills/insufficient skills assigned to project, regardless of availability.
11.4 Lack of available skilled personnel: People with the right skills are not available when you need them.

The risks in the corporate environment section is interesting because game studios are said to have a different culture than other software development organizations. They’re out of scope, worth noting.

Relationship management items involve end users and other stakeholders. It is possible that when you consider yourself part of your target market (as I suspect we will see when team members identify with the product they’re building) you may 1) consider fewer sorts of stakeholders, or 2) communicate with stakeholders less frequently.

The project management category includes a note regarding scope creep, even though it’s also mentioned in the scope section and implied in the requirements section. Here, it’s said to be a function of poor change management. I’ll suggest that you can still get scope creep with excellent change management, should you start out with a deficient spec and fix it competently while the project is running.

The scope category isn’t all applicable to my work. Poor definition of one’s scope and allowing too many cooks in the scope kitchen are not in my personal scope. We see overlap, too. Changes in the corporate environment have their own category, and yet scope changes are illuminated as “Business changes or reorganizes part way through the project”. 5.3 defines scope creep as a function of poor definition of scope and requirements, but again I’ll suggest that scope can change for other reasons. With 5.4, I’ll suggest that identity may play a part in starting a project without a sound business case, and that developing a system “for the sake of technology” is perhaps not so terribly different from developing a system for the sake of making a game you find fun.

The requirements category also impacts spec changes, This one seems focused on users, but does not appear in the relationship management section.

Personnel may lack skills or knowledge. Staffing seems to overlap. Are there other ways in which staff might be wrong for a project? Or at least sub-optimal, causing unnecessary risk?

The study identifies a number of risks not previously identified in literature:

Given the number of “new risks identified in this study, it is important to ask the question, “Are these risks really new?” Had earlier risk studies been conducted in a systematic manner similar to this study, it is conceivable that such risk factors would have surfaced. But it is more likely that the increasing pervasiveness of IT in all aspects of our daily lives has sensitized executives to the need to exercise much more oversight on IS projects. This, in turn, has pushed these risks to prominence in the minds of project managers.

In a similar fashion, I suggest the commercialization of IT and our increasing understanding of how humans identify with technology will show researchers new risk factors and alter our understanding of known factors.

The study shows significant differences between perceptions of risk between the three studied cultures.

Perceived level of control relates clearly with cultural differences in individualism, power distance, and uncertainty avoidance.

It does not attempt to differentiate by other factors. The authors assert that culture affect risk perceptions, so if we find cultural differences between segments of IT (rather than just by national border) then we might expect perceptions of risk to differ along those lines as well.

Proper risk assessment and the development of strategies to counter the risks requires an understanding of (1) what the actual risks are, (2) which of these risks managers perceive to be more deserving of their attention, and (3) how these risk perceptions differ from one culture to another.

In short, culture can affect risk. Identity is the internalization of perceived cultural meaning. One could expect identity to be related to risk perception and behavior.