SEI CMM & CMMI
In
July, 2003, Cimarron Software Products Division's Software Engineering Institute (SEI) Capability Maturity Model (CMM) processes were independently assessed
at Level 3.
We continue to internally assess our processes to improve our software development processes. We are currently planning for initiating CMM Integrated (CMMI) activities. We are applying the organizational structure utilized in the CMM processes in our program management processes.
Areas that are proving noteworthy include implementing monthly project reviews with Cimarron Executive Management, applying lessons learned throughout Cimarron departments and programs, identifying risks for program management, and improved training for managers.
The Software Engineering Institute
The Software Engineering Institute (SEI) is a federally funded research and development center sponsored by the U.S. Department of Defense and operated by Carnegie Mellon University (CMU). CMU SEI has defined a Software Capability Maturity Model (CMM) for judging the maturity of the organizational and managerial processes of an organization, and for identifying the key practices that are required to increase the maturity of these processes. The CMM is organized into five maturity levels, from level 1 to level 5, and has become an industry standard for assessing and improving software processes. The NASA and the DOD depend heavily upon SEI certification as a determining factor in selecting software development contractors. The CMM is less used in commercial environments. However, ISO9001 requires in its mandatory section, Section 4.4 Design Control, quality control procedures at a high level that the CMM, when implemented, satisfies in full. Basically, the CMM is a fitting implementation of ISO 9001 Section 4.4 Design Control for a software development organization.
Each higher level assumes the previous level's process maturity. For a development process to be certified, auditors from the SEI analyze the documented processes for their compliance with the Key Process Areas of the level using the CMM's Key Practices that define the attributes of a compliant Key Process Area. As a development process gets under more management control and becomes more robust, the process becomes more highly rated.
CMM Level 1 (the Initial Level) defines a chaotic environment. In the SEI's viewpoint, chaotic means that the product is dependent on the direction and control that each individual or small group accomplishing the work exercises. The quality of a product will vary based on who is assigned to the product development. CMM Level 1 does not necessarily mean that what is produced is of less quality, just that it depends on the individual talents that are assigned.
CMM Level 2 (the Repeatable Level) is one where rules and regulations have been established in developing a product, that the results of the process are documented and audited by management, and that the rules and regulations are occurring. A product set developed by Level 2 processes thus becomes less dependent on who is assigned. It is possible with Level 2 processes that different product sets developed by different product teams will become more predictable in their cost per unit of production and of similar overall quality.
CMM Level 3 (the Defined Level) institutionalizes a particular set of processes for all products being developed. The requirements of the process are:
- software quality is tracked
- management and software engineering processes are stable and repeatable
- a quality organization monitors the organizations processes (a Software Engineering Process Group or SEPG)
- an organization-wide training program is in place to assure that all personnel know the defined roles and responsibilities of the individuals and the functional organization within the process.
While it is not required that processes be automated, the intent of Level 3 is to use the results of all the audits to fine-tune processes. For instance, if it is found that 5 out of 10 database calls are typically wrong at design or code inspection time, then an action is generated to correct whatever information or training is being provided to the staff whereby they are not currently able to code the database calls correctly.
CMM Level 4 (the Managed Level) is highly automated. All software processes are instrumented with quantitative quality measures. The results of the measurements are fed back into the process in the form of schedule and cost updates, process improvements and corrective action. Control of the process and quality of the product is driven by measuring and then narrowing the variation in the quantitative measurements.
At CMM Level 5 (the Optimizing Level) the entire organization focuses its attention on continuous process improvement, using the quantitative measures of the process to drive the improvement. Effectiveness data taken from the quantitative measurements of the process is used to perform cost benefit analysis of new technologies and proposed changes to the processes. Innovations that exploit the best software engineering practices are identified and transferred throughout the organization. Defects are analyzed and strategies to prevent their future occurrence are instituted. Prediction of changes in process may be made and then compared to actual quantitative results
Current experience with the CMM is that the higher the Level rating the more expensive the process. It has been widely reported by NASA's Onboard Shuttle Computer contractor that it costs $240 per line of code to operate at SEI Level 5. Level 5 is appropriate in man-critical or mission-critical systems where loss of life or mission would mean loss of program, there is no reasonable chance of re-establishing the program, and the program is considered a national or business priority of highest rank. However, it is not clear that a Level 5 MUST cost more, especially as an organization learns and assimilates the processes into the way they do business over time. As time passes and commercial tools are built to support the CMM processes in the industry, then achieving a Level x will get less costly for everyone.
An SEI Level 2 process is appropriate for many business applications and really does not ask anything of the processes or participants that common sense would not already guide them to do anyway (at least today after 10 years or so of SEI). Getting from Level 2 to Level 3 is relatively difficult, not because of the additional processes required, but because of the investment in the SEPG, the inevitable automation and standardization of the processes across the whole enterprise, and the resistance to the culture change that many software people evidence when they are constrained to an institutionalized set of processes.
