CMMI VS ISO 9001 : A comparison

This article was written by me, Huan Pang, Mubeen Arshid and Noman Latif. I wish to share this article with all those who seek information about ISO and CMMI. Of course, if you feel somewhere I am wrong in this article, don't hesitate to point me out.

Abstract— Capability Maturity Model-Integrated (CMMI) and International Organization for Standardization (ISO) are standards that are widely being used in the industries. While ISO is more like a general standard that is applicable to any type of organization, CMMI focuses purely on the software developing or systems developing firms. In this article the authors discuss and state the weaknesses and strengths of each of this standard.

I. Introduction

Software process improvements (SPI) methods assure the delivery of successful software project. There are two different types of SPI:

1. Model Based SPI

2. Inductive SPI

Model based SPI is based on the external knowledge, pre-packaged and best practices. On the other hand inductive SPI is based on the internal knowledge of the organization e.g. Quality Improvement Paradigm (QIP) [1]. Both CMMI and ISO are the examples of model based SPI. This report will give overall reflection of both ISO and CMMI in depth.

The structure of the report is given as below:

· Section II gives the detail characteristics of CMMI and ISO.

· Section III explains the main differences between CMMI and ISO.

· Strength and weaknesses of both CMMI and ISO have been explained in the section IV.

· In section V, three most important process areas of CMMI have been discussed in detail.

A conclusion has been made at the end of the report for discussing the overall goal of SPI in this article.

II. Characteristics of CMMI and ISO

This section explains the intentions for developing ISO and CMMI. It also explains how it helps in improving the final product quality.

A. International Organization for Standardization (ISO)

ISO is a global organization which identifies and creates the required international standards for organizations, government bodies. Next, these standards are implemented, adopted and made available worldwide [2]. It was started in 1947. There are more than 18000 defined international standards at the moment. The purpose of developing international standards was to make international coordination and unification between organizations in order to have better quality and compatible products [3][4].

ISO 9000 series are quality and process management standards which were launched to reduce the problem of large number of standards [5][3]. ISO 9001 is a standard in the 9000 series which specifies the requirements. These requirements enable the organizations to provide the products according to customer and regulatory requirement [6]. ISO defines standards which tell “what” to do not “how” to do. They are general in nature and not specific to any application domain, development paradigm, life-cycle model, process model, type of development and type of product [3].

In order to get ISO certificate, the company need to fulfill all the requirements mentioned in ISO document. Then a certifier does the audit and gives the certificate to the organization. However it requires a lot of time and effort to fulfill ISO requirements It works as following [7].

· Plan to get ISO and gain commitment of people, particularly of the higher management.

· Assign the responsibility of the process to someone either from the company or consultant.

· Perform the assessment of current processes and find the gaps. This analysis figures out where the organization currently stands and what are the required changes to meet ISO requirements.

· Fill the gap by revising, adding or improving the processes and documenting the system to meet the ISO requirements. This is the most difficult and time consuming part. Plan-Do-Check (Study)-Act (PDCA or PDSA) is very useful model at this stage for improvement and analysis.

· Perform internal audit and if any problem is found, resolve them.

· Find the certifier and perform an external audit. The auditor will check all the ISO requirements whether or not it fulfills the requirements. After all requirements are verified, a certificate is given to the organization.

B. Capability Maturity Model-Integrated (CMMI)

CMMI is an approach for process improvement which provides organizations, essential element of effective processes to improve their performance [8]. It particularly focuses on system engineering and software engineering [9]. It was first published in 2000 as a collaborative effort of Software Engineering Institute (SEI), government representatives and industry representatives. The purpose of developing CMMI was to develop an integrated model, by merging popular and successful models, which is consistent with many other well known models [4]. According to Software Engineering Institute (SEI)

“It helps integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes”

Many people have misunderstanding that it is a methodology which tells the organizations how to implement processes. CMMI is a model not a methodology. It only tells what should be done, not how it should be done [4].

CMMI has two flavors: Staged and continuous. Selection of flavor depends on the organization’s goal. Staged model has five levels: Level 1 Initial, Level 2 Managed, Level 3 Defined, Level 4 Quantitatively managed and Level 5 Optimizing.

Each level is associated with process areas which must be met to achieve a certain level. Figure 1 shows staged levels like a stair of improvement for an organization.

In continuous model, each level is composed of practices related to one process which allow organizations to select a process area to concentrate upon [4]. For example if one organization wants to focus more on support process, they can use continuous model to focus on support. Figure 2 shows the CMMI continuous Levels. In order to get CMMI, companies assess their processes according to the level they want to achieve. Then fill the gaps by revising, adding or improving the processes.

II. Differences between CMMI and ISO

There are many differences between CMMI and ISO. Some of them are given below:

A. 1^st Difference (CMMI VS ISO)

CMMI has been developed by Software Engineering Institute. It’s an improvement of the previous CMM model. The basic use of CMM model was to determine whether the software intensive systems are mature enough or not. CMMI V 1.3 is the most recent version which has been released on November 1, 2010. CMMI addressed three different areas [10]:

· Development (CMMI-DEV)

· Services (CMMI-SVC)

· Acquisition (CMMI-ACQ)

For example, CMMI-DEV is used for checking the organizational maturity in development process by making a comparison with some best industry practices available [11].

ISO belongs to a family of quality management standards. These standards have been developed by International Organization for Standardization (ISO). There are different standards of ISO for different things and there is change in specification of ISO with time [11].

B. 2^nd Difference (Conceptual Difference)

The main difference between CMMI and ISO is the conceptual difference [11].

CMMI is referred to as process model. On the other hand, ISO is referred to as an audit standard [11]. In CMMI, different organizations can get rating from level 1 to level 5 depending upon the maturity of processes defined in every process level [12]. ISO is a certification tool and one organization can get this certification after confirming some standards [11].

C. 3^rd Difference (Scope Difference)

There is also scope difference between CMMI and ISO. CMMI is considered only to improve businesses related to software industry [12]. Main focuses of CMMI are on project management and other engineering disciplines. There are 22 process areas in CMMI (V1.2) and organizations can select any process area relevant to organization’s own need [11]. ISO is generic in nature. ISO is very flexible and can be implemented in any manufacturing industry. ISO certification requirements are same for all organizations and industries [11].

D. 4^th Difference (Approach Difference)

There is requirement in the CMMI for an organization to adopt ingraining processes [12]. The main purpose of this adoption is that all processes can become the part of the organizational culture and these processes can’t be affected with pressure of deadlines as well [11]. There are also organized and technical disciplines in CMMI for managing risk. There was a neutral approach in ISO for risk management before ISO 31000:2009. This ISO standard now provides some general guidelines for risk management [11].

CMMI links processes to different business goals for getting maturity and ISO gives emphasis to customer satisfaction.

E. 5^th Difference (Implementation Difference)

For implementation, CMMI makes a comparison between existing processes and industrial best practices [11]. On the other end, ISO makes an adjustment between existing processes and specific ISO requirements [11].

IV. Strengths / weaknesses of ISO and CMMI

A. Strengths of ISO 9001

1) Broad applicability

The strength of ISO is that it can be applied to any process assessment and improvement effort. It can be used for broad implementation in variety of industries, environments etc. [12]. ISO 9001 contributes to most organization entities, such as management, human resources, production, engineering, and quality. It can affect most of the functional areas of an organization [12].

2) International standard

Due to its benefits of enhancing customer satisfaction and experience by systematically improving the processes in an organization, ISO 9001 received an international recognition and appeal [12]. It helps in accessing larger market in a global setting industry. Due to standardization of processes between different organizations, customers and suppliers can both understand each other’s way of working. Thus, ISO facilitates the integration and collaboration between different organizations.

3) Freedom of implementation

As mentioned in previous section, ISO can be implemented in any organization flexibly. All the requirements contained in quality system elements of ISO 9001 can be interpreted, tailored and implemented according to the specific needs of the organization. This is because ISO states what to do instead of how to do [15]. In order to achieve more positive effects, the requirements can be implemented differently for different objectives in a company. Secondly, organizations can select specific parts of the standard according to their needs and objectives [15].

4) Performance improvement

The ISO 9001 provides guidance to quality management and assurance. It helps to specify quality system requirements that can be used to demonstrate supplier’s capability of providing adequate product quality, and enhanced performance [15].

B. Weaknesses of ISO 9001

1) Lack of specific guidelines and solid understanding

ISO 9001 is too general, because it does not provide specific guidelines for its implementation [12]. In order to understand the requirements correctly, people need to read and understand some other standards of ISO 9000 family, for example, ISO 9000-1, ISO 9000-2 and ISO 9004-1, which contain guidance for the design and implementation of quality systems [15].

A previous study shows that ISO 9001 have no empirical evidence, no theory, and no explicit model to show or explain relation between the suggestions and accomplishment of objectives [15]. Nobody knows if the suggested solutions of ISO 9001 can adequately reflect the problems in a specific organization [15].

2) No support for continuous improvement

The scope of ISO 9001:2000 does not include the continuous improvement. It is contained in ISO 9004-1 and 9004-4.

3) The focus on certification

Most software suppliers believe that the certification of ISO 9001 is the key factor to obtain market competitiveness. Although the certification process can be a strong motivation and encouragement for company’s staff members, it also has negative effects. The organization will ignore other important standards when they concentrate on certification of ISO 9001 [15]. It could be a possibility that only organization documents fulfill the criteria for acquiring the certification, however, no processes are actually changed by the management.

Moreover, ISO requires organizations creating their own quality management system (QMS). During the certification process, many organizations spend a lot of time and effort on developing and implementing their QMS. Due to this the organizations may not focus enough on understanding improvement, because more efforts are spent on promoting specification, control, and procedures [16].

C. Strengths of CMMI

1) Inclusion of institutionalization practices

CMMI emphasizes institutionalization through generic goals and generic practices. This is considered as critical to process improvement success. This gives the strength to CMMI [4]. Through the institutionalization goal, it points out a set of prerequisites needed to ensure that the specific practices are implemented [12].

2) Continual process improvement through maturity and capability levels

CMMI provides capability levels and maturity levels. Through these levels, improvement progression and status can be defined. It provides a “roadmap” and a proven sequence for improvement by advancing to next level [12]. CMMI focuses on continuous improvement during the process. The progress of improvement can be reflected by comparing the process areas across and among organizations [17]. A high maturity level is attained by progressing and clearing each maturity level. This is because skipping maturity levels is usually counterproductive [17].

3) Recognition of organizational process versus project-defined processes

CMMI emphasizes comprehensive program management practices [18]. It makes it possible to first stabilize the management activities in an organization before introducing advanced technology into processes [12]. By moving forward to higher maturity level, processes in organization and project are improved to ensure high quality of delivered product or service.

4) Sufficient guidelines

CMMI provides detailed guidelines for systematic implementation of process improvement [12]. It helps in measuring and improving development and management performance, as well as ensuring the quality of final product or services. By following the guidelines, productivity, efficiency, and performance are enhanced as well.

5) Long-term benefit

The fruits for efforts of implementing the requirements of CMMI, appear quite late in the process improvement. It can be regarded as a weakness because employees of organizations may start losing the focus on process improvement. However, accelerated process improvement methodology helps to lower this weakness. In the long-run the real benefits of achieving higher CMMI level can be met [4].

D. Weaknesses of CMMI

1) Specific applicability

Unlike ISO, CMMI does not cover all organizational aspects. It is just intended for application to the area of software engineering, system engineering, product development, supplier sourcing [12][17]. Also, CMMI does not address the issues related to IT operation, for example, security, configuration and change management, and incident response, etc. [16]. It does not cover human resources also.

2) Lack of an explicit model discription

As mentioned earlier, CMMI also lacks model explanation. Study has shown that people need to spend a lot of time on learning how, when, why, and for whom process improvement is helpful, and understanding the critical factors that cause success and failure [19].

References

[1] L. Briand, K. El Emam, and W. L. Melo, “ANSI–An Inductive Method for Software Process Improvement: Concrete Steps and Guidelines,” 1995.

[2] “ISO in Brief. Link = http://www.iso.org/iso/isoinbrief_2008.pdf, Last visited = 2010-12-17.”

[3] C. Gencel, “Lecture 4: ISO and CMM. Lecture devlivered in subject "Software Quality Management",” 10-Dec-2010.

[4] D. Jacobs, Accelerating process improvement using agile techniques. CRC Press, 2005.

[5] M. C. Paulk, “How ISO 9001 compares with the CMM,” Software, IEEE, vol. 12, no. 1, pp. 74–83, 2002.

[6] “ISO 9001:2000. link = http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=21823. Last visited = 2010-12-17.” .

[7] “How to get ISO. linke = http://www.mapwright.com.au/how_to_get_ISO9001_process.html. Last visited = 2010-12-17.” .

[8] “CMMI Overview. link = http://www.sei.cmu.edu/cmmi/index.cfm. last visited = 2010-12-17.” .

[9] “Update observations of the relationship between CMMI and ISO 9001:2000. link = http://www.asq509.org/ht/a/GetDocumentAction/id/1319. last visited = 2010-12-17.”

[10] “CMMI Version 1.3 Information Center link = http://www.sei.cmu.edu/cmmi/tools/cmmiv1-3/. last visited = 2010-12-18.” .

[11] N. Nayab, “Difference Between CMMI vs ISO link = http://www.brighthub.com/office/project-management/articles/69310.aspx. last visited = 2010-12-18.” .

[12] B. Mutafelija and H. Stromberg, Systematic process improvement using ISO 9001: 2000 and CMMI. Artech House on Demand, 2003.

[13] “The ISO 9000 family – core standards. linke = http://www.iso.org/iso/iso_catalogue/management_standards/quality_management/iso_9000_selection_and_use/iso_9000_family_core_standards.htm. Last visited = 2010-12-20.” .

[14] “Components of CMMI Model Wikimedia Commons. link = http://www.brighthub.com/office/project-management/articles/69310.aspx?image=73481 last visited =2010-12-20.” .

[15] D. Stelzer, W. Mellis, and G. Herzwurm, “A critical look at ISO 9000 for software quality management,” Software Quality Journal, vol. 6, no. 2, pp. 65–79, 1997.

[16] “The 'quality' you can't feel. link =http://www.systemsthinking.co.uk/6-quality.asp. last visited = 2010-12-20.” .

[17] “Maturity Model Or Conformity Standard. link = http://www.slideshare.net/ROUSES63/maturity-model-or-conformity-standard. last visited = 2010-12-20.” .

[18] “Maturity Model or Conformity Standard: CMMI or ISO 9001: Which is Better link =http://www.slideshare.net/ROUSES63/maturity-model-or-conformity-standard last visited = 2010-12-20.” .

[19] J. Herbsleb, A. Carleton, J. Rozum, J. Siegel, D. Zubrow, and C. U. P. P. S. E. INST, Benefits of CMM-based software process improvement: Initial results. Citeseer, 1994.

[20] “http://www.sei.cmu.edu/cmmi/. last visited = 2010-12-20.” .

[21] M. , H.J, and K.O. Hartley, “Project Planning and
Performance,” Project Management Journal, Mar. 1986.

[22] Rehessar, “Project Management Success Factors.,” University of New South Wales, 1996.

[23] H. Zhang, B. Kitchenham, and R. Jeffery, “Planning Software Project Success with Semi-Quantitative Reasoning,” in Software Engineering Conference, 2007. ASWEC 2007. 18th Australian, pp. 369–378, 2007.

[24] T. A. Clark, Project Management for Planners: A Practical Guide. 2002.

[25] L. Liu, Y. Jiang, and C. Zhu, “Process-related software requirements management,” in Computer Science and Information Technology (ICCSIT), 2010 3rd IEEE International Conference on, vol. 9, pp. 361–365, 2010.

[26] S. Datta and R. van Engelen, “Effects of changing requirements: a tracking mechanism for the analysis workflow,” in Proceedings of the 2006 ACM symposium on Applied computing, pp. 1739–1744, 2006.

[27] W. N. Robinson, S. D. Pawlowski, and V. Volkov, “Requirements interaction management,” ACM Computing Surveys (CSUR), vol. 35, no. 2, pp. 132–190, 2003.

[28] N. G. Leveson, Safeware: system safety and computers. ACM New York, NY, USA, 1995.

[29] J. L. Lions, “Ariane 5: Flight 501 failure report,” Paris, France: European Space Agency (ESA) & National Center for Space Study (CNES) Inquiry Board, July, vol. 5, no. 6, p. 7, 1996.

[30] B. Nuseibeh, “Ariane 5: who dunnit?,” IEEE SOFTWARE, pp. 15–16, 1997.

[31] B. W. Boehm, “Software risk management: principles and practices,” IEEE software, pp. 32–41, 1991.