When I wanted to learn about SPICE, CMMI, ISO, I faced some problem related to learning of SPICE. So I decided to share this information here. I will try to provide the differences between ISO, CMMI and SPICE in a series of posts.
1. Introduction
Due to the increased awareness of the fact that the quality of a product depends on the quality of process, more and more organizations started to study and adopt international standards such as Spice, ISO 9001:2000 and CMMI etc. to improve their software process. International standards help to initiate software process improvement in organizations.
International standards are created by dozens of organizations and hundreds of professionals, academics etc. working together across the globe [1]. With more and more organizations moving towards global software development, many conflicts arise due to geographical, technological and cultural differences [2]. International standards are considered as an adaptable way to prevent or overcome barriers and incompatibility issues caused by such differences. In this section the authors will give a description of the three widely used international standards and models in the world.
A. CMMI
CMMI (Capability Maturity Model-Integrated) is a well-known, widely used and standardized model which primarily focuses on process assessment and process improvement of software and systems [3]. CMMI offers continuous and staged improvement through maturity levels and capability levels [4]. These levels assist in indentifying and evaluating the progress and the status of the improvement. It presents a “roadmap” and a demonstrated sequence of process improvement by advancing to next level. By providing guidelines, systematic implementation of process improvement is made possible by CMMI [5].
B. ISO
ISO (International Organization for Standardization) is a global standard organization that identifies and develops standards for business, government and society [6]. According to the updated list of ISO standards, it has developed over 18000 international standards covering a variety of subjects. Almost 1100 new ISO standards are published every year [7]. The purpose of ISO is to facilitate global consensus agreements on international quality standards, and make international coordination and unification among organizations. It has resulted in a system for certifying suppliers to make sure they meet internationally accepted standards for quality management. These standards contribute and influence every aspect of our lives, because they assist in ensuring and enhancing essential features, such as quality, safety, economy, reliability, compatibility, efficiency and effectiveness. As a result, spread knowledge, shared technological advances and skilled management practices are achieved [6].
Every standard of ISO has specific focus. ISO 9000 family focuses on quality management which ensures the product or service meet the expectation and need of customers or other stakeholders. ISO 9001 is one of the standards in the ISO 9000 family, which specifies quality requirements that can be used to demonstrate supplier’s capability of providing adequate product quality, and enhanced performance [8]. ISO 15504 focuses on process improvement and capability evaluation.
Comparing with CMMI, ISO is more general and applicable to any type of organization and environments. Both CMMI and ISO are the examples of model based Software process improvements (SPI), widely being used in the industries.
C. SPICE
ISO/IEC 15504, commonly referred as SPICE, is the international standard for process assessment, which was initiated in 1993 as the SPICE Project [9]. The project was to support the development, validation of a practical international standard for software process improvement. It was jointly developed by ISO (International Organization for Standardization) and the IEC (International Electrotechnical Commission) [1][9][10][11]. This framework helps the organizations in terms of controlling planning, managing, monitoring and improving the acquisition, supply, development, operation, evolution and support of software [12]. The initial versions of ISO/IEC 15504 focused exclusively on software development processes. Later, it was extended to cover all processes related to software life cycle, for example, project management, configuration management, quality assurance etc. [10].
ISO/IEC 15504 can be considered as the product of SPICE project, which comprises of a guide for performing an assessment for software development processes [12]. As part of this standard, framework for software process assessment, a model about how to perform an assessment, a description of tools that can be used as part of the assessment process and a discussion of factors that can lead to a successful assessment are provided [13]. The structured approach provided by ISO/IEC 15504 can be used for process improvement as well as for capability determination [14][15]. ISO/IEC 15504 makes it possible for organizations to use the standard for process capability determination mode, process improvement mode and self-assessment mode [16].
ISO/IEC 15504 enhances confidence in a supplier's quality management for the intent of ISO 9000 and provides acquirers with a framework for assessing whether potential suppliers have the capability to meet their needs in a comparable and repeatable way [12].
The development of ISO/IEC 15504 was based on the weaknesses of previous standards and models. It is considered as to complement other international standards and models for assessing the capability, effectiveness and quality of processes and organizations [12]. Other standards influence ISO/IEC 15504 to some extent. ISO 9001 TickIT influenced ISO/IEC 15504 model in the way that it encouraged quality assurance and quality management [17]. SW-CMM, Trillium influenced ISO/IEC 15504 in the way that it determines the capability of processes [17].
In this article, we focus on studying ISO/IEC 15504 and how it relates to CMMI and ISO 9001:2000. The article is organized in the following way.
Section II describes ISO/IEC 15504 in detail. Section III provides a critical analysis of the structure and focus of ISO/IEC 15504. A comparison of ISO/IEC 15504 with ISO 9001:2000 and CMMI is presented in section IV, based on analyzing the similarities and differences, the strength and weakness of these standards and models.
2. Overview of SPICE and characteristics
“ISO/IEC 15504 is a framework for the assessment of processes” [7]
As stated earlier, ISO/IEC 15504 is the result of efforts of SPICE project. Initially, SPICE had 9 parts. However, in 2004, after a major revision, to the draft standard, the 9 parts were reduced to 5. Later on, ISO/IEC 15504 was extended further.
A. Structure of the standard
At the moment, ISO/IEC 15504 consists of seven parts [9]. Another part, part eight, is currently under development. These parts are stated as under [13].
Part 1. ISO/IEC 15504-1:2004 (Concepts and vocabulary)
This part is the most important part of the standard as it defines the terms, twin contexts of process improvement and process capability determination and the concepts of process assessment [18].
Part 2. ISO/IEC 15504-2:2003 ( Performing an assessment)
The actual requirements for performing process assessment are described in this part. These requirements should be used as a basis for use in process improvement and capability determination [13].
The process improvement requirements in ISO/IEC 15504-2:2003 helps to [13]:
· facilitate self-assessment;
· provide a basis for use in process improvement and capability determination;
· produces a process rating;
· focus on the ability of the process to achieve its purpose;
· be applicable across all application domains and different sizes of organizations
· Provide an objective benchmark between organizations.
Part 3. ISO/IEC 15504-3:2004 (Guidance on performing an assessment)
This part gives detailed steps regarding how to meet the minimum requirements for performing an assessment presented in ISO/IEC 15504-2:2003 [19].
Part 4. ISO/IEC 15504-4:2004 (Guidance on use for process improvement and process capability determination)
ISO/IEC 15504-4:2004 gives directions regarding the effective utilization of conformant process assessment within a process improvement or process capability determination project [20].
Part 5. ISO/IEC 15504-5:2006(An exemplar process assessment model)
Guidance is provided in this part on the nature and structure of process assessment models (PAM), and on the function of process performance and capability indicators. This guidance is provided by example [21].
Part 6. ISO/IEC 15504-6:2008 (An exemplar system life cycle process assessment model)
Based on the requirement PAM of system life cycle processes in ISO/IEC 15504-2, exemplar guidance is provided in this part. This part also describes the overall structure of the PAM and the capability dimension from ISO/IEC 15288 and ISO/IEC 15504-2 respectively [22].
Apart from this, this part also provides detailed descriptions of work products and steps for defining additional assessment indicators [22].
Part 7. ISO/IEC TR 15504-7:2008( Assessment of organizational maturity)
Organizational maturity relates to the degree the organization implements processes, in order to achieve its business goals, within a defined scope. ISO/IEC TR 15504-7:2008 defines the conditions for the assessment of organizational maturity and provides a framework which helps to conduct assessment of organizational maturity [23].
B. Activites
In order to conduct the assessment, it is necessary to know how the processes are divided and identified.
1) Reference Model
A two dimensional process reference model (PRM), containing the process dimension and a capability dimension, is defined under ISO/IEC 15504-2:2003. ISO/IEC 12207 and ISO/IEC 15288 define process set characterized by statements of process purpose and process outcome. They define the process dimension of ISO/IEC 15504-2.
2) Processes
The process dimension defines processes divided into 5 categories. These are as under:
- Organization
- Management
- Engineering
- Support
- Customer-Supplier
3) Maturity levels and their attributes
The measurement framework of ISO/IEC 15504-2 defines six capability levels. It also assigns process attributes to each level. At level 0, no attributes are assigned [14][12][13]. Figure 1 shows the six capability levels of the standard. Figure 2 shows the capability levels with their attributes assigned.
 |
Figure 1: Six Capability levels of ISO/IEC 15504 [24] |
 |
Figure 2: Capability levels of ISO/IEC 15504 with assigned attributes [3] |
4) Assessment
Part 2 and part 3 of ISO/IEC 15504 provides guidelines for performing assessment.
a) Assessment model
The PAM is an elaboration of PRM. As stated already, PRM uses ISO/IEC 12207 and ISO/IEC 15288 for the definition of process dimension.
The following steps show the generalized steps of process assessment.
- assessment sponsor
- Assessment team and assessor are selected
- Assessment is planned (resources set aside, targets / goals identified etc.)
- pre-assessment briefing
- start data collection
- start data validation
- process rating based on guidelines
- reporting the assessment result
The assessor collects the data by interviewing the assessment team. He ensures that the data collected is accurate and covers the assessment scope completely. This data is evaluated against the process’s practices and capability dimensions expected practices. After careful review rating is provided. The results are sent to the sponsor.
Next week, I will write about the critical analysis of SPICE, highlighting its strengths and weaknesses.
References
[1] T. P. Rout, “ISO/IEC 15504 and Spice, link = http://onlinelibrary.wiley.com.miman.bib.bth.se/doi/10.1002/0471028959.sof171/full, last visited = 2011-01-06.”
[2] H. Holmstrom, E. O. Conchuir, P. J. Agerfalk, and B. Fitzgerald, “Global software development challenges: A case study on temporal, geographical and socio-cultural distance,” 2006.
[3] N. Ehsan, A. Perwaiz, J. Arif, E. Mirza, and A. Ishaque, “CMMI/SPICE based process improvement,” in Management of Innovation and Technology (ICMIT), 2010 IEEE International Conference on, pp. 859–862, 2010.
[4] D. Jacobs, Accelerating process improvement using agile techniques. CRC Press, 2005.
[5] B. Mutafelija and H. Stromberg, Systematic process improvement using ISO 9001: 2000 and CMMI. Artech House on Demand, 2003.
[6] “ISO in Brief, link = http://www.iso.org/iso/isoinbrief_2008.pdf, last visisted = 2011-01-06.” .
[7] “ISO official website, link = http://www.iso.org/iso/home.htm, last visited = 2011-01-06.” .
[8] D. Stelzer, W. Mellis, and G. Herzwurm, “A critical look at ISO 9000 for software quality management,” Software Quality Journal, vol. 6, no. 2, pp. 65–79, 1997.
[9] Scott P. Duncan, “MAKING SENSE OF ISO 15504 (AND SPICE),” Quality/Process Improvement Consultant SoftQual Consulting 6029 Flat Rock Rd #437, Columbus, GA 31907 (706) 565-9468.
[10] T. P. Rout, K. El Emam, M. Fusani, D. Goldenson, and H. W. Jung, “SPICE in retrospect: Developing a standard for process assessment,” Journal of Systems and Software, vol. 80, no. 9, pp. 1483–1493, 2007.
[11] T. P. Rout, “Guest editorial software process improvement and capability determination: selected articles from SPICE,” J. Softw. Maint. Evol.: Res. Pract, vol. 22, pp. 239–241, 2010.
[12] “SPICE, link = http://www.sqi.gu.edu.au/SPICE/, last visited = 2011-01-06.” .
[13] “ISO Standard: ISO/IEC 15504-2:2003.”
[14] A. Dorling, “SPICE: Software process improvement and capability determination,” Software Quality Journal, vol. 2, no. 4, pp. 209–224, 1993.
[15] “Software Process Improvement and Capability dEtermination, link = http://searchsoftwarequality.techtarget.com/definition/Software-Process-Improvement-and-Capability-dEtermination, last visisted = 2011-01-07,” .
[16] M. Konrad, M. Paulk, and A. Graydon, “An overview of SPICE's model for process management,” in Proc. 5th International Conference on Software Quality.
[17] “History and relationship of
process standards/models, link = http://www.tantara.ab.ca/a_isorel.htm, Last visited = 2011-01-06.” .
[18] “ISO Standard: ISO/IEC 15504-1: 2004.”
[19] “ISO Stanndard: ISO/IEC 15504-3:2004.”
[20] “ISO Standard: ISO/IEC 15504-4:2004.”
[21] “ISO Standard: ISO/IEC 15504-5:2006.”
[22] “ISO Standard: ISO/IEC 15504-6:2008.”
[23] “ISO Standard: ISO/IEC 15504-7:2008.”
[24] “CMMI, SPICE, Six Sigma...What's their business value? link = http://www.bcs.org/server.php?show=conWebDoc.7901, last visited = 2011-01-07.”
[25] “ISO/IEC 15504. link = http://en.wikipedia.org/wiki/ISO/IEC_15504, last visited = 2011-01-06.” .
[26] B. Barafort, B. Di Renzo, and O. Merlan, “Benefits resulting from the combined use of ISO/IEC 15504 with the Information Technology Infrastructure Library (ITIL),” Product Focused Software Process Improvement, pp. 314–325, 2002.
[27] “Software process engineering systems: models and industry cases, link = http://herkules.oulu.fi/isbn9514265084/html/x199.html, last visited = 2011-01-06.” .
[28] A. Coletta, “The SPICE project: An international standard for software process assessment, improvement and capability determination,” Objective Software Quality, pp. 49–63, 1995.
[29] M. E. Fayad and M. Laitnen, “Process assessment considered wasteful,” Communications of the ACM, vol. 40, no. 11, pp. 125–128, 1997.
[30] T. B. Bollinger and C. McGowan, “A critical look at software capability evaluations,” Software, IEEE, vol. 8, no. 4, pp. 25–41, 2002.
[31] E. M. Gray and W. L. Smith, “On the limitations of software process assessment and the recognition of a required re-orientation for global process improvement,” Software Quality Journal, vol. 7, no. 1, pp. 21–34, 1998.
[32] H. W. Jung, “Evaluating the ordering of the SPICE capability levels: an empirical study,” Information and Software Technology, vol. 47, no. 3, pp. 141–149, 2005.
[33] “Software Process Improvement and Capability dEtermination, link =http://searchsoftwarequality.techtarget.com/definition/Software-Process-Improvement-and-Capability-dEtermination, last visited = 2011-01-06.” .
