ABSTRACT

It is essential to study quality of software production like others. Software productions have special properties. They are intangible. So, qualitative evaluation encountered with complexity. Hence, proposing a model in order to evaluate the quality of Software productions is considerable to most software managers and experts. In this paper, regarding to improve software productions quality, the process of software production has been determined, using CMM standard framework of maturity level. In CMM it does not present a method for measurement and evaluation maturity level, the presented process in CMM standard mapped by COBIT control objectives has been combined in the process of software productions development in developed hybrid framework. In this research, the processes have been mapped utilizing focus and established group, in parallel of software production in different maturity level of CMM mapped by COBIT framework. In order to show the capabilities of proposed framework, the hybrid evaluation model was employed in a software developing organization as a case study. According to the results of evaluation, improvements proceedings and action plans have been proposed and discussed to enhance the software production processes.

1. Introduction

Software developers try to analyze business environment using diverse analysis and design methods.

Even in some cases, analysts improve processes before designing and implementing software, and then propose a logic model and required documents for software producers.

They try hard to produce software according to the customer’s needs and in this aspect, improve the quality of their software. Software producers, also try to deliver their projects to the customers on time using estimated time, cost and HR models, in regard to improve the quality of their software.

Many researchers and experts believe that delay in delivering projects results from incorrect estimation in required resources, such as time and cost. Another factor leads to delay, is the lack of coordination in related sectors of a project. Corporation and interaction in different sectors is one of the requirements of planning and implementation phase.

The other influential reason on software projects’ delay is changing customer requirements and lack of proper management in changing requirements. Some problems will have been happening in the environment that different groups of project don’t have any coordination to one another and customers’ requirements aren’t managed. These Issues can influence the quality of project.

The keys are often available in experts’ hands of successful company. It means that loyal and cooperative experts can solve the observed problems resulted from improper planning and project groups disorganization, but in condition that experts don’t have these qualified personals or the system is complex, there isn’t any guarantee for software quality.

Observed complexity in evaluation software production quality level leads to company’s evaluation inability to measure customers’ consent and the quality of software production process. This research utilizes standard, successful and updated methods to propose a documented plan excluded from characteristic and qualification of individuals in order to improve required process for software quality enhancement.

In this paper a hybrid model of CMM and COBIT framework is proposed to evaluate software production quality.

CMM and COBIT are introduced shortly, soon after literature review, and then the method of process mapping of these two standards is described.

In case study section of this essay, failure reasons of a software project are examined.

Finally, function of the model in evaluation and improvement of software production process has been described and concluded.

2. Literature Reviews

The quality and effectiveness of any software can be measured by customer satisfaction. In present era, all software companies up to make quality software within and cost [1,2]. The most important quality attributes of a software product is usefulness; i.e. the software must satisfy user needs [3]. The quality of software product is defined in terms of its basic components, via which it’s constructed, and each component of product is uniquely characterized. Also the conformance to applicable specification and standard that is agreement between user and software product developer [3].

Software quality is difficult to define is no single comprehensive and complete definition of its lexicon. The quality is very difficult task to accomplish in the result of there is not explicit necessities of each customer [1,2].

Software engineering and quality assurance has to be an integral part of system engineering right from the beginning of projects and be organized according to Industrial standards to be prepared for challenges of many industries [4].

Common problems to attain the quality are: poor definition of requirements, poor performance of system analyst, poor testing, poor documentation, wrong estimation of time and cost [1,5]. The quality of software can be provided through various ways like: by testing, by inspection, by standard.

Testing is overwhelming process for maintain the quality of the software, instead of using it inspection of code is time economy process has been decide on. The software quality through inspection cannot be increased as this is not an effective to eradicate all bugs [1,6].

In order to enhance the quality, productivity and reduced cost of the software organizations are promoting to produce the reuse-oriented products [7].

Today various organizations are adopting a variety of international standards for providing high quality software to their customers. These standards define process framework for a number of processes used in software development like designing coding testing etc. [1,8].

Software architecture design is a critical step of software development. Currently, there are various design methods available. However, the use of quality-based design methods is limited in software product line (SPL) because of the complexity and variability existing in SPL architecture [9]. The Software Engineering Institute (SEI) developed an initial version of a capability maturity model (CMM) at the request of the government and with the assistance of the MITRE Corporation [10].

Software Process Improvement (SPI) “best practice” models such as ISO 9000 and the Capability Maturity Model Integrated (CMMI) have been developed to assist software development organizations by harnessing their experience and providing them with support so that they can produce software products on time, within budget and to a high level of quality [11].

Advantages of SPI and CMM model are: Increase quality [12], risk management [13], gain to business goals [14].

In order to evaluate the maturity of the software development process in a software company, software measurements are needed and used to measure specific attributes of a software product or software process [6, 15].

CMM evaluation depends on external agency and its limitation is CMM lacking improvement through measurement [1,6].

In CMMI, the lead appraisers can know the effects and the performance of institutionalized process in one company according to specific and generic goals, and generic and specific practices defined in process area. Lead appraisers can also evaluate in qualitative description based on questionnaire, interview and document by appraisal requirements for CMMI, and standard CMMI appraisal method for software development process. The CMMI evaluation results are usually dependent on the lead appraiser’s subjective judgment [16-18].

CMMI works only on achieving maturity levels instead of quality improvement so organizations even reaching the maturity level are not fully satisfied because People using the new processes in most cases feel the improvement, while management expects measurable contribution to a company’s business objectives [1].

In this paper, COBIT framework has been employed in order to prevent weakness of CMM. This framework directed IT processes in the organization to gain business objectives. COBIT can evaluate IT processes with control objectives. COBIT processes cover all CMM processes. Therefore this paper has been employed COBIT framework to evaluate CMM processes maturity and directed processes to business goals.

During performing this research model, another group was also researching a model for mapping CMM and COBIT [19]. Their results do not match with this paper finding. Hence, Section 6 of this paper describes some main differences between these two maps.

3. A Summary Definition of CMM

The Software Engineering Institute (SEI) developed an initial version of a maturity model and maturity questionnaire at the request of the government and with the assistance of the MITRE Corporation.

Throughout the development of the model and the questionnaire, the SEI has paid attention to advice from practitioners who are involved in developing and improving software processes. The Capability maturity model (CMM) has been to provide a framework which is based on actual practices; reflects the best of the state of the practice.

The CMM is composed of five maturity levels. With the exception of Level 1, each maturity level is composed of several key process areas. Each key process area is organized into five sections called common features. The common features specify the key practices that, when collectively addressed, accomplish the goals of the key process area.

Figure 1 represents the key process areas for each maturity level in the CMM.

4. A Summary Definition of COBIT

The IT Governance Institute (ITGI) was established to advance international standards in guiding and controlling an enterprise’s information technology efforts. An effective IT governance standard can make sure that IT supports business objectives, manages and optimizes IT-related works.

ITGI has designed and created COBIT^® 4.1, primarily as an educational resource for chief information officers (CIOs), IT management and professionals. IT governance is the responsibility that ensures that the enterprise’s IT supports and aligns with the organization’s strategies and objectives.

Furthermore, IT governance integrates best demonstrated practices to support that the Organization’s IT enhances the business goals. IT governance can help the enterprise to have the advantage of its information and increasing competitive advantage [19].

Control Objectives for Information and related Technology (COBIT^®) provides good practices across a domain and process framework and presents activities in a manageable and logical structure. COBIT’s good practices represent the consensus of experts.

COBIT strongly concentrated on control more than execution. These practices will help optimize IT-enabled investments, support service delivery and provide a measurement against the personal judgment.

For a successful IT to support business requirements, management should set an internal control system or framework. The COBIT control framework provides

these needs by:

• Making a link to the business requirements;

• Organizing IT activities into a generally accepted process model;

• Identifying the major IT resources to be leveraged;

• Defining the management control objectives to be considered.

After the text edit has been completed, the paper is ready for the template. Duplicate the template file by using the Save As command, and use the naming convention prescribed by your journal for the name of your paper. In this newly created file, highlight all of the contents and import your prepared text file. You are now ready to style your paper.

5. Research Methodology

This paper is the result of a practical research. The type of data gathering is questionnaire. In considering the essence of this research, the researcher look for people who are as expert in CMM and COBIT concepts and mapping between them. Therefore, a focus group of all available experts of CMM and COBIT is used. At first a questionnaire was designed mapping CMM and COBIT processes. The questionnaire was modified by experts’ opinion and then the final mapping model was designed. In case study, the questionnaire designed utilizing COBIT control objectives answered by statistical sample such as managers and experts of department in order to evaluate the maturity of project process.

In this way, next after gathering data, maturity level of the project processes was determined by employing proposed method.

The research methodology is represented in Figure 2.

6. The Hybrid Model of CMM and COBIT in Evaluation and Improvement of Processes

In this research, the equivalent of CMM processes in COBIT framework has been searched conceptually, in the condition that it has been begun from the first CMM process of project. Details to this process and its objectives have been considered, and then the related domain of this process in COBIT framework has been found. In the next phase, conceptual equivalent of CMM process in COBIT framework has been found in accurate consideration of observed domain process.

4 domains have been defined in COBIT processes and they cover all of the CMM processes. Therefore, the CMM equivalent processes were found In COBIT framework, and then control objectives of COBIT were mapped to evaluate CMM processes.

During performing this research model, another group

also introduced a model for mapping CMM and COBIT [20]. These research results and their finding does not match completely. Hence, in the following are described some of the main difference between these two models.

The difference between the two maps of CMM and COBIT.

The specific mapping of this article The “requirement management “process in CMM was mapped to the “Manage Changes” and “Determine technology direction” process in COBIT framework.

The “Software project planning” process in CMM was mapped to the “Manage IT Human Resource” process in COBIT framework.

The “training program” process in CMM was mapped to the “Personnel training” process in COBIT framework.

The “software product engineering” process in CMM was mapped to the “Determine technology direction” process in COBIT framework.

The “software quality management” process in CMM was mapped to the “Monitor and evaluate” process in COBIT framework.

The “inter group coordination” process in CMM was mapped to the “Manage third party services” process in COBIT framework.

The “defect prevention” process in CMM was mapped to the “Manage Problems” and “Monitor and evaluate Internal control” process in COBIT framework.

Table 1 represents the difference between presented model and ISACA mapping.

The parts of the mapping omitted from ISACA map The “quantitative process management” process in CMM has been mapped to the “Define a strategic IT plan” process in COBIT Framework but was omitted in this paper model.

The “organization process definition” process in CMM has been mapped to the “Manage the configuration” process in COBIT Framework but was omitted in this paper model.

The “training program” process in CMM has been mapped to the “Educate and train user” process COBIT framework but was omitted in this paper model.

The “software product engineering” process in CMM has been mapped to the “Manage Problems” process in COBIT Framework but was omitted in this paper model.

The “quantitative process management” process in CMM has been mapped to the “Manage Problems” process in COBIT Framework but was omitted in this paper model.

Table 2 represents the parts of the mapping omitted from ISACA mapping in represented model.

7. Case Study

The developed framework is applied to a software company.

Case study is about one of the projects which have been started since 5 years ago but it hasn’t leaded to any production. Customers prefer their traditional method and don’t accept the system. Company also can not finish the project. It has been decided to use the proposed model of this research.

In order to find the reason of the problem, a questionnaire has been planned to evaluate the project processes of each section in this project.

This questionnaire is derived from COBIT control objectives. Andrea Pederiva proposed an evaluation method for this questionnaire [20].

Figures 3-6 show the evaluation results of every COBIT domains.

7.1. Analyzing the Results of Case Study

According to the definition of software quality described in literature review, process related to quality shown in the table of CMM process in Figure 1 has been come to consideration then COBIT evaluation result from each process in Figures 3-6 has been searched and shown in column diagram.

Then to drill down the problem, circumstances of the process maturity level have been analyzed and the effectiveness rate of processes on each other and the Project quality are examined.

The processes related to the software quality have two categories as described. The processes related to customer consent and the process related to the project management. Although these two categories are considered separately, in one of the COBIT processes they coincide and influence on software quality.

At first, the maturity level of each group of above processes is defined. Next, we achieve to the process related to every two categories and can see the relation between maturity of this process and those two categories of maturity process.

This relation shows that software quality is influenced by two factors: customer discontent and lack of coordination between technical groups.

• 7.2. Some Reasons of Customer Discontent Are

• The needs informally discuss in individual groups meeting and requirements are sometimes documented.

• Individuals define the solutions in response to vendor offering.

• Applications are typically obtained based on IT with no consideration of actual requirements.

• Every new application has infrastructure changes without any overall plan.

• The test environment production is the environment.

• Documentation is distributed to groups inconsistently.

• There is no consideration of update the documentation and procedures.

• Different systems and business units have not integration of procedures.

• Individual teams produce the training materials and quality depends on them.

• Documents and procedures are corrected reactively.

• Changes made with no control because change management process is not defined.

• There is no awareness the change management benefits.

• The approaches taken for testing is vary and the individual teams do that.

Company has problem in defining the strategic goals. It must be performed more certainly and the organization does not maintain an awareness of available technology solutions potentially relevant to its business.

There is minimal structured research or analysis of available technology.

The maturity level of IT investigation is 1.7. It shows that company didn’t use its resources. Indeed, the corporation resources are wasted.

• 7.3. Some Reasons of Scheduling Disruption

• The project and business objectives are not defined align.

• The project management process is reactive.

• The staff roles and responsibilities are not assigned and informal.

• The customer requirement is response inconsistently.

• Individual knowledge and skills make decisions.

• The relationships with steering committees, internal audit and vendor management are not formalized.

• The responsibilities, accountability and Process ownership are not defined.

• There is no IT control environment.

• Addressing the requirements of the environment is poor.

• An Overall development framework does not exist.

• The quality program is not defined by the senior management and IT staff members.

• There is no system development life cycle (SDLC) methodology and QMS planning process.

• Lack of risk assessment for processes and business decisions.

• Individual managers decide regarding the use of project management techniques.

• User involvement in defining projects is poor.

7.4. Drill down the Process Influence on Quality

Analyzing the evaluation can determine the effective factors on quality. Figure 7" target="_self"> Figure 7 shows, the effective process respect to their effectiveness on each other. The sequence of these process finish in quality management.

Studying this book shows the base of problems.

Figure 7 represents the process influence on quality has been studied in customers’ and developers’ view.

The processes sequences have been continued to “quality management”.

Drill down have been begun from last process, called “quality management”.

The factors influences on quality management are product engineering process, project planning and quality assurance Every factor has been described in follow:

1) Product engineering influences on quality management and has 1.4 maturity levels. Ignorance of the modern technology during project definition and system design are some factors of undesirable process maturity level.

2) Project planning: This process is depending on these three processes, requirement management, intergroup coordination and risk management.

a) Requirement management: “change management” is one of the key processes of requirement management and have 0.6 maturity level and leads to reduction of “requirement management” process maturity level to 1.2.

b) Intergroup coordination: maturity level = 1.35.

c) Risk management: maturity level = 0.7.

3) Quality assurance: maturity level = 1.4.

Among the mentioned effective process, “intergroup coordination” has special complications and also influence on other processes which will have been described. The maturity level of “intergroup coordination” is 1.35.

This process depends on following processes:

Third party services management: The goal is satisfaction of third party services and cost and benefits transparency as well.

Definition of the IT processes, organization and relationships (PO4): The goal is implicating quickness in order to meet business strategy.

Communication with aims and direction of management (PO6): The goal is to provide detailed and on time information about current and future services and risks and responsibilities as well.

The processes defined in PO4 and PO6 domains in COBIT framework emphasize that IT processes and its organizations and relationships must be defined and organized and risk and responsibilities must be defined and

aligned with management. The processes of DS2 domain emphasizes on determination third party services transparency. In CMM all definitions are in planning phase.

These definitions are: estimate project domain, lifecycle definition, Resource project planning, planning for essential knowledge and skills, project plan. The study of processes defined in CMM planning phase show that the IT processes haven’t been defined correctly in this case and experts don’t assign properly to the project. This means that PO4, PO6, DS2 haven’t been considered. In large scale project, consultation by an expert team is vital. It shows the top down view.

The expert group defining the fellow groups and their tasks and organizing the relationship between groups let them assign properly. It also decreases deviation between planning and implementation.

In importance of the mentioned processes in this project “HR management” is considered significantly. However HR management has two maturity level the project is not successful.

The holistic approach in project management and focusing on each technical team in their era are the reasons that the project can’t be finished despite experts working and spending time and cost since 5 years and the company also paid fine.

The influence of “intergroup coordination” on risk management The “intergroup coordination” belongs to CMM third level processes.

This section tries to mention the process in 4^th and 5^th level which are depend on this process. These processes are:

4^th level—“quantitative process management”;

4^th level—“software quality management”;

5^th level—“process change management”: relate to the PO3, PO8, AI3 processes;

5^th level—“defect prevention” = maturity level = 1.15.

The evaluation shows that the maturity level of these processes is lower than 2^th and 3^th level processes.

Analyzing maturity processes done by proposed model and the factors related to “software quality management” as a chain expresses that, the project case study isn’t base on proper method.

That’s why the project had been failed in spite of having Human Resources and project management standards.

Factors influenced on quality can be identified as follow: neglecting the users’ needs, neglecting the new technology in developing an acceptable product on customer’s view, lack of coordination in different groups and organizations, the holistic approach, neglecting risk management and defect prevention, that last two factors are related to each other.

Figures 8 and 9 show the two factors influenced on quality: The factors influenced on customers’ consent and the factors influenced on project management.

Figure 8 shows the factors influenced on customers’ consent which are:

• Identify Automated Solutions;

• Install and Accredit Solutions and Changes;

• Change management;

• Procure IT Resources;

• Enable Operation and Use;

• Acquire and Maintain Technology Infrastructure;

• Acquire and Maintain Application Software.

Between the above processes, “Change Management” process has the lowest maturity level and “Procure IT Resources” have the highest maturity level.

Figure 9 shows the factors influenced on project management which are:

• Define a Strategic IT Plan;

• Manage Project;

• Assess and Manage IT Risks;

• Manage Quality;

• Communicate Management Aims and Direction;

• Define the IT Processes, Organisation and Relationships;

• Determine Technological Direction;

• Between the above processes, “Assess and Manage IT Risks” process have the lowest maturity level and “Manage Project” process have the highest maturity level.

8. Conclusions

In this paper the factors affect on software quality considered and hybrid model of CMM and COBIT proposed.

CMM defines the processes for software production and COBIT presents a framework to evaluate IT governance on organization and in this research, it has been employed to evaluate CMM processes.

The results of mapping hybrid model of CMM and COBIT in case study show that the company encountered with problem in meeting customers’ needs and in organizing Human resource as well. These two factors are the reasons of project failure.

Lack of proper management in transformations, and passive attitude regarding to documentation and test are the reasons of low quality in customers’ view, in addition they influenced on project management activities such as: time and cost planning. There is a hidden factor in “project management” delay.

This hidden factor is the lack of coordination in different departments related in one project. It is necessary to consider the coordination of different technical groups of the project in planning phase.

In this way, senior managers of the groups have to consider their commitments in scheduling. The existing expert team included of experts in all technical groups is completely aware of all subsystems and their scheduling is essential. In this research, the results show that: software developing and meeting customer needed in multi groups’ projects encountered with a problem and in case that companies neglect this issue, software quality has been influenced completely.

REFERENCES