ABSTRACT

The success of a software development project requires the early objective determination of the project’s correctness or incorrectness and the identification of the most effective solution for project management. However, few studies have been conducted on the reliable quantitative early judgment of correctness or incorrectness. In recent years, the collection and accumulation of actual attribute data from Japanese domestic software development projects have been conducted by the Software Engineering Centre of the Information-Technology Promotion Agency of Japan. In a previous article, we proposed a precise definition of project correctness or incorrectness and identified the important factors in successful projects; we also proposed a quantitative decision-making method for judging project correctness or incorrectness objectively and quantitatively on the basis of discriminant analysis using project completion attribute data. On the basis of the previous results, we propose a quantitative decision-making technique for the early judging of project correctness or incorrectness based on the attribute data of design stage as early stage of development.

Keywords:

Software Reliability, Software Development Project, Correctness or Incorrectness, Prediction Model, Project Management, Success Degree, Judgment Technique

1. Introduction

In our information-oriented society, many software development projects are being conducted with organizational management issues. The criteria for the success of a project were suggested in the previous study [1] and many factors of project success [2] [3] [4] [5] . The relationship between project success and success factors have been suggested [6] [7] . However, the definition of a project success factor was subjective. Therefore, in a software development project, the success correctness or incorrectness of a project after completion is determined by subjective judgment based on the experience of similar past projects or the feelings of project stakeholders.

Judgments of correctness or incorrectness also do not use the attributes of project management scales, such as actual man-hours or number of malfunctions after software development completion that might have influenced correctness or incorrectness.

On the other hand, the objective judgment of success is extremely important because we need to rotate on a Plan-Do-Check (PDC) cycle to improve project management in an organization and enhance project productivity. However, the objective judgment of success is very difficult because collecting quantitative attribute data of projects during development is difficult. Requirements, such as delivery time and customer demand during development, also change. Furthermore, it is important to put in place measures to remove the cause for leading a project to correctness by studying the identified causes of project incorrectness. Consequently, a project fails to be completed as a result of failing to achieve a judgment of success. If we judge the failed project as correct, a PDC cycle might not occur to improve project management.

To resolve these issues, we attempted in our previous studies [8] [9] to develop prediction models for the reliability of outcomes on the basis of the quantitative review and testing of development attribute data from actual software.

On the other hand, the collection and accumulation of actual data from more than 3325 (Software engineering center of the information-technology promotion agency Japan, 2014) software development projects of Japanese domestic information service companies have been conducted in recent years by the Software Engineering Centre of the Information-Technology Promotion Agency of Japan (IPA/SEC) as a national project for the purpose of improving project management [10] .

For this reason, we defined objective criteria and studied the relationships between the success factors and project success in previous study [11] .

We also quantified for the correctness or incorrectness of the quantification standards of the success degree of project management quality of IPA/SEC after-project completion on the basis of this standard. We evaluated the precision of diagnosis methods by analyzed the success degree and quantitative relations with the attribute data of project scale after the completion of development [11] . Also, we developed a multiple regression model to estimate the correctness or incorrectness of a project by using the actual attribute data of projects that were reliable. Moreover, we inspected the effectiveness of estimation models [12] . However, the prediction of project correctness or incorrectness was not possible from the design stage as an early stage of development. Given that the prediction of project correctness or incorrectness at the design stage is necessary for the improvement of project productivity for performance measures, we can improve the success degree and productivity of a project by taking measures to improve the project if we can predict correctness or incorrectness at the design stage. In the current study, we applied the same approach as in the preceding study [11] to the design stage of a project and developed a judgment technique to predict the project correctness or incorrectness at the design stage. We also inspected the prediction technique for the diagnosis of project correctness or incorrectness. In this study, we adopted the approach of multivariate analysis based on the preceding study [11] [12] .

In this article, we propose the concept of project correctness or incorrectness in Section 2, summarize the article in Section 3, present the results of the verification of the judgment techniques in Section 4, and draw conclusions and propose future works in Section 5.

2. Concept of Project Quality

2.1. Framework of Management of Software Development Project

The concept of planning and management based on the framework of whole organized project management is shown in Figure 1. The square boxes located around the process in the figure indicate the input resources, constraints, supporting infrastructures, and process outcomes. The box located at the center shows the project execution process. The direction of the larger arrow shows the relationships between the process and its cause.

The quality of planning of a software development project is thought to be the precision of the planned value of various attributes, such as the target quality, delivery time, cost, development scale, number of personnel, and man-hours described. It is thought that the actual value of attribute data after project completion is influenced by the results of the development and the process quality of the project. On the other hand, planning attributes can influence execution attributes. Furthermore, the attributes of the final result of a project are influenced by the execution attributes, as shown in Figure 1.

Consequently, it is thought that the actual value of the attribute data of project planning/execution and the correctness/incorrectness of a project are closely related. The final judgment of project correctness or incorrectness is typically conducted on the basis of the results of a hearing or questionnaire survey by the stakeholders concerned with the project-after-project completion, generally from the viewpoint of whether a project achieved its purpose. However, in the previous study, the judgment of correctness or incorrectness was suspect because the definition of correctness or incorrectness as a purpose variable was vague. Therefore, in the current study, we performed a judgment of the correctness or incorrectness of a project in terms of whether the project was accomplished to

the planned degree from the viewpoint of the difference between the planned and actual degrees (Figure 1), because the collection of IPA/SEC attribute data was limited to the time from the beginning of the project to the end.

2.2. Concept of Correctness or Incorrectness of a Project

In this study, the case in which a project achieved its plan was defined as primary success, and the case in which the outcomes of a project satisfied the demands of customers was defined as secondary success.

Table 1 shows the relationships among the correctness or incorrectness of primary success, secondary success, and the results of diagnosis. This table also shows the relationships among primary successes, secondary successes, and occurrences of specification changes.

2.3. Judgment Criteria of Correctness or Incorrectness of Project

Table 2 shows the criteria (Software engineering center of the information-technology promotion agency Japan, 2014) for the correctness or incorrectness of project planning.

From Table 2, we judged planning to have achieved correctness when both the feasibility of planning and the suitability of the planned value of attribute data were clear. We judged the planning to have achieved incorrectness when the reliability of planning was low if the possibility of the planned value was as described even if the plan’s own body did not exist or was unknown.

The criteria of correctness or incorrectness of the actual result of a project that involved specification change during development are shown in Table 3.

○: Yes; ×: No; ●：Yes; ×: No.

In this study, even if the actual value of attribute data exceeded the planned value of the project in that the planning “failed” from the criteria that we showed for correctness or incorrectness of planning in Table 2, we did not judge the results of the project to have achieved incorrectness; if an unrealizable low value was defined, we cannot judge it to have necessarily achieved incorrectness even if the actual value of the project exceed the planned value.

On the other hand, we did not judge a project to have necessarily achieved correctness even if the actual value was less than the planned value when it was higher than the possible achievable value, and the planned value was set.

In this case, if the planned value is reasonable, it might lose an expected original advantage that would have been expected to be provided by the project.

In this study, we used the fault density (number of faults/man-hour) within six months after delivery as the attribute evaluating quality. From Table 3, it was thought that the correctness or incorrectness of a project could not be judged regardless of the major and minor aspects of the planned and actual values if project planning failed. Furthermore, we judged the correctness or incorrectness of a project as correctness when the actual value of an attribute regarding quality, delivery time, and cost was equal to the planned value under the assumption that the planned value was reliable and specification changes did not occur during development.

We considered the correctness or incorrectness of a project as incorrectness when the actual value was beyond the planned value. Thereafter, we did not necessarily judge the correctness or incorrectness of a project as incorrectness when the actual value regarding quality, delivery time, or cost exceeded the planned value if specification changes occurred during development because the actual value can increase with additional work if specification changes occurred.

On the other hand, we judged the correctness or incorrectness of a project as excellence in the case in which the actual value was equal to or less than the planned value because we thought that the project achieved the planned value through the effort and inventive ideas of project members.

We judged the general correctness or incorrectness of a project as correctness in the case in which all of the actual values of quality, delivery time, and cost of correctness or incorrectness were judged as correctness if specification changes occurred less during development. Furthermore, we judged the general correctness or incorrectness of a project as incorrectness in the case in which one actual value of quality, delivery time, or cost of correctness or incorrectness of the project was judged as incorrectness if specification changes occurred less during development.

We judged the general correctness or incorrectness of a project as excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness was judged as correctness or excellence or judged as more than one excellence if the specification changes occurred less during development. On the other hand, we did not necessarily judge the general correctness or incorrectness of a project as incorrectness in the case in which one actual value of quality, delivery time, and cost of correctness or incorrectness was judged as incorrectness if specification changes occurred during the development. Moreover, we judged the general correctness or incorrectness of a project as excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness of the project was judged as excellence or correctness or judged more than once as excellence if specification changes occurred less during development.

In this study, we judged the general correctness or incorrectness of a project as super excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness was judged as excellence. However, in such a case, we defined the general correctness or incorrectness of a project as excellence because we treated super excellence as excellence when specification changes occurred less during development.

2.4. Quantification Standard of Success Degree

In this study, we introduced the quantification indicator of success degree of IPA/SEC for the evaluation of qualitative correctness or incorrectness. We thought that the evaluation result based on the evaluation criteria of correctness or incorrectness of quality, delivery time, and cost was more likely to show the correctness or incorrectness of a project concretely and objectively than the conventional questionnaire survey. Furthermore, we defined the criteria of correctness or incorrectness of primary success defined in Table 1 on the basis of the definition in Table 2 and Table 3 because the data collection range was limited to the period through six months after completion of a project. We quantified the qualitative evaluation result of the correctness or incorrectness of each plan and the execution of quality, delivery time, and cost and defined the quantification standard of success degree, as shown in Table 4.

For the success degree, we set a value for incorrectness that was lower than for correctness; furthermore, we set a value for excellence that was higher than correctness on the basis of the degree of difference of the actual and planned values of attribute data (Table 4) for the criteria of correctness or incorrectness of Table 2 and Table 3.

1) Quantification of Success Degree of Planning and Execution

We defined the value of success degree that quantified the qualitative evaluation of the result of correctness or incorrectness of a project intended from the viewpoint of quality, delivery time, and cost on the basis of the quantification standard. The quantification standard of success degree of quality, delivery time, and cost concerning the planning and execution of a project is shown in Table 4. Moreover, we defined the actual value of the general success degree of a project from the grand total of the actual value of the success degree of quality, delivery time, and cost on the basis of Equation (1).

Note: x: Quality; y: Delivery; z: Cost Judgment Result = E: Excellence; R: Correctness; W: Incorrectness; I: Impossible to judge (Definition of the attribute of the project references, pp. 359-381) of the Appendix A data [10] .

To show this in Table 4, as a description of the evaluation result regarding the correctness or incorrectness of each planning of quality, delivery time, and cost, we set a large value such that the precision of planning would be high; we also defined +1.0 and 0.0 as the success degrees for correctness and incorrectness, respectively.

Similarly, in response to the evaluation result of execution, we settled on values from 0.2 to 1.2. The description of the evaluation result of the correctness or incorrectness of project planning regarding quality, delivery time, and cost is shown in Table 4. We set a large value when the precision of planning is high, and we set the value of success degree from 0.0 for incorrectness to +1.0 for correctness. Similarly, for the evaluation result of execution, we set the value from +0.2 to +1.2. We set a high value such that the value that we did it and then pulled the actual value from the planned value in standard with correctness as +1.0 when the planned value and actual value were equal for the large evaluation result of the correctness or incorrectness of each result of quality, delivery time, and cost. We set +1.2 as the success degree from 0.2 of incorrectness to +1.2 of excellence. From Table 4, we set a high value such that the value that we did it, and pulled actual value from the planned value in standard with correctness as +1.0 when the planned value and actual value were equal for the description of the large evaluation result of the correctness or incorrectness of each result of quality, delivery time, and cost. We also set +1.2 of excellence as the success degree from 0.2 of incorrectness.

2) Judgment of Correctness or Incorrectness Based On the Success Degree

We judged the actual results of the success degree of quality to be correctness in the case in which every success degree of planning such as 121, 122, and 120 were +1.0 and 124 was +1.0, as defined on the basis of the judgments in Table 4. We judged the actual results of correctness or incorrectness of the quality of a project as correctness in the case in which 124 received the value +1.0. We judged the actual results of correctness or incorrectness of the quality of a project as incorrectness in the case in which 124 had a value less than +1.0. We judged the actual results of correctness or incorrectness of the quality of a project as excellence in the case in which 124 had a value greater than +1.0. By using the same approach, we judged the actual results of the correctness or incorrectness of the delivery time and cost of a project. Furthermore, we judged the actual results of general correctness or incorrectness to be general correctness in the case in which every success degree of planning such as 121, 122, or 120 were greater than +1.0, and the total actual result of the success degree was equal to +3.0. We judged the actual results of general correctness or incorrectness to be incorrectness in the case in which the success degree of planning is more than +3.0, and the total actual value of the success degree is less than +3.0. We judged the actual results of general correctness or incorrectness to be excellence in the case in which every success degree of planning is greater than +3.0, and the total actual value of the success degree is greater than +3.0.

3. Implementation of the Concept

First, we extracted the reliable attribute data of projects that were qualitative evaluation results of correctness or incorrectness and related data from the data of IPA/SEC. In this study, we adopted the approach that predicts the success degree from the actual data of project design. Second, we quantified the success degree on the basis of the criteria of project success (Table 4) and then distinguished correctness or incorrectness. Furthermore, we attempted correlation analysis with the success degree of projects and the results of actual attribute data of a project design. We also identified the attributes that have strong correlation with the success degree of a project, as shown in Table 5. We thought that we would have the possibility to predict the success degree if we could identify the attribute of success factors, and we developed the multiple regression models

The definition of the attribute of the project references ( [10] Cf. definition pp. 359-381) of the Appendix A data item.

to predict the success degree of a project on the basis of the attribute of success factors of design stage, as shown in Table 6. Furthermore, we verified the effectiveness of the prediction model that predicts the success degree of a project, as shown in Table 7 and Table 8.

3.1. Process of this Study

The process of this study is as follows:

[Step 1] We distinguished the correctness or incorrectness of a project into correctness and incorrectness.

If the success degree ≥3.0 and <3.0, the project is considered to have achieved correctness and incorrectness, respectively, according to the quantification standard of Table 3 from the viewpoint of quality, delivery time, and cost and the general of the project. Furthermore, we replaced the result with two fixed-quantity values, namely, correctness as +0.4762 and incorrectness as −0.5238, on the basis of Equation (2) to develop correctness vs. incorrectness prediction model (3).

[Step 2] We performed correlation analysis with the success degree defined at Step 2 and the attributes that have influence on the correctness or incorrectness of a project included in the actual IPA/SEC attribute data. We then identified the success factors of a project, as shown in Table 5.

psd_i: Predicted value of the general success degree of project r_n: Partial regression coefficient ( n = 0 ~ 2).

If (−0.10 < psd_i < +0.10) t hen rsd_i = RSD_i.

[Step 3] We extracted 54 projects that are necessary for the analysis of this study from the data of 3325 IPA/SEC projects. Furthermore, on the basis of the 10,050 actual project man-hours, we sorted the work scale of the project sequentially and generated 21 random projects other than for multiples of 3, as shown in Table 7. The 54 projects do not have specification changes during development.

[Step 4] We developed plural multiple regression models to predict the success degree, as shown in Table 6. This prediction model was developed form the actual value of the general success degree as a purpose variable and the strong attribute correlation as the explanatory variable.

[Step 5] On the basis of the results of multiple regression analysis, we inspected the effectiveness of the prediction models for correctness or incorrectness that we developed by using the associated attributes based on the 21 project data provided by Step 3.

[Step 6] We determined the partial regression coefficient of the prediction model of success degree that was determined by the results of multiple regression analysis based on the other 20 projects other than the target project for prediction. The partial regression coefficient of the prediction model and the predicted value of the general success degree of each project are shown in Table 7. Furthermore, we calculated the hitting ratio of general correctness or incorrectness on the basis of Equation (4) and (5) from the actual and predicted the values of correctness or incorrectness. Moreover, for the purpose of improvement of prediction precision, we set the rule of the threshold to maximize the hitting ratio of correctness or incorrectness. The result of the inspection of the prediction technique of correctness or incorrectness of a project is shown in Table 8.

[Step 7] Furthermore, we applied the prediction model of general success degree to 33 actual projects except for the 21 projects that we used for the development of the prediction model.

The predicted value of the general success degree is shown in Table 8. In Table 8, none of the projects experienced specification changes. Furthermore, we predicted correctness or incorrectness by applying the threshold and the rule to maximize the hitting ratio value; we then revised the correctness or incorrectness.

[Step 8] The results of the inspection of the prediction technique and the judgment of general correctness or incorrectness are shown in Table 8.

3.2. Target Data for Analysis

A summary of the collected attribute data of the project provided by IPA/SEC are shown in Figure 2. IPA/SEC has the data of 3,325 cases of Japanese domestic software development projects, including 611 items of data that were collected from 2004 to 2014 on the basis of the waterfall software development model. However, the attribute data of the project were not necessarily being recorded, and data loss can be seen. Therefore, in this study, we used reliable project

attribute data that are necessary for the development of a prediction model for the correctness or incorrectness of a project.

Figure 3 is a process diagram of the extraction of suitable project attribute data used by the prediction model of correctness or incorrectness. First, we extracted 1650 projects for which qualitative success degrees of quality, delivery time, and cost of planning and execution of a project were recorded, as shown in Table 1.

We then extracted 1026 projects that had recorded the scale of projects, such as the number of persons with an average malfunction indication number to be related to the correctness or incorrectness of projects that we were able to confirm in the preceding article [11] . Furthermore, the design review indication number for influencing the correctness or incorrectness of projects was filled in, the consistency among attribute data was recognized by the secretariat, and the judgment (A, B) extracted data of 193 reliable projects.

Finally, we extracted 54 projects for which basic attributes, such as development scale and development man-hours, were recorded. Furthermore, the scales of projects were greater than 20 personnel per month, and there was no data loss. On the other hand, we thought that we could not treat language differences, such as among COBOL, JAVA, or C, concerning the development scale by the same standard. Consequently, we excluded them for the analysis of the attribute data of the study.

3.3. Estimation of Project Success

In this article, we thought that the relations of the additional characteristics and trade-offs made ends meet in the success degree of quality, delivery time, and

cost for the general success degree of a project. We calculated the general success degree of a project from Equation (1) from the grand total of success degrees of quality, delivery time, and cost:

$PS{T}_i={124}_i+{125}_i+{123}_i$ (1)

PST_i: Actual value of general success degree of a project;

124_i: Actual value of success degree of quality of a project;

125_i: Actual value of success degree of delivery time of a project;

123_i: Actual value of success degree of cost of a project;

i: Sample number of targeted projects for analysis (i = 1 ~ N, N = 21).

Furthermore, we distinguished the general correctness or incorrectness of a project by following our method and setting values for judging it.

We converted the success degree by using Equation (2) in consideration with the number of projects judged as correctness and incorrectness to perform linear discriminant analysis:

$\begin{array}{l}{Y}_{id}=N_F/N\left(\text{if}PS{T}_i\ge 3.0\right)\\ Y_{id}=N_S/N\left(\text{if}PS{T}_i<3.0\right)\end{array}$ (2)

Y_id : Success degree after conversion;

N: Number of samples;

N_F: Number of samples for which the result of success degree is higher than 3.0;

N_S: Number of samples for which the result of success degree is below 3.0.

$PS{D}_i=+Y_{id}\left(PS{T}_i\ge 3.0\right)$

$PS{D}_i=-Y_{id}\left(PS{T}_i<3.0\right)$

$RS{D}_i=1\left(PS{D}_i\ge 0.0\right)$

$RS{D}_i=0\left(PS{D}_i<0.0\right)$

RSD_i: Judgment result of the general correctness or incorrectness of the project.

In this study, we formulated multiple regression models by using explanatory variables correlated with the correctness or incorrectness of the project that we identified in Table 5 to estimate the correctness or incorrectness of the project by Equation (3). We then verified the effectiveness of the estimation model of a project, as shown in Table 6:

$ps{d}_i=r_0+r_1{a}_{1i}+r_2{a}_{2i}\cdots r_n{a}_{ni}$ (3)

psd_i: Predicted value of the general correctness or incorrectness of project;

a_ni: Explanatory valuable (n: number);

r_n: Partial regression coefficient (n = 0 ~ 3);

r₀: Term of constant;

${}_i$ : Number of project sample (i = 1 ~ N, N = 21).

We found the forecast of correctness or incorrectness from the predicted value of correctness or incorrectness by using the following method:

$rs{d}_i=1\left(ps{d}_i\ge 0.0\right)$

$rs{d}_i=0\left(ps{d}_i<0.0\right)$

rsd_i: Result of the prediction of the general correctness or incorrectness of a project.

Furthermore, we calculated the hitting ratio of the general correctness or incorrectness of a project based on the judgment results and the predicted value of correctness or incorrectness, as show in Table 7 and Table 8, by using Equations (4) and (5):

$ers{d}_i=\sqrt{{\left(rs{d}_i-RS{D}_i\right)}^2}$ (4)

$HR=1-\left({\displaystyle {\sum }_{i=1}^{N}ers{d}_i}\right)/N$ (5)

ersd_i: Predicted error of the general correctness or incorrectness of a project;

HR: Hitting ratio of the forecast of the general correctness or incorrectness of a project.

4. Verification of the Judgment Technique

4.1. Correlation Analysis of Project Success and Attribute

In this article, we have identified the attributes of a project that have influence on the success degree.

The results of correlation analysis are as follows. The standard deviation and correlation coefficient greater than 0.3 for attributes of the approved general project of correctness or incorrectness are as shown in Table 5. Given that it is generally said that there is no correlation if the coefficient of correlation is less than 0.2, we assumed that the threshold of correlation is greater than 0.3.

From Table 5, there was strong negative correlation among the actual values of the general success degree of a project and the attributes of project design stages such as 5249, 5249dm, 5249dtm and 5249rbm.

Therefore, we can confirm that the success degree of a project decreases when the values of these attributes is large. Therefore, predicting the success degree of a project requires paying attention to the general success degree of a project, and there was strong correlation with the attribute concerning design review.

4.2. Development of the Prediction Model of Success Degree

In this study, for the purpose of developing a prediction model for success degree, we thought that the actual values necessarily increased if specification changes occurred during development because additional work was performed. Therefore, we thought that there was no validity of the actual values of attributes data after the completion of the project and no judgment of correctness or incorrectness based on the early stage of difference from the planned value before the specification changes as the criteria of correctness or incorrectness of the project, as shown in Table 3.

Therefore, we chose 21 projects from 54 projects in which a large specification change did not occur during development from the 54 projects being analyzed. We developed a prediction model of success degrees by using the 21 projects via Step 3. To develop the prediction model, we chose multiple explanatory variables from the candidates for approved attributes, such as those with correlation coefficients greater than 0.3 with the actual general success degree of projects such as PSD and PST, as shown in Table 5. The results of multiple regression analysis of both prediction models for the success degree of psd_i and pst_i are shown in Table 6.

In this study, we repeated the analysis by using all combinations of candidates of the explanation variable that we showed in Table 5, and having performed multiple regression analysis, the F-number of the multiple regression types was largest, and the P value of the partial regression coefficient that became small used the model as shown in Table 6.

As a result, the final explanatory variable was a ratio, the design review indication number that we indicated in the 5249dm and 5249rdm cases. Correlation analysis showed that there was correlation between these attributes, but the dependency between each two variables did not appear in the scatter diagram.

The dispersion expansion coefficient of each variable maximum 5249dm case was 1.2, and the variance inflation factor value was less than 10.0. Hence, there were no multiplex collinearity characteristics of the explanation variable.

The result of the multiple regression analysis of the models is shown in Table 6. For the results of the multiple regression analysis of the models in predicting the success degree of the multiple correlation coefficient is 0.5440, and of the decision coefficient is 0.2960; the F-number is 3.7838 (F0 = 3.4928, m1 = 2, m2 = 20), and the P-number is 0.0425. Therefore, there was a significance of 5%.

Furthermore, for the result of multiple regression analysis of the models in predicting the success degree of pst (Table 6), the multiple correlation coefficient is 0.8474, the decision coefficient is 0.7181, the F-number is 22.9293 (F0 = 3.4928, m1= 2, m2 = 20), and the P-number is 0.0000. Therefore, there was a significance of 1%.

4.3. Verification of the Judgment Technique of Correctness or Incorrectness

Table 7 shows the difference between the predicted and actual judgment results of general correctness or incorrectness. In the case in which the predicted value psd_i ≥ 0, we judged correctness.

In the case in which the predicted value psd_i < 0, we judged incorrectness.

In the case in which the predicted value −0.1 < psd_i or psd_i ≤ +0.1, we decided on a judgment result that was the actual judgment result, and we rearranged it to correctness = 1 or incorrectness = 0 for each.

From Table 7, the hitting ratio of the judgment results of correctness or incorrectness of a project HR was 86%.

Table 8 shows the difference between the predicted and actual judgment results of general correctness or incorrectness. The reliability of project attribute data and the judgment results of correctness or incorrectness of details of a project, such as the quality, delivery time, and cost of projects, are shown in Table 8. We estimated the correctness or incorrectness of a project as correctness because the evaluation results value of general success degree is greater than 0.0. However, it is possible that the success degree of project quality, delivery time, or cost might have failed.

Table 8 shows that none of the projects have specification changes.

Therefore, it is thought that we can predict the general success degree appropriately when we suppose that there is statistical validity. In cases 2, 5, 8, and 27, the judgment results of the general success of a project are determined to be correctness with the result that the prediction values of the general success degree psd are 0.243, 0.177, 0.198, and 0.215 even if the actual result is incorrectness. In cases 2 and 27, the actual evaluation result of the general success degree PST is 2.8.

In these cases, the actual evaluation result of quality might be too severe; hence, the actual value of quality 124 was 0.8. In case 5, the actual evaluation result of the general success degree PST is 2.6. In this case, the evaluation result of quality and cost might be too severe; hence, the actual value of quality 124 and cost 123 were 0.8. In case 8, the actual evaluation result of the general success degree PST is 2.8. In this case, the actual evaluation result of the success degree of both quality and deli-very might be too severe; hence, the actual values of quality 124 and delivery 125 were 0.8. However, the actual evaluation result of the success degree of cost might not be severe; hence, the actual value of the cost 123 was 1.2. In this case, the result contradicts the experience of project management because we usually fail in cost when the quality and delivery fail, and there is no success. In this case, though the quality failed on the due date of delivery, we might still forcibly complete a project within the desired cost.

On the other hand, in cases 16 and 28, the judgment results of general success of a project were determined to be incorrectness; hence, the results of the prediction of the general success degree psd are −0.646 and −0.643 even if the actual result is correctness. In cases 16 and 28, the evaluation of the actual success degree of quality 124, delivery 125, or cost 123 might not be severe; hence, the actual evaluation result of the general success degree PST was 3.0.

Thus, the abovementioned result does not contradict the experience of project management, and it is thought that we showed the effectiveness of our predictive judgment technique for suggestions without the occurrence of specification changes during development.

From Table 8, six judgment results of projects 2, 5, 8, 16, 27, and 28 were different from the actual judgment results.

The hitting ratio of the judgment results of correctness or incorrectness of a HR was 82%. Therefore, it is thought that we can predict the success correctness or incorrectness of a project correctly at the design stage as an early stage of software development by using the proposed technique when specification changes have not occurred during development.

5. Concluding Remarks

This study contributed the development of the decision making technique to enable the judgment of objective and quantitative correctness or incorrectness of a project based on the attribute data of design review at an early stage of software development. In this study, we confirmed the need to pay attention to attributes such as the review indication number and the performance man-hours of the design stage. The results of this study show that the proposed diagnosis technique of the correctness or incorrectness of a software development project is effective for prediction at an early stage of development. If we can predict the correctness or incorrectness of a project at an early stage of development by observing the results of a judgment and the presence of specification changes, then we can take effective measures at an early stage of development to ensure the success of the project. Furthermore, if we identify the cause of the predicted incorrectness of a project, we can take effective measures at an early stage of development to improve the productivity of software development project.

In future works, we would like to investigate experimentally an application of the proposed prediction technique to a software development project. The success factors of a software development project must be analyzed quantitatively in depth on the basis of the proposed prediction technique. Furthermore, a more useful method for assessing and improving the quality of project management needs to be developed.

Cite this paper

Esaki, K. (2018) Design-Stage Prediction of Project Correctness or Incorrectness. Intelligent Information Management, 10, 49-68. https://doi.org/10.4236/iim.2018.102004

References