The success of a software development project requires the early objective determination of the project’s correctness or incorrectness and the identification of the most effective solution for project management. However, few studies have been conducted on the reliable quantitative early judgment of correctness or incorrectness. In recent years, the collection and accumulation of actual attribute data from Japanese domestic software development projects have been conducted by the Software Engineering Centre of the Information-Technology Promotion Agency of Japan. In a previous article, we proposed a precise definition of project correctness or incorrectness and identified the important factors in successful projects; we also proposed a quantitative decision-making method for judging project correctness or incorrectness objectively and quantitatively on the basis of discriminant analysis using project completion attribute data. On the basis of the previous results, we propose a quantitative decision-making technique for the early judging of project correctness or incorrectness based on the attribute data of design stage as early stage of development.
In our information-oriented society, many software development projects are being conducted with organizational management issues. The criteria for the success of a project were suggested in the previous study [
Judgments of correctness or incorrectness also do not use the attributes of project management scales, such as actual man-hours or number of malfunctions after software development completion that might have influenced correctness or incorrectness.
On the other hand, the objective judgment of success is extremely important because we need to rotate on a Plan-Do-Check (PDC) cycle to improve project management in an organization and enhance project productivity. However, the objective judgment of success is very difficult because collecting quantitative attribute data of projects during development is difficult. Requirements, such as delivery time and customer demand during development, also change. Furthermore, it is important to put in place measures to remove the cause for leading a project to correctness by studying the identified causes of project incorrectness. Consequently, a project fails to be completed as a result of failing to achieve a judgment of success. If we judge the failed project as correct, a PDC cycle might not occur to improve project management.
To resolve these issues, we attempted in our previous studies [
On the other hand, the collection and accumulation of actual data from more than 3325 (Software engineering center of the information-technology promotion agency Japan, 2014) software development projects of Japanese domestic information service companies have been conducted in recent years by the Software Engineering Centre of the Information-Technology Promotion Agency of Japan (IPA/SEC) as a national project for the purpose of improving project management [
For this reason, we defined objective criteria and studied the relationships between the success factors and project success in previous study [
We also quantified for the correctness or incorrectness of the quantification standards of the success degree of project management quality of IPA/SEC after-project completion on the basis of this standard. We evaluated the precision of diagnosis methods by analyzed the success degree and quantitative relations with the attribute data of project scale after the completion of development [
In this article, we propose the concept of project correctness or incorrectness in Section 2, summarize the article in Section 3, present the results of the verification of the judgment techniques in Section 4, and draw conclusions and propose future works in Section 5.
The concept of planning and management based on the framework of whole organized project management is shown in
The quality of planning of a software development project is thought to be the precision of the planned value of various attributes, such as the target quality, delivery time, cost, development scale, number of personnel, and man-hours described. It is thought that the actual value of attribute data after project completion is influenced by the results of the development and the process quality of the project. On the other hand, planning attributes can influence execution attributes. Furthermore, the attributes of the final result of a project are influenced by the execution attributes, as shown in
Consequently, it is thought that the actual value of the attribute data of project planning/execution and the correctness/incorrectness of a project are closely related. The final judgment of project correctness or incorrectness is typically conducted on the basis of the results of a hearing or questionnaire survey by the stakeholders concerned with the project-after-project completion, generally from the viewpoint of whether a project achieved its purpose. However, in the previous study, the judgment of correctness or incorrectness was suspect because the definition of correctness or incorrectness as a purpose variable was vague. Therefore, in the current study, we performed a judgment of the correctness or incorrectness of a project in terms of whether the project was accomplished to
the planned degree from the viewpoint of the difference between the planned and actual degrees (
In this study, the case in which a project achieved its plan was defined as primary success, and the case in which the outcomes of a project satisfied the demands of customers was defined as secondary success.
From
The criteria of correctness or incorrectness of the actual result of a project that involved specification change during development are shown in
Result of judgment | Satisfy a objective of plan | Satisfy a customer needs | ||||||
---|---|---|---|---|---|---|---|---|
Unachieved | Achieved | Unachieved | Achieved | |||||
× | ○ | × | ○ | |||||
Specification changes | ○ | × | ○ | × | ○ | × | ○ | × |
Impossible judgment | ○ | × | ○ | × | ○ | × | ○ | × |
Primary incorrectness | -- | ● | -- | × | -- | × | -- | × |
Primary correctness | -- | × | -- | ● | -- | ● | -- | ● |
Secondary incorrectness | -- | -- | -- | -- | -- | ● | -- | × |
Secondary correctness | -- | -- | -- | -- | -- | × | -- | ● |
○: Yes; ×: No; ●:Yes; ×: No.
Judgment of success correctness or incorrectness of planning | ||
---|---|---|
Correctness | Incorrectness | |
Quality | The grounds of the planned values of the attributes of scale are clear and have been examined for feasibility. | The grounds of planned values of scale of attribute data were not clear or of a feasible nature have not been examined, or planning did not occur. |
Delivery | ||
Cost | ||
General | All of the planned values of quality, delivery, and cost are successfully correctness. | Either of the planned value of quality, delivery or cost are judged as incorrectness. |
Correctness or incorrectness of planning | Specification change | Judgment of correctness or incorrectness of actual result | |||
---|---|---|---|---|---|
Correctness | “Quality”, “Delivery”, “Cost” | ||||
Actual value > Planned value | Actual value = Planned value | Actual value < Planned value | |||
No | Incorrectness | Correctness | Excellence | ||
Yes | Judgment impossible | Excellence | Super excellence | ||
General | |||||
Either of the actual result of “quality”, “delivery” or “cost” are Incorrectness | All of the actual result of “quality”, “delivery” and “cost” are Correctness | All of All of the planning of “quality”, “delivery” and “cost” are Correctness and one over of Excellence are included | |||
No | Incorrectness | Correctness | Excellence | ||
Yes | Judgment impossible | Excellence | Super excellence | ||
Incorrectness | Judgment impossible | ||||
In this study, even if the actual value of attribute data exceeded the planned value of the project in that the planning “failed” from the criteria that we showed for correctness or incorrectness of planning in
On the other hand, we did not judge a project to have necessarily achieved correctness even if the actual value was less than the planned value when it was higher than the possible achievable value, and the planned value was set.
In this case, if the planned value is reasonable, it might lose an expected original advantage that would have been expected to be provided by the project.
In this study, we used the fault density (number of faults/man-hour) within six months after delivery as the attribute evaluating quality. From
We considered the correctness or incorrectness of a project as incorrectness when the actual value was beyond the planned value. Thereafter, we did not necessarily judge the correctness or incorrectness of a project as incorrectness when the actual value regarding quality, delivery time, or cost exceeded the planned value if specification changes occurred during development because the actual value can increase with additional work if specification changes occurred.
On the other hand, we judged the correctness or incorrectness of a project as excellence in the case in which the actual value was equal to or less than the planned value because we thought that the project achieved the planned value through the effort and inventive ideas of project members.
We judged the general correctness or incorrectness of a project as correctness in the case in which all of the actual values of quality, delivery time, and cost of correctness or incorrectness were judged as correctness if specification changes occurred less during development. Furthermore, we judged the general correctness or incorrectness of a project as incorrectness in the case in which one actual value of quality, delivery time, or cost of correctness or incorrectness of the project was judged as incorrectness if specification changes occurred less during development.
We judged the general correctness or incorrectness of a project as excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness was judged as correctness or excellence or judged as more than one excellence if the specification changes occurred less during development. On the other hand, we did not necessarily judge the general correctness or incorrectness of a project as incorrectness in the case in which one actual value of quality, delivery time, and cost of correctness or incorrectness was judged as incorrectness if specification changes occurred during the development. Moreover, we judged the general correctness or incorrectness of a project as excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness of the project was judged as excellence or correctness or judged more than once as excellence if specification changes occurred less during development.
In this study, we judged the general correctness or incorrectness of a project as super excellence in the case in which every actual value of quality, delivery time, and cost of correctness or incorrectness was judged as excellence. However, in such a case, we defined the general correctness or incorrectness of a project as excellence because we treated super excellence as excellence when specification changes occurred less during development.
In this study, we introduced the quantification indicator of success degree of IPA/SEC for the evaluation of qualitative correctness or incorrectness. We thought that the evaluation result based on the evaluation criteria of correctness or incorrectness of quality, delivery time, and cost was more likely to show the correctness or incorrectness of a project concretely and objectively than the conventional questionnaire survey. Furthermore, we defined the criteria of correctness or incorrectness of primary success defined in
For the success degree, we set a value for incorrectness that was lower than for correctness; furthermore, we set a value for excellence that was higher than correctness on the basis of the degree of difference of the actual and planned values of attribute data (
1) Quantification of Success Degree of Planning and Execution
We defined the value of success degree that quantified the qualitative evaluation of the result of correctness or incorrectness of a project intended from the viewpoint of quality, delivery time, and cost on the basis of the quantification standard. The quantification standard of success degree of quality, delivery time, and cost concerning the planning and execution of a project is shown in
Evaluation of success correctness or incorrectness of planning | Evaluation of success correctness or incorrectness based on the planned value and actual value of project that specifications change did not occur during development | ||||||||
---|---|---|---|---|---|---|---|---|---|
121_ Planning of Quality | 122_Planning of Delivery | 120_Planning of Cost | Judge | 124_Actual result of Quality | Judge | 125_Actual result of Delivery | Judge | 123_Actual result of Cost | Judge |
x = 1.0: Target of Quality is clear and feasibility has been examined. | y = 1.0: Grounds of delivery time is clear and feasibility has been examined. | z = 1.0: Grounds of cost is clear and feasibility has been examined. | R | x = 1.2: Fewer than planned value more than 20% (0 ≤ x ≤ 80) | E | y = 1.2: Earlier than delivery time. (y < Planned Delivery time) | E | z = 1.2: Accomplished with cost that is lower than 20% of planned value. (0 ≤ z ≤ 90) | E |
x = 1.0: Fewer than planned value. (80 < x ≤ 100) | R | y = 1.0:According to Delivery time (y = Planned Delivery time) | R | z = 1.0: According to planned value. (Less than ±10%) (90 < z < 110) | R | ||||
x = 0.8: Excess within 50% of planned value. (100 < x ≤ 150) | W | y = 0.8: Less than 10 days late on Delivery time. (Planned Delivery time < y < Planned Delivery time +10) | W | z = 0.8: Excess within 30% of planed value. (110 ≤ z ≤ 130) | W | ||||
x = 0.6: Excess within 100% of planned value. (150 < x ≤ 200) | y = 0.6: Less than 30 days late on Delivery time. (Planned Delivery time +10 ≤ y< Planned Delivery time +30) | z = 0.6: Excess within 50% of planned value. (130 | |||||||
x = 0.2: Excess more than 100% of planned value. (200 < x) | y = 0.2: More than 30 days late on Delivery time. (Planned Delivery time +30 ≤ y) | z = 0.2: Excess more than 50% of planned value. (150 < z) | |||||||
x = 0.0: Quality target is not clear or feasibility is not examined. | y = 0.0: Grounds of the time of delivery plan are not clear and feasibility is not examined. | z = 0.0: Grounds of the cost calculation are not clear or feasibility is not examined. | W | --------- | I | --------- | I | --------- | I |
x, y or z = 0.0: No planning | I | I | I |
Note: x: Quality; y: Delivery; z: Cost Judgment Result = E: Excellence; R: Correctness; W: Incorrectness; I: Impossible to judge (Definition of the attribute of the project references, pp. 359-381) of the Appendix A data [
To show this in
Similarly, in response to the evaluation result of execution, we settled on values from 0.2 to 1.2. The description of the evaluation result of the correctness or incorrectness of project planning regarding quality, delivery time, and cost is shown in
2) Judgment of Correctness or Incorrectness Based On the Success Degree
We judged the actual results of the success degree of quality to be correctness in the case in which every success degree of planning such as 121, 122, and 120 were +1.0 and 124 was +1.0, as defined on the basis of the judgments in
First, we extracted the reliable attribute data of projects that were qualitative evaluation results of correctness or incorrectness and related data from the data of IPA/SEC. In this study, we adopted the approach that predicts the success degree from the actual data of project design. Second, we quantified the success degree on the basis of the criteria of project success (
Attribute of design stage | Description | Coefficient of actual success degree | |
---|---|---|---|
PSD | PST_ | ||
10052 | Actual performance man-hour _of requirement definition | 0.0029 | 0.1412 |
10053 | Actual performance man-hour _of basic design | −0.2381 | −0.0072 |
10054 | Actual performance man-hour _of detail design | −0.1857 | −0.0147 |
1005234 | Actual performance man-hour total of design = 10052 + 10053 + 10054 | −0.1974 | 0.0069 |
100523 | =10052 + 10053 | −0.2035 | 0.0364 |
100534 | = 10053 + 10054 | −0.2071 | −0.0119 |
100524 | = 10052 + 10054 | −0.1718 | 0.0149 |
5249 | Actual review indication number of basic design | −0.4511 | −0.3745 |
5249dm | =5249/10053 | −0.4733 | −0.7891 |
5249dtm | = 5249/1005234 | −0.4785 | −0.7920 |
5249rbm | = 5249/100523 | −0.4534 | −0.8178 |
5249bdm | = 5249/100534 | −0.4129 | −0.6395 |
5249rdm | = 524/100524 | −0.1753 | −0.0251 |
The definition of the attribute of the project references ( [
to predict the success degree of a project on the basis of the attribute of success factors of design stage, as shown in
The process of this study is as follows:
[Step 1] We distinguished the correctness or incorrectness of a project into correctness and incorrectness.
If the success degree ≥3.0 and <3.0, the project is considered to have achieved correctness and incorrectness, respectively, according to the quantification standard of
[Step 2] We performed correlation analysis with the success degree defined at Step 2 and the attributes that have influence on the correctness or incorrectness of a project included in the actual IPA/SEC attribute data. We then identified the success factors of a project, as shown in
Attribute of explanation valuables | Partial regression coefficient | P-value | ||
---|---|---|---|---|
5249dm = 5249/10053 | r2 | −1.5763 | 0.0179 | |
5249rdm_ = 5249/100524 | r1 | −0.0065 | 0.1918 | |
Constant | r0 | 0.2796 | ------ | |
Result of multiple regression analysis | R: multiple correlation coefficient | 0.5440 | ||
R2: decision coefficient | 0.2960 | |||
F-value | 3.7838 | 0.0425 | ||
F0 (m1, m2, 0.05 ) m1 = 2, m2 = 20 | 3.4928 | |||
F0 (m1, m2, 0.01) m1 = 2, m2 = 20 | 5.8489 | |||
psdi: Predicted value of the general success degree of project rn: Partial regression coefficient ( n = 0 ~ 2).
Success correct or incorrect | Prediction model | Result of multiple regression analysis | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
No | Actual | Prediction | Deference | Constant | Partial regression coefficient | Explanation valuable | Multiple correlation coefficient | Decision coefficient | F-value | P-value | |||
i | RSDi | psdi | rsdi | ei | r0 | r1 | r2 | 5249dm | 5249rdm | R | R2 | F | P |
1 | 1 | −0.114 | 0 | 1 | 0.2595 | −1.6412 | −0.0064 | 0.2083 | 5.0000 | 0.5730 | 0.3283 | 4.1551 | 0.0339 |
2 | 1 | −0.023 | 1 | 0 | 0.2894 | −1.5792 | −0.0073 | 0.1429 | 12.0000 | 0.5835 | 0.3404 | 4.1291 | 0.0358 |
3 | 0 | −0.007 | 0 | 0 | 0.3018 | −1.5336 | −0.0069 | 0.2000 | 0.2632 | 0.5530 | 0.3058 | 3.7447 | 0.0449 |
4 | 1 | 0.090 | 1 | 0 | 0.2538 | −1.5246 | −0.0065 | 0.0645 | 10.0000 | 0.5390 | 0.2905 | 3.4807 | 0.0541 |
5 | 1 | 0.193 | 1 | 0 | 0.2549 | −1.5188 | −0.0062 | 0.0408 | 0.0179 | 0.5265 | 0.2773 | 3.2607 | 0.0633 |
6 | 0 | 0.100 | 0 | 0 | 0.3180 | −1.5993 | −0.0071 | 0.1333 | 0.2632 | 0.5744 | 0.3299 | 4.1848 | 0.0333 |
7 | 0 | 0.081 | 0 | 0 | 0.3091 | −1.6451 | −0.0055 | 0.0520 | 26.0000 | 0.5678 | 0.3224 | 4.0437 | 0.0366 |
8 | 1 | 0.168 | 1 | 0 | 0.2547 | −1.5075 | −0.0065 | 0.0148 | 10.0000 | 0.5292 | 0.2801 | 3.3067 | 0.0612 |
9 | 1 | 0.022 | 1 | 0 | 0.2534 | −1.5706 | −0.0061 | 0.1471 | 0.1064 | 0.5490 | 0.3015 | 3.6682 | 0.0474 |
10 | 0 | 0.002 | 0 | 0 | 0.3030 | −1.5398 | −0.0069 | 0.1918 | 0.8311 | 0.5547 | 0.3076 | 3.7770 | 0.0439 |
11 | 1 | 0.226 | 1 | 0 | 0.2566 | −1.5115 | −0.0063 | 0.0039 | 4.0000 | 0.5234 | 0.2739 | 3.2069 | 0.0658 |
12 | 1 | 0.197 | 1 | 0 | 0.2551 | −1.5183 | −0.0062 | 0.0378 | 0.0352 | 0.5261 | 0.2768 | 3.2525 | 0.0637 |
13 | 0 | −1.757 | 0 | 0 | 0.4141 | −2.8916 | −0.0074 | 0.7499 | 0.4148 | 0.5916 | 0.3500 | 4.5777 | 0.0257 |
14 | 1 | 0.121 | 1 | 0 | 0.2541 | −1.5042 | −0.0067 | 0.0171 | 16.0000 | 0.5346 | 0.2858 | 3.4008 | 0.0572 |
15 | 0 | −0.141 | 0 | 0 | 0.2869 | −1.4850 | −0.0068 | 0.2872 | 0.1555 | 0.5311 | 0.2821 | 3.3398 | 0.0598 |
16 | 0 | 0.185 | 1 | 1 | 0.2432 | −1.5066 | 0.0004 | 0.0653 | 97.0000 | 0.5143 | 0.2645 | 3.0574 | 0.0734 |
17 | 0 | −0.255 | 0 | 0 | 0.2792 | −1.4768 | −0.0066 | 0.3611 | 0.1741 | 0.5174 | 0.2677 | 3.1066 | 0.0708 |
18 | 1 | 0.257 | 1 | 0 | 0.2581 | −1.5161 | −0.0062 | 0.0008 | 0.0004 | 0.5207 | 0.2711 | 3.1620 | 0.0680 |
19 | 0 | 0.278 | 1 | 1 | 0.3495 | −1.7391 | −0.0075 | 0.0407 | 0.0348 | 0.6122 | 0.3748 | 5.0954 | 0.0185 |
20 | 1 | 0.174 | 1 | 0 | 0.2543 | −1.5214 | −0.0062 | 0.0528 | 0.0232 | 0.5286 | 0.2794 | 3.2955 | 0.0617 |
21 | 0 | 0.064 | 0 | 0 | 0.3120 | −1.5733 | −0.0071 | 0.1569 | 0.0976 | 0.5666 | 0.3210 | 4.0186 | 0.0372 |
Number of predictive error | 3 | i: Sample No (i = 1 ~ 2) | F0 (2, 20, 0.05) = 3.4928 | ||||||||||
HR hitting ratio (%) | 86 | F0 (2, 20, 0.01 ) = 5.8489 |
If (−0.10 < psdi < +0.10) t hen rsdi = RSDi.
Success correctness or incorrectness | Attribute data of project | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
No | Actual | Prediction | Difference | Evaluation result of success degree | Reliability of data | |||||
Predicted value | General | Quality | Delivery | Cost | ||||||
i | RSDi | psdi | rsdi | ersdi | PSTi | 124i | 125i | 123i | 102 | 10085 |
1 | 0 | −0.290 | 0 | 0 | 2.8 | 0.80 | 1.00 | 1.00 | A | B |
2 | 0 | 0.243 | 1 | 1 | 2.8 | 0.80 | 1.00 | 1.00 | B | B |
3 | 0 | 0.069 | 0 | 0 | 2.8 | 0.80 | 1.00 | 1.00 | A | B |
4 | 1 | 0.048 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | B | B |
5 | 0 | 0.177 | 1 | 1 | 2.6 | 0.80 | 1.00 | 0.80 | B | B |
6 | 0 | −0.473 | 0 | 0 | 2.6 | 0.80 | 1.00 | 0.80 | A | B |
7 | 0 | −0.036 | 0 | 0 | 2.8 | 0.80 | 0.80 | 1.20 | B | B |
8 | 0 | 0.198 | 1 | 1 | 2.8 | 0.80 | 0.80 | 1.20 | B | B |
9 | 1 | 0.187 | 1 | 0 | 3.2 | 1.20 | 1.00 | 1.00 | B | B |
10 | 0 | −0.023 | 0 | 0 | 2.2 | 0.80 | 0.60 | 0.80 | A | A |
11 | 1 | 0.275 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | B | B |
12 | 0 | −0.173 | 0 | 0 | 2.8 | 1.00 | 1.00 | 0.80 | A | B |
13 | 1 | 0.267 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | B | B |
14 | 1 | 0.278 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | B | A |
15 | 0 | 0.032 | 0 | 0 | 2.8 | 0.80 | 1.00 | 1.00 | A | A |
16 | 1 | −0.646 | 0 | 1 | 3.0 | 1.00 | 1.00 | 1.00 | A | B |
17 | 1 | 0.253 | 1 | 0 | 3.4 | 1.20 | 1.00 | 1.20 | A | B |
18 | 1 | 0.054 | 1 | 0 | 3.4 | 1.20 | 1.00 | 1.20 | B | B |
19 | 1 | 0.279 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | B | A |
20 | 1 | 0.274 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
21 | 1 | −0.049 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | B |
22 | 1 | 0.178 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
23 | 1 | 0.242 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
24 | 1 | 0.256 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | B |
25 | 1 | 0.219 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
26 | 1 | 0.196 | 1 | 0 | 3.2 | 1.00 | 1.00 | 1.20 | A | A |
27 | 0 | 0.215 | 1 | 1 | 2.8 | 0.80 | 1.00 | 1.00 | A | A |
28 | 1 | −0.643 | 0 | 1 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
29 | 1 | 0.064 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | A |
30 | 0 | 0.107 | 0 | 0 | 2.6 | 0.60 | 1.00 | 1.00 | A | B |
31 | 1 | 0.215 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | B |
32 | 0 | −0.904 | 0 | 0 | 1.2 | 0.20 | 0.20 | 0.80 | A | B |
33 | 1 | 0.279 | 1 | 0 | 3.0 | 1.00 | 1.00 | 1.00 | A | B |
Number of predictive error (count) | 6 | HR : Hitting ratio = 82 (%) |
[Step 3] We extracted 54 projects that are necessary for the analysis of this study from the data of 3325 IPA/SEC projects. Furthermore, on the basis of the 10,050 actual project man-hours, we sorted the work scale of the project sequentially and generated 21 random projects other than for multiples of 3, as shown in
[Step 4] We developed plural multiple regression models to predict the success degree, as shown in
[Step 5] On the basis of the results of multiple regression analysis, we inspected the effectiveness of the prediction models for correctness or incorrectness that we developed by using the associated attributes based on the 21 project data provided by Step 3.
[Step 6] We determined the partial regression coefficient of the prediction model of success degree that was determined by the results of multiple regression analysis based on the other 20 projects other than the target project for prediction. The partial regression coefficient of the prediction model and the predicted value of the general success degree of each project are shown in
[Step 7] Furthermore, we applied the prediction model of general success degree to 33 actual projects except for the 21 projects that we used for the development of the prediction model.
The predicted value of the general success degree is shown in
[Step 8] The results of the inspection of the prediction technique and the judgment of general correctness or incorrectness are shown in
A summary of the collected attribute data of the project provided by IPA/SEC are shown in
attribute data that are necessary for the development of a prediction model for the correctness or incorrectness of a project.
We then extracted 1026 projects that had recorded the scale of projects, such as the number of persons with an average malfunction indication number to be related to the correctness or incorrectness of projects that we were able to confirm in the preceding article [
Finally, we extracted 54 projects for which basic attributes, such as development scale and development man-hours, were recorded. Furthermore, the scales of projects were greater than 20 personnel per month, and there was no data loss. On the other hand, we thought that we could not treat language differences, such as among COBOL, JAVA, or C, concerning the development scale by the same standard. Consequently, we excluded them for the analysis of the attribute data of the study.
In this article, we thought that the relations of the additional characteristics and trade-offs made ends meet in the success degree of quality, delivery time, and
cost for the general success degree of a project. We calculated the general success degree of a project from Equation (1) from the grand total of success degrees of quality, delivery time, and cost:
P S T i = 124 i + 125 i + 123 i (1)
PSTi: Actual value of general success degree of a project;
124i: Actual value of success degree of quality of a project;
125i: Actual value of success degree of delivery time of a project;
123i: Actual value of success degree of cost of a project;
i: Sample number of targeted projects for analysis (i = 1 ~ N, N = 21).
Furthermore, we distinguished the general correctness or incorrectness of a project by following our method and setting values for judging it.
We converted the success degree by using Equation (2) in consideration with the number of projects judged as correctness and incorrectness to perform linear discriminant analysis:
Y i d = N F / N ( if P S T i ≥ 3.0 ) Y i d = N S / N ( if P S T i < 3.0 ) (2)
Yid : Success degree after conversion;
N: Number of samples;
NF: Number of samples for which the result of success degree is higher than 3.0;
NS: Number of samples for which the result of success degree is below 3.0.
P S D i = + Y i d ( P S T i ≥ 3.0 )
P S D i = − Y i d ( P S T i < 3.0 )
R S D i = 1 ( P S D i ≥ 0.0 )
R S D i = 0 ( P S D i < 0.0 )
RSDi: Judgment result of the general correctness or incorrectness of the project.
In this study, we formulated multiple regression models by using explanatory variables correlated with the correctness or incorrectness of the project that we identified in
p s d i = r 0 + r 1 a 1 i + r 2 a 2 i ⋯ r n a n i (3)
psdi: Predicted value of the general correctness or incorrectness of project;
ani: Explanatory valuable (n: number);
rn: Partial regression coefficient (n = 0 ~ 3);
r0: Term of constant;
i : Number of project sample (i = 1 ~ N, N = 21).
We found the forecast of correctness or incorrectness from the predicted value of correctness or incorrectness by using the following method:
r s d i = 1 ( p s d i ≥ 0.0 )
r s d i = 0 ( p s d i < 0.0 )
rsdi: Result of the prediction of the general correctness or incorrectness of a project.
Furthermore, we calculated the hitting ratio of the general correctness or incorrectness of a project based on the judgment results and the predicted value of correctness or incorrectness, as show in
e r s d i = ( r s d i − R S D i ) 2 (4)
H R = 1 − ( ∑ i = 1 N e r s d i ) / N (5)
ersdi: Predicted error of the general correctness or incorrectness of a project;
HR: Hitting ratio of the forecast of the general correctness or incorrectness of a project.
In this article, we have identified the attributes of a project that have influence on the success degree.
The results of correlation analysis are as follows. The standard deviation and correlation coefficient greater than 0.3 for attributes of the approved general project of correctness or incorrectness are as shown in
From
Therefore, we can confirm that the success degree of a project decreases when the values of these attributes is large. Therefore, predicting the success degree of a project requires paying attention to the general success degree of a project, and there was strong correlation with the attribute concerning design review.
In this study, for the purpose of developing a prediction model for success degree, we thought that the actual values necessarily increased if specification changes occurred during development because additional work was performed. Therefore, we thought that there was no validity of the actual values of attributes data after the completion of the project and no judgment of correctness or incorrectness based on the early stage of difference from the planned value before the specification changes as the criteria of correctness or incorrectness of the project, as shown in
Therefore, we chose 21 projects from 54 projects in which a large specification change did not occur during development from the 54 projects being analyzed. We developed a prediction model of success degrees by using the 21 projects via Step 3. To develop the prediction model, we chose multiple explanatory variables from the candidates for approved attributes, such as those with correlation coefficients greater than 0.3 with the actual general success degree of projects such as PSD and PST, as shown in
In this study, we repeated the analysis by using all combinations of candidates of the explanation variable that we showed in
As a result, the final explanatory variable was a ratio, the design review indication number that we indicated in the 5249dm and 5249rdm cases. Correlation analysis showed that there was correlation between these attributes, but the dependency between each two variables did not appear in the scatter diagram.
The dispersion expansion coefficient of each variable maximum 5249dm case was 1.2, and the variance inflation factor value was less than 10.0. Hence, there were no multiplex collinearity characteristics of the explanation variable.
The result of the multiple regression analysis of the models is shown in
Furthermore, for the result of multiple regression analysis of the models in predicting the success degree of pst (
In the case in which the predicted value psdi < 0, we judged incorrectness.
In the case in which the predicted value −0.1 < psdi or psdi ≤ +0.1, we decided on a judgment result that was the actual judgment result, and we rearranged it to correctness = 1 or incorrectness = 0 for each.
From
Therefore, it is thought that we can predict the general success degree appropriately when we suppose that there is statistical validity. In cases 2, 5, 8, and 27, the judgment results of the general success of a project are determined to be correctness with the result that the prediction values of the general success degree psd are 0.243, 0.177, 0.198, and 0.215 even if the actual result is incorrectness. In cases 2 and 27, the actual evaluation result of the general success degree PST is 2.8.
In these cases, the actual evaluation result of quality might be too severe; hence, the actual value of quality 124 was 0.8. In case 5, the actual evaluation result of the general success degree PST is 2.6. In this case, the evaluation result of quality and cost might be too severe; hence, the actual value of quality 124 and cost 123 were 0.8. In case 8, the actual evaluation result of the general success degree PST is 2.8. In this case, the actual evaluation result of the success degree of both quality and deli-very might be too severe; hence, the actual values of quality 124 and delivery 125 were 0.8. However, the actual evaluation result of the success degree of cost might not be severe; hence, the actual value of the cost 123 was 1.2. In this case, the result contradicts the experience of project management because we usually fail in cost when the quality and delivery fail, and there is no success. In this case, though the quality failed on the due date of delivery, we might still forcibly complete a project within the desired cost.
On the other hand, in cases 16 and 28, the judgment results of general success of a project were determined to be incorrectness; hence, the results of the prediction of the general success degree psd are −0.646 and −0.643 even if the actual result is correctness. In cases 16 and 28, the evaluation of the actual success degree of quality 124, delivery 125, or cost 123 might not be severe; hence, the actual evaluation result of the general success degree PST was 3.0.
Thus, the abovementioned result does not contradict the experience of project management, and it is thought that we showed the effectiveness of our predictive judgment technique for suggestions without the occurrence of specification changes during development.
From
The hitting ratio of the judgment results of correctness or incorrectness of a HR was 82%. Therefore, it is thought that we can predict the success correctness or incorrectness of a project correctly at the design stage as an early stage of software development by using the proposed technique when specification changes have not occurred during development.
This study contributed the development of the decision making technique to enable the judgment of objective and quantitative correctness or incorrectness of a project based on the attribute data of design review at an early stage of software development. In this study, we confirmed the need to pay attention to attributes such as the review indication number and the performance man-hours of the design stage. The results of this study show that the proposed diagnosis technique of the correctness or incorrectness of a software development project is effective for prediction at an early stage of development. If we can predict the correctness or incorrectness of a project at an early stage of development by observing the results of a judgment and the presence of specification changes, then we can take effective measures at an early stage of development to ensure the success of the project. Furthermore, if we identify the cause of the predicted incorrectness of a project, we can take effective measures at an early stage of development to improve the productivity of software development project.
In future works, we would like to investigate experimentally an application of the proposed prediction technique to a software development project. The success factors of a software development project must be analyzed quantitatively in depth on the basis of the proposed prediction technique. Furthermore, a more useful method for assessing and improving the quality of project management needs to be developed.
Esaki, K. (2018) Design-Stage Prediction of Project Correctness or Incorrectness. Intelligent Information Management, 10, 49-68. https://doi.org/10.4236/iim.2018.102004