American Journal of Industrial and Business Management
Vol.04 No.11(2014), Article ID:51930,10 pages
10.4236/ajibm.2014.411073
Disclosure of the State of the Art of Performance Evaluation Applied to Project Management
Sandra Rolim Ensslin1, Leonardo Ensslin2, Rogério Tadeu de Oliveira Lacerda3, Victor Hugo Aurélio de Souza4
1Accounting Department, Federal University of Santa Catarina (UFSC), PPGC/CSE/UFSC, Florianópolis, Brazil
2Business Administration Department, Southern University of Santa Catarina (UNISUL), Florianópolis, Brazil
3Business Administration Department, Federal University of Santa Catarina (UFSC), CCGA/CSE/UFSC, Florianópolis, Brazil
4Production Engineering Department, Federal University of Santa Catarina (UFSC), PPGEP/CTC/UFSC, Florianópolis, Brazil
Email: sensslin@gmail.com, leonardoensslin@gmail.com, rogerlacerda@gmail.com, vhsouza@gmail.com
Copyright © 2014 by authors and Scientific Research Publishing Inc.
This work is licensed under the Creative Commons Attribution International License (CC BY).
Received 7 October 2014; revised 11 November 2014; accepted 24 November 2014
ABSTRACT
This study aims to explore the subject of performance evaluation in project management through the selection of relevant articles on the theme and their subsequent bibliometric analysis, in order to build knowledge among researchers on the subject. The research is characterized as exploratory, descriptive, theoretical-illustrative and adopts ProKnow-C (Knowledge Development Process-Con- structivist) as an intervention tool. As a result, we obtain a set of 18 publications comprising the sample of relevant articles on the topic. From this sample, we found that the International Journal of Project Management is the most aligned with the research topic and the article “Plans are nothing, changing plans is everything: The impact of changes on project success” is the most cited in the sample. This paper highlights the most prominent authors and keywords of the bibliographic portfolio.
Keywords:
Performance Measurement, Project Management, Bibliometric Analysis
1. Introduction
Over recent years, project management has established itself as an important change and development tool in organizations. However, with this growing trend, there is also a need for guidance in this area as well as an evolutionary need of the models, which may allow project-oriented companies to continue to deliver desirable results. In this sense, project management provides organizations with a competitive contribution to improve their performance and minimize the risk of failures in their projects [1] .
Aiming to gain a better understanding of how this field of knowledge is being explored, it is important to mention the research done by Kwak and Anbari [2] , who presented a study that highlighted the practical and theoretical field in which the project management discipline is growing and gaining importance. An important outcome, according to the study, is that performance measurement is the only area in which the research focus grew more than 100% during the 2000s, albeit making a small contribution among the overall subjects (only 7%). This diagnostic show that the issue of performance measurement applied to project management is growing and there is much to be explored.
However, even with this field continuing to be explored, it is still quite difficult to assess whether a project has been delivered with excellence or not, mainly due to the misalignment of understanding about project success achievement, which is still very uncertain and dependent on the perception of each stakeholder [3] . In the view of these authors, in the early development of a project, it is extremely important to understand from a sponsor’s perspective how the working team could achieve project success.
In addition to it, according to Behn [4] , the act of measuring performance and data results does not contribute any accomplishment itself; it may only be considered useful when someone applies actions to achieve his/her targets from these outcomes.
Thus, it is important to know by reading qualified scientific papers how performance evaluation is being applied to projects and then identify literature related to this subject, the research questions guiding this study:
1) What are the scientifically recognized papers on the subject of project management and performance evaluation?
2) What are the prominent journals, authors, articles and keywords in these relevant articles?
To address these research questions, this study aims to explore the topic of performance evaluation in project management conducted through the selection of relevant articles on the theme and their subsequent bibliometric analysis, in order to build knowledge among researchers on the subject.
To achieve this goal, ProKnow-C (Knowledge Development Process-Construtivist) process was applied— which begins with a researcher’s interest in a specific field of knowledge—aiming not only to consolidate his/her knowledge but also to develop a relevant theoretical basis which may support the development of his/her research [5] .
Finally, this study is structured in the following sections: section 2, which deals with the theoretical framework supporting the arguments about the results; section 3, which presents the research methodology; section 4, which presents the selection process of the theoretical references; section 5, which presents the results and finding analysis; and section 6, which presents the conclusions and final considerations of the researchers, as well as the references that guided this study.
2. Theoretical Foundations of Project Management Evaluation
From the moment that project management emerged as an important discipline, in the 1950s, managers have considered planning and quality as key requirements for any successful project. Since then, numerous tools have emerged to assist them in these important activities, such as PERT and CPM, demonstrating the great attention academia has paid to planning processes-mainly related to scheduling and budgeting [6] .
Likewise, performance evaluation systems applied to projects are a recurring theme of discussion, but are rarely defined; the usual approach is to measure success through a limited or one-dimensional criterion [7] . In fact, for the vast majority of project managers, the Iron Triangle (time, cost and quality) defines the dimensions of project success [8] . However, they should consider other dimensions to achieve success in order to incorporate the specific features of each context [9] .
Regarding this majority group, according to Dai and Wells [1] the project manager must maintain and monitor the project to avoid deviations of cost, time and quality. However, to reach this objective, the manager has the support of control systems that can flag problems early in the project, both analysing the past and predicting scenarios.
On the other hand, Atkinson [8] showed that the Iron Triangle has gaps in highlighting the project results. Cooke-Davies [10] contribution reinforced the need to consider in addition the general objectives of the project, extending beyond the triad of cost, time and quality in its performance evaluation.
Based on these observations, Lauras et al. [9] proposed the construction of a method to support decision making for different contexts of projects, which must be adaptable to the needs of decision makers in terms of: 1) the monitoring of activities; 2) the use of measurable criteria for project performance and 3) the observation of the dimensions of performance evaluation of projects. The purpose of the model developed by these authors is to allow managers to evaluate and control their projects pragmatically and thoroughly. It maintains that by providing a holistic view of the project, the decision making for corrective actions may be faster and more assertive, which is not possible when analysing only time, cost and quality.
Another tool that is also widely used for the evaluation of projects is the AHP (analytic hierarchy process), a method for multicriteria decision making proposed by Saaty [11] whose main contribution is to incorporate both tangible and intangible criteria of the possible alternatives. However, although the community was adopting the AHP recognizes its limitations such as rank reversal order [12] and a lack of reference levels for pairwise criteria comparison [13] .
Due to these multiple frameworks and criteria to measure project performance and its success, the importance of developing a comprehensive and updated analysis regarding the different methods found in the scientific literature about project performance evaluation is evident. Hereby, this field of study still allows the research environment opportunities to explore it through a theoretical filiation that guide this research. In order to fulfill that, this research has adopted the following constructivist performance evaluation concept:
“performance evaluation is characterized as the process which aims to build knowledge in the decision maker regarding a specific context which he/she objectives to manage, permitting him/her to comprehend the impact of the current situation in the aspects that he/she assumes are essential to support the management decisions, through the set of the following procedures: identify; organize; measure both ordinally and cardinally; and integrate the appropriate criteria” (Lacerda et al., 2012).
3. Methodology
This section explores in detail the methodological framework of the research, as well as the intervention tool, ProKnow-C.
3.1. Methodological Framework
The research, regarding its goals, is characterized as exploratory and descriptive [14] . It is exploratory as it promotes researchers’ reflection on a particular theme and collaboration for knowledge generation; with its boundaries, it defines a theoretical basis to adopt for the desired study, the bibliographic portfolio (BP), and in addition delimits the scientific community through the application of the investigative process ProKnow-C, the procedure of which is presented in detail during this study. The study is also characterized as descriptive as it describes the characteristics of this portfolio of publications and its references in terms of prominent journals, authors, articles and keywords most used.
Moreover, with regard to its nature, the study is defined as theoretical-illustrative, because it presents a process to perform the literature search with an emphasis on the interest of researchers (authors of this study)— performance evaluation of project management—by means of an approximate guide with steps to be followed for bibliographic searches on a specific topic, adding a practical case of application of ProKnow-C and its sub- stages, the bibliometric analysis [15] .
Regarding the research logic, it is inductive, since the search for knowledge by researchers is conducted through the empirical observation of reality, the problem of which has not undergone deserved exploration or may even demonstrate new perspectives to the scientific environment [14] . Moreover, it states that the development of the researchers’ knowledge occurs throughout the process.
When examining the data collection, the research began making use of primary data, that reflected the limits of researchers imposed according to the definition of BP, in sequence, after the identification of the PB, made use of secondary data, since the information for authors, journals, keywords were collected in the BP publications [14] .
From a problem approach perspective, the research is characterized as qualitative and quantitative. It is qualitative due to the elaboration of the bibliographic portfolio and quantitative because of the bibliometric analysis [14] .
Finally, because the researchers are able to conduct the study based on a specific portfolio (journals, articles, authors), we might consider it as applied research through theoretical procedures for bibliographical research [14] , which performs analyses based on publications from journals cited in the CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior) database-Coordination for the Improvement of Higher Education Personnel.
3.2. Intervention Tool: Proknow-C
Due to the large volume of information available in the scientific environment, the selection of relevant articles to support the development of a study on a particular theme has become very complex, according to Azevedo et al. [16] . For the present study, the intervention tool used achieve the proposed objectives is Proknow-C-Know- ledge Development Process-Constructivist, as already presented in other works [17] -[19] .
The stages that comprise the process are:
1) Selection of a portfolio of relevant articles regarding a specific topic;
2) Bibliometric analysis of the portfolio;
3) Systemic analysis; and
4) Definition of the research question and research objective.
This study, however, focuses on the first two steps of the process.
4. Bibliographical Portfolio Selection Process
From the moment the researchers defines the research theme, they apply the Proknow-C process in order to select a bibliographic portfolio relevant to the topic. This portfolio corresponds to a delimited set of articles with scientific recognition and prominence, with a title, abstract and full content aligned with the research topic that the researcher determined, according to the research boundaries [5] .
The steps to obtain the bibliographic portfolio are made explicit below:
1) Raw articles portfolio selection;
2) Raw articles portfolio filtering;
3) Bibliographic portfolio representativeness test.
The practical implementation of each of these steps is presented below.
4.1. Raw Articles Portfolio Selection
The first step in defining the BP emerges as the selection of the portfolio of raw articles, that is, the set of publications generated from scientific recognition databases and aligned by keyword with the research theme.
4.1.1. Research Chronology
The researchers conducted the procedures for developing the present research from March 2013 to May 2013.
4.1.2. Keywords
To start the procedure, first it is necessary to define two research axes and the respective keywords, all aligned with the theme and according to the researchers’ perception boundaries. The primary axis, which defines the area of research knowledge, refers to performance evaluation, for which the selected keywords are “measurement”, “evaluation”, “appraisal”, “assessment” and “performance management”, while the second axis, which concerns the particular subject about which the researchers aim to learn, refers to the subject of project management, the keywords of which are “project office”, “project management office”, “PMO”, “project management maturity” and “project performance”.
With the crossing of the two axes, we obtained a combination of keywords that resulted in 25 search alternatives for each selected database.
4.1.3. Databases
After defining the research axes and respective keywords, the 25 combinations of alternatives served as a parameter for searches in 6 databases, all available in CAPES Scientific Journals Portal and aligned with the areas of interest, in this case including engineering content as well as multidisciplinary knowledge.
According to the researchers’ perspective and under these delimitations, the selected databases were Scopus, Emerald, Proquest, Science Direct, Web of Science (or ISI) and Engineering Village, which we considered to be the more scientifically relevant databases for the research topic desired.
4.1.4. Article Research on Databases through the Selected Keywords
Therefore, the search for articles on databases began with the combination of keywords defined in the research axes, looking for results regarding article titles, keywords and abstracts. It is important to mention that the research considered the time horizon for publications between 1 January 2000 and 30 March 2013.
The result of this research was a set of 1.273 articles from journals of high scientific relevance, named the “raw articles portfolio”. The initial filter had already eliminated books, editorials and book reviews and these were not part of the set determined.
4.1.5. Keywords’ Test of Adherence
It is important to mention that we also contemplated the keywords “success and strategy” as a possible alternative to the second axis, since they were present in two randomly selected articles within the raw portfolio selected. However, we ruled out the term since it is a goal to be reached and we considered it not to belong to any of the research axes.
4.2. Raw Articles Portfolio Filtering
From this moment the filtering procedure of the raw articles portfolio took place; the software Endnote managed the constituent items. First, it eliminated 452 duplicated publications, so the total of 1.271 articles previously selected now resulted in an initial portfolio of 821 unrepeated articles with a title, keywords or abstract aligned with the research topic.
Then, we ascertained the titles of each article of this set, whereby we removed from the portfolio publications with titles that were clearly misaligned with the research topic, according to researcher assessment. This step removed 548 articles from the raw database, resulting in a group of 273 unrepeated publications with a title aligned with the research topic.
The next step for filtering the raw articles portfolio aimed to analyze the publications regarding their scientific recognition. Accordingly, we listed the number of citations of the remaining 273 articles through Google Scholar and ordered them in a descending way.
For this ordered set, we defined a minimum number of citations as the cut-off value (7 or more citations), which corresponded to 95% of the total 273 articles remaining in the portfolio so far. With this, we obtained 116 articles, which represent the majority of the scientific recognition of the raw articles portfolio until that moment; we identified this set as “repository p”. We identified the remaining 157 articles as “repository k”, corresponding to publications that have low scientific recognition or have had no time to prove it yet.
After the selection of publications with proven scientific recognition, we read the articles’ abstracts to check their alignment with the research theme. From this set of 116 articles, we retained only 17 publications (corresponding to “repository a”), for which we listed the authors and grouped them in a special pool named Pool of Authors (PA).
Hence, these 17 publications present a title and abstract aligned with the research topic, as well as having proven their scientific recognition through a significant number of citations. They represent the core of the bibliographic portfolio; however, it is not yet complete.
Regarding the 157 articles included in “repository k” (publications with low scientific recognition or that have had no time to prove it yet), we analysed the possibility of integrating some of them with the 17 articles previously filtered.
Initially, we selected the articles published from 2011 onwards, as we considered them as recent articles that might have had no time to prove their scientific recognition so far and for this reason there is the potential to incorporate these publications into the raw articles portfolio. From the 157 articles that form “repository k”, 83 articles were published in this period, for which we read the abstracts to ascertain whether the publications were adherent to the research topic or not. From these 83 articles, we selected 2 to join the raw articles portfolio, then forming “repository b”.
For the remaining 74 items in “repository k”, published in 2011 onwards, we determined whether they belonged to any of the authors listed in the Pool of Authors: only one of them had this match. After reading the abstract of this article, we discarded the publication since it was not aligned with the defined research topic.
Therefore, as we selected no articles in this second check, “repository b” consists of two items aligned with the research topic with the potential to gain scientific recognition. We then added these to “repository a” (comprising 17 articles), therefore forming “repository c”, consisting of 19 articles.
To finish the process of filtering the raw articles portfolio, we checked the full availability of free versions of the 19 publications. From this group, we only discarded one publication.
Finally, reading the articles integrally to verify their adherence to the research theme, we eliminated only one publication from the database, producing a final raw articles portfolio made up of 17 articles.
4.3. Portfolio Representativity Test
The last step in the procedure for theoretical portfolio selection takes place by testing its representativity, the goal of which is to check the number of citations (source: Google Scholar) of article references selected for the bibliographic portfolio and consequently to determine their representativity level. It is important to highlight that only references published in journals in the period between 2000 and 2013 were considered in the test.
We listed the article references, a total of 160 publications, in descending order according to the number of citations and then selected 30 of these, a group that corresponds to 80% of the total citations of BP article references. We then inserted this group into Endnote and finally exported it in spreadsheet format aiming to order and validate the article references.
From these 30 articles, we found that one of them had high scientific recognition and content aligned with the research theme. Therefore, we added this article to the final bibliographic portfolio, which now contained 18 publications, which can be found below in Table 1.
5. Bibliographic Portfolio Bibliometric Analysis
Subsequent to the selection of the bibliographic portfolio, we carried out a bibliometric analysis of its publications and references. This analysis consisted of a process that aims to highlight quantitatively a set of publications (BP) made by the number counting of different parameters [17] . For the present study, we chose the following parameters for the article analysis of the bibliographic portfolio and its references:
1) Journals’ relevance and impact factors;
2) Articles’ scientific recognition;
3) Most prominent authors;
4) Most used keywords.
5.1. Journals’ Relevance
The first parameter we analyzed in the bibliometric process was the journals’ relevance of the BP publications and their references.
The outstanding journal among the 18 articles that constitute the BP is the International Journal of Project Management, which contributed with 10 selected articles. The remaining 8 publications come from different journals, showing the high relevance of the highlighted journal according to academia regarding the research topic.
We also verified this relevance by analyzing the more prominent journals among the bibliographic portfolio references, in which the International Journal of Project Management appears with 34 of a total of 160 articles originating from 45 different journals. Moreover, the Project Management Journal with 18 articles and the journals Decision Support Systems and Journal of Construction Engineering and Management, with 8 publications each, are prominent among all the BP references articles, as shown in Figure 1.
Finally, we analysed the journals’ relevance both in the BP articles and in their references, certifying the International Journal of Project Management as the most prominent in both sets, which is due to the periodic focus on the second subject field of the researchers’ interest, project management.
5.2. Journals’ Impact Factor
Another important bibliometric analysis refers to the verification of the journals’ impact factor from the BP ar-
Table 1. Bibliographic portfolio articles. Source: authors.
Figure 1. Bibliographic portfolio references number of articles per journal. Source: authors.
ticles, which aims to attest the relevance and influence of journals across the scientific world. In this way, the factors chosen for analysis originate from the ISI Web of Knowledge (JCR) and Scopus (SJR). These indices refer to the average number of citations that the articles published in a particular journal received in the past two years.
Thus, analysing the ISI database, we verified that 7 out of the 9 journals presented in the BP publications have a calculated bibliographic impact factor. In this analysis, the journal Research Policy is highlighted with a JCR of 2520.
Moreover, when analysing the Scopus database, we attested that all the journals present calculated impact factors. The journal featured in this reference is also Research Policy, with an SJR of 2192.
Finally, by comparing the two impact factors, we verified that the journal Research Policy is the most prominently featured in both databases, as shown in Figure 2.
5.3. Articles’ Scientific Recognition
The next analysis concerns the verification of scientific recognition regarding the BP articles, as well as their references. Analysing the first set, the publication that stands out with 140 citations, according to the Google Scholar site, is “Plans are nothing, changing plans is everything: The impact of changes on project success”, published by D. Dvir and T. Lechler in 2004. Besides, we can highlight the articles “Defining ‘success’ for software projects: An exploratory revelation”, written by N. Agarwal and U. Rathod in 2006, with 129 citations, and the article “An exploration of project management office features and their relationship to project performance”, published by C. X. Dai and W. G. Wells in 2004, with 121 citations.
The next step consists of analysing the most cited article presented in BP publications’ references. In this regard, standing out with 670 citations (source: Google Scholar) is the article “Innovation in project-based, service-enhanced firms: The construction of complex products and systems”, written by D. M. Gann and A. J. Salter in 2000.
Finally, we analysed the correspondence between the BP articles’ number of citations and the most-cited author among the BP reference publications. The highlight, again, is the publication “Plans are nothing, change plans are everything: The impact of changes on project success”, published by D. Dvir and T. Lechler in 2004, which presents 140 citations in Google Scholar and is present 14 times in the BP references (Figure 3).
5.4. Prominent Authors
Among the authors presented in the BP articles, there is no verified prominent name; meanwhile, of the 37 au
Figure 2. Portfolio journals with prominent impact factors. Source: authors.
Figure 3. Prominent articles in the bibliographic portfolio and their authors. Source: authors.
thors presented in the portfolio, 6 have 2 items in this set. They are: D. Gourc, G. Marques, L. Ensslin, M. Lauras, R. T. Lacerda de Oliveira and S. R. Ensslin.
When analysing the authors present in the references, however, some appear with relative prominence. In this case, within a universe of 203 authors, we can highlight the researchers D. Dvir, with 14 articles, A. J. Shenhar with 10, T. Lechler with 6 and F. Y. Y. Ling and H. G. Gemunden with 5 items each.
5.5. Most-Used Keywords
Subsequent to the bibliometric analysis from the perspective of journals, articles and authors, we checked the most common keywords presented in BP articles.
Thus, it appears that the most prominent keywords among the BP publications are “project management” with six appearances and “project performance” with four appearances. Through this analysis, it is easy to attest to the alignment between the two research axes, “performance evaluation” and “project management”, with the keywords presented in the articles.
6. Conclusions and Recommendations for Further Research
The present study aimed to operationalize a tool—ProKnow-C—to illustrate the process of selecting a bibliographic portfolio deemed relevant by the scientific community, but above all, in line with the theoretical affiliation of the investigator who makes use of such an instrument to build knowledge on a particular subject—this research project management from the perspective of performance evaluation.
The article is divided into two main phases:
1) Bibliographic portfolio selection;
2) Bibliographic portfolio bibliometric analysis.
The goal was firstly to select a set of articles that were fully aligned with the researchers’ theme, then se- condly to analyse the selected portfolio and its references according to different bibliometric parameters: periodicals, articles, authors and keywords.
Besides, the process allowed us to find, from an initial set of 1271 articles, a final portfolio of 18 publications, which have proven scientific recognition and are aligned with the researchers’ theme.
In the analysis, it became clear that the International Journal of Project Management has the highest number of articles published in the set. In relation to the authors, the highlights are D. Gourc, G. Marques, L. Ensslin, M. Lauras, R. T. Lacerda de Oliveira and S. R. Ensslin, each with two publications in the set.
Next, when analysing the references of this bibliographic portfolio, the International Journal of Project Management appeared first again (this time with 34 articles published). Moreover, the journals Decision Support Systems and Journal of Construction also presented relevance, with 8 articles published each.
On the other hand, when analysing the prominent authors both in the portfolio and in the references, the relevant names were: D. Dvir, T. Lechler, H. G. Gemunden and F. Y. Ling.
Another disclosure concerns the scientific relevance of the publications present in the BP. When analysing the number of articles’ citations, according to the website Google Scholar, through the number of appearances of the main references in the authors’ portfolio, it emerged that the feature article written by a prominent author is: “Plans are nothing: Plans are changing everything”; this article, with 140 citations and written by D. Dvir, appeared 14 times in the portfolio references.
Finally, the keywords highlighted in the set were: “project management” and “project performance”, both present six times in selected portfolio.
Thus, it is possible to ensure that we answered the research questions adequately: what are the scientifically recognized papers on the subject? What are the prominent journals, authors, articles and keywords among these relevant articles?
Likewise, the study also addressed its objective. However, there are limitations to the research developed, which are database restrictions: only six of them were considered; only publications written in English were considered; and, due to the portfolio delimitation, only articles published in journals were considered, excluding monographs, theses, dissertations, books and scientific conferences.
Finally, in order to serve as recommendations for future research, the analysis of the content of these relevant papers would be useful in order to understand how the performance evaluation area has been aiding the project management field in order to reach the desirable results.
References
- Dai, C.X. and Wells, W.G. (2004) An Exploration of Project Management Office Features and Their Relationship to Project Performance. International Journal of Project Management, 22, 523-532. http://dx.doi.org/10.1016/j.ijproman.2004.04.001
- Kwak, Y.H. and Anbari, F.T. (2009) Analyzing Project Management Research: Perspectives from Top Management Journals. International Journal of Project Management, 27, 435-446. http://dx.doi.org/10.1016/j.ijproman.2008.08.004
- Agarwal, N. and Rathod, U. (2006) Defining “Success” for Software Projects: An Exploratory Revelation. International Journal of Project Management, 24, 358-370. http://dx.doi.org/10.1016/j.ijproman.2005.11.009
- Behn, R.D. (2003) Why Measure Performance? Different Purposes Require Different Measures. Public Administration Review, 63, 586-606. http://dx.doi.org/10.1111/1540-6210.00322
- Lacerda, R.T.D.O., Ensslin, L. and Ensslin, S.R. (2014) Research Opportunities in Strategic Management Field: A Performance Measurement Approach. International Journal of Business Performance Management, 15, 158-174. http://dx.doi.org/10.1504/IJBPM.2014.060165
- Dvir, D. and Lechler, T. (2004) Plans Are Nothing, Changing Plans Is Everything: The Impact of Changes on Project Success. Research Policy, 33, 1-15. http://dx.doi.org/10.1016/j.respol.2003.04.001
- Neely, A., Gregory, M. and Platts, K. (1995) Performance Measurement System Design: A Literature Review and Research Agenda. International Journal of Operations & Production Management, 15, 80-116. http://dx.doi.org/10.1108/01443579510083622
- Atkinson, R. (1999) Project Management: Cost, time and Quality, Two Best Guesses and a Phenomenon, Its Time to Accept Other Success Criteria. International Journal of Project Management, 17, 337-342. http://dx.doi.org/10.1016/S0263-7863(98)00069-6
- Lauras, M., Marques, G. and Gourc, D. (2010) Towards a Multi-Dimensional Project Performance Measurement System. Decision Support Systems, 48, 342-353. http://dx.doi.org/10.1016/j.dss.2009.09.002
- Cooke-Davies, T. (2002) The “Real” Success Factors on Projects. International Journal of Project Management, 20, 185-190. http://dx.doi.org/10.1016/S0263-7863(01)00067-9
- Saaty, T.L. (1977) A Scaling Method for Priorities in Hierarchical Structures. Journal of Mathematical Psychology, 15, 234-281. http://dx.doi.org/10.1016/0022-2496(77)90033-5
- Sarkis, J. (2003) Quantitative Models for Performance Measurement Systems—Alternate Considerations. International Journal of Production Economics, 86, 81-90. http://dx.doi.org/10.1016/S0925-5273(03)00055-0
- Barzilai, J. (2001) On the Foundations of Measurement. Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Tucson, 7-10 October 2001, 401-406.
- Richardson, R.J. and de Sousa Peres, J.A. (1999) Pesquisa social: Métodos e técnicas. Atlas, Säo Paulo.
- Alavi, M. and Carlson, P. (1992) A Review of MIS Research and Disciplinary Development. Journal of Management Information Systems, 8, 45-62.
- Azevedo, R.C., Ensslin, L., Lacerda R.T.O., França, L.A., González, C.J.I., Jungles, A.E. and Ensslin, S.R. (2011) Avaliação de desempenho do processo de orçamento: Estudo de caso em uma obra de construção civil. Revista Ambiente Construído, Porto Alegre.
- Tasca, J.E., Ensslin, L., Ensslin, S.R. and Alves, M.B.M. (2010) An Approach for Selecting a Theoretical Framework for the Evaluation of Training Programs. Journal of European Industrial Training, 34, 631-655. http://dx.doi.org/10.1108/03090591011070761
- Marafon, A.D., Ensslin, L., de Oliveira Lacerda, R.T. and Ensslin, S.R. (2012) The Implications of R & D Management for Organizational Effectiveness: A Literature Review. Proceedings of the IEEE International Technology Management Conference (ITMC), Dallas, 25-27 June 2012, 202-211.
- Da Rosa, F.S., Ensslin, S.R., Ensslin, L. and Lunkes, R.J. (2012) Environmental Disclosure Management: A Constructivist Case. Management Decision, 50, 1117-1136. http://dx.doi.org/10.1108/00251741211238364
- Adenfelt, M. (2010) Exploring the Performance of Transnational Projects: Shared Knowledge, Coordination and Communication. International Journal of Project Management, 28, 529-538. http://dx.doi.org/10.1016/j.ijproman.2009.10.004
- Ahsan, K. and Gunawan, I. (2010) Analysis of Cost and Schedule Performance of International Development Projects. International Journal of Project Management, 28, 68-78. http://dx.doi.org/10.1016/j.ijproman.2009.03.005
- Barclay, C. (2008) Towards an Integrated Measurement of IS Project Performance: The Project Performance Scorecard. Information Systems Frontiers, 10, 331-345. http://dx.doi.org/10.1007/s10796-008-9083-6
- Biedenbach, T. and Müller, R. (2012) Absorptive, Innovative and Adaptive Capabilities and Their Impact on Project and Project Portfolio Performance. International Journal of Project Management, 30, 621-635.
- Blindenbach-Driessen, F., Van Dalen, J. and Van Den Ende, J. (2010) Subjective Performance Assessment of Innovation Projects. Journal of Product Innovation Management, 27, 572-592. http://dx.doi.org/10.1111/j.1540-5885.2010.00736.x
- Cao, Q. and Hoffman, J.J. (2011) A Case Study Approach for Developing a Project Performance Evaluation System. International Journal of Project Management, 29, 155-164. http://dx.doi.org/10.1016/j.ijproman.2010.02.010
- De Oliveira Lacerda, R.T., Ensslin, L. and Ensslin, S.R. (2011) A Performance Measurement Framework in Portfolio Management: A Constructivist Case. Management Decision, 49, 648-668. http://dx.doi.org/10.1108/00251741111126530
- De Oliveira Lacerda, R.T., Ensslin, L. and Ensslin, S.R. (2011) A Performance Measurement View of IT Project Management. International Journal of Productivity and Performance Management, 60, 132-151. http://dx.doi.org/10.1108/17410401111101476
- Ling, F.Y.Y., Low, S.P., Wang, S.Q. and Lim, H.H. (2009) Key Project Management Practices Affecting Singaporean Firms’ Project Performance in China. International Journal of Project Management, 27, 59-71. http://dx.doi.org/10.1016/j.ijproman.2007.10.004
- Marques, G., Gourc, D. and Lauras, M. (2011) Multi-Criteria Performance Analysis for Decision Making in Project Management. International Journal of Project Management, 29, 1057-1069. http://dx.doi.org/10.1016/j.ijproman.2010.10.002
- Toor, S.-U.-R. and Ogunlana, S.O. (2010) Beyond the “Iron Triangle”: Stakeholder Perception of Key Performance Indicators (KPIs) for Large-Scale Public Sector Development Projects. International Journal of Project Management, 28, 228-236. http://dx.doi.org/10.1016/j.ijproman.2009.05.005
- Unger, B.N., Gemünden, H.G. and Aubry, M. (2012) The Three Roles of a Project Portfolio Management Office: Their Impact on Portfolio Management Execution and Success. International Journal of Project Management, 30, 608-620. http://dx.doi.org/10.1016/j.ijproman.2012.01.015
- Wi, H. and Jung, M. (2010) Modeling and Analysis of Project Performance Factors in an Extended Project-Oriented Virtual Organization (EProVO). Expert Systems with Applications, 37, 1143-1151. http://dx.doi.org/10.1016/j.eswa.2009.06.051
- Yuan, J., Wang, C., Skibniewski, M., and Li, Q. (2012) Developing Key Performance Indicators for Public-Private Partnership Projects: Questionnaire Survey and Analysis. Journal of Management in Engineering, 28, 252-264. http://dx.doi.org/10.1061/(ASCE)ME.1943-5479.0000113