ABSTRACT

The main objective of this study is to measure the relative efficiency of Indonesian universities in 2015. There are twenty five DMUs with four inputs and five outputs that are analyzed. Due to the low number of Indonesian scientific publications, this study analyses the performance of the top 25 universities based on the Webometrics ranking as it has been used as one of the indicators of university achievements by the Higher Education of Indonesia. The Data Envelopment Analysis (DEA) is used to obtain the scores of efficiency, while the Fuzzy approach is applied to address the possibility of errors from the auditor’s assessment in determining the input and output variables correctly. The FDEA can be used in measuring the universities performances under imprecise inputs and outputs. Both the CRS (constant returns to scale) and the VRS (variable returns to scale) models are presented. The empirical results show that 36 percent of universities perform efficiently on the CRS model while 52 percent of universities have efficient performances under the VRS model. Furthermore, the well-known universities have shown relatively low scores, which indicate they need to improve their performances in publishing scientific work, as well as providing useful information to the public through the official websites. Generally, the results of the VRS model are better than the CRS model for both the DEA and the FDEA methods.

Keywords:

Efficiency Measurement, Universities, DEA, FDEA

1. Introduction

Nowadays, the role of universities in a country is very important in creating high quality human resources so that they are easily adaptable to rapidly changing times. Countries with qualified people tend to have higher quality education institutions. It is important to note that performance efficiency of universities can be used as inputs to the government to allocate human resources optimally. By knowing the efficiency of universities performances the government allows to make future framework of long term plans based on valid data. The low numbers of scientific publications of Indonesian researchers have received special attention from the Higher Education of Indonesia. According to the Indonesian Institute of Sciences, there are only 4500 to 5500 scientific works that have been published successfully. Moreover, the total number of international level journal publications is only 415 works throughout 2015. Meanwhile, at the national level there are 887 scientific papers that have been published. This figure is very alarming when in comparison to the total population of Indonesia that reaches up to 250 million [1] . This is a barrier for universities in Indonesia to enter the ranks of World Class Universities. Therefore, Indonesian universities need to improve their efforts not only in publishing scientific work, but also in providing useful information to the public through the official websites.

This paper investigates the top 25 universities in Indonesia based on the ranking that was conducted by Webometrics in 2015 in order to measure the efficiency of their performances. The ranking of universities has some positive benefits, as not only does it raise their prestige, but also can be used as promotional material to attract new students. Webometrics is a system to measure and assess the progress of the world class universities based on the university’s website. In its official website, Webometrics specifies that the purpose of this ranking is to promote the presence of academic websites so that scientific work and cultural knowledge can be accessed openly. The ranking of universities through the website does not merely indicate the quality of the university in general, but rather the power of its official website to produce scientific information for public access. A publication based off of this rating is considered to be an effective way to encourage universities around the world to provide better services to the community. Another important aspect is the university’s academic communities, which are encouraged to be productive in research. Although there are still doubts about the level of validity, the Webometrics ranking has been used as one of the indicators of university achievements by the Higher Education of Indonesia. The Data Envelopment Analysis (DEA) method is used to measure the efficiency of universities performances. However, this approach requires the accuracy and precision of the inputs and outputs. In reality, available data are often vague; there are qualitative data or linguistic data. This study applies the Fuzzy Data Envelopment Analysis (FDEA) to deal with undetermined data. This approach may reduce the possibility of errors made by the auditor in determining the input and output variables. Besides that, this approach tends to produce better results efficiency measurements due to the fact that the efficiency scores are not in crisp numbers but are in the form of fuzzy numbers instead. The main purpose of this study is to measure the performance efficiency of state and private university that are categorized or ranked as the top 25 universities in Indonesia based on Webometrics ranking by using the interval efficiency of FDEA.

The applications of the DEA and the FDEA have been done in various fields of sciences. [2] , to investigate the efficiency of the bus transit system, while [3] the use of the DEA method to measure the container terminal in Europe. [4] , the DEA method is also used to analyze the policing performances that focus on crime management undertaken by the Audit Commission in England and Wales. The efficiency measurement for the performance appraisal of employees is investigated by [5] . Whereas studies that use the DEA method concerning airline efficiencies are conducted by [6] [7] [8] [9] etcetera. [10] , used credibility approach of FDEA in measuring post offices performances in Serbia. [11] , the Fuzzy theory is applied to the measurement of bank efficiencies in Tunisia using the DEA method. By using the financial and nonfinancial data, as many as 14 commercial banks in Tunisia are used as DMU’s during the period 2010-2012. The results report that small and mid-sized banks are the least efficient.

A large number of studies concerned with the performance of the education institutions have been studied by researchers (see [12] [13] [14] [15] ) [16] , apply the method in public higher education sectors. [17] , use the DEA method to investigate the productivity growth in Australian universities while [18] uses the method to measure public university performances in the Czech Republic. The efficiency measurements on university departments are also studied by [19] . [20] , compares the results of the classic DEA and the FDEA in measuring and evaluating school activities in Turkey during the period of 2012-2013 by using 25 high schools as the DMUs. The empirical results of this study strongly recommend using the FDEA method to get more accuracy in measuring school activities. [21] , investigates the efficiency of business school performances in Taiwan using the DEA method and the AR-DEA (Assurance Region-DEA) method. The results report that public schools have better performances than private schools. Furthermore, the empirical results indicate that the AR-DEA is better than the DEA. [22] , use the DEA method to evaluate the relative technical efficiencies of 30 academic departments at the Islamic University of Gaza (IUG) during the academic period of 2004-2006. The results report that the average efficiency scored is 68.5 percent. Furthermore, there are 10 departments performing efficiently. [23] , evaluate the department performances at Transylvania University during the years of 2014 & 2015. There are 30 departments that are considered as DMUs with two inputs and four outputs, with the results reporting that there are 12 departments with efficient performances.

2. Data Envelopment Analysis (DEA) and Fuzzy Data Envelopment Analysis (FDEA)

2.1. Basic Model of Data Envelopment Analysis (DEA)

The DEA is a method for measuring the efficiency which is usually linked to the performances of a unit, an organization or program that reflects on the ratio between outputs and inputs. A unit or an organization can be said to have an efficient performance if the efforts that are being made can produce maximum outputs. It performs efficiently when the minimum efforts can achieve a certain output. The most well-known method to measure the efficiency of the performance is the Data Envelopment Analysis (DEA) method, which is a nonparametric approach based on linear programming to investigate the DMUs with a lot of inputs and outputs. A DMU is said to be relatively efficient if the efficiency score is equal to 1 (or efficiency value equals to 100 percent), however, when the efficiency score is less than 1, then the DMU has an inefficient performance.

There are two models of DEA which are often mentioned in literatures, namely the CRS and the VRS models. The CRS model (constant returns to scale) is developed by [24] so it is also known as the CCR model. This model assumes that the ratio between the addition of input and output is the same. Moreover, this model assumes that each DMU operates at an optimal scale. Meanwhile, the VRS model (variable returns to scale) is developed by [25] and is also known as the BCC model, which is a development of the first model. This model assumes that each DMU is not or at least not yet operating at an optimal scale. Then, the ratio of extra input and output is not the same, which can either be an increasing return to scale (IRS) or a decreasing return to scale (DRS). The CCR model defines the DEA efficiency as the ratio of weighted outputs to weighted inputs subject to the condition that similar ratios for every DMU is less than or equal to unity. Let denote the observed of input i for entity j where

and denote as the observed output r for entity j where. The k-line, i.e. X_k and Y_k, of these matrixes show quantified inputs/outputs of a unit DMU_k. Then the mathematical form of the CCR model for the selected entity k can be written as follows:

(2.1)

Subject to

And

where

: The relative efficiency of DMU_k

: The weighted output r

: The weighted input i

: The quantity of output r produced by unit

: The quantity of input i produced by unit

m: The number of inputs

s: The number of outputs

n: The number of entities

: Small positive value

Equation (2.1) is nonlinear with fractional programming problem which have intractable nonlinear and non-convex properties [24] . In order to solve Equation (2.1), then it is transformed into a linear programming form is given as follows:

(2.2)

Subject to

Or

The model above is a linear program that can be solved with a regular program for a linear equation system. Due to the fact that the traditional DEA method is based on frontiers or boundaries, it requires accurate and precise data to obtain good results because even a minor change of the data can change the measurement efficiency significantly. Therefore, the selection of the DMUs and the variables of input and output should provide valid and accurate data. However, most of the data available are not accurate and precise, but they are complex and volatile, sometimes the data is qualitative or even linguistic, thus the traditional DEA method is less suitable to be applied to these types of data. The Fuzzy theory, which is introduced by [26] , can be applied to this model to deal with imprecise and vague data. The Fuzzy DEA model can be used to deal with all kinds of input and output under fuzzy numbers. There are three main procedures of fuzzy logic implementation, namely fuzzification, which is a process to change the crisp numbers into fuzzy numbers. The Fuzzy inference system is a process to explain the relationship between the input and output variables, while defuzzification is a process of changing the fuzzy variables into crisp data.

2.2. Fuzzy Data Envelopment Analysis (FDEA)

The use of the fuzzy set theory on the DEA model is introduced by [27] by applying the concept of the fuzzy linear programming to allow the fuzzy nature of the objective function and constraints on the CCR model under conditions of imprecise data. The study introduces two approaches to solve the DEA model when there is a lack of information of input and output variables. The first approach uses a probabilistic efficiency frontier to deal with the imprecise data which is generated by a stochastic mechanism while the second approach uses the fuzzy theory with two types of membership functions, which are the linear and nonlinear membership functions. [28] , the use of the fuzzy parametric programming to deal with the DEA framework when the production plans are not shown in crisp numbers but in the fuzzy environment.

Generally, FDEA model is defined as follows. Let a group of n decision making units, each DMU_k () uses a set of fuzzy inputs to produce a set of fuzzy outputs where components of and are fuzzy numbers. Then the fuzzy CCR model can be formulated as follows:

(2.3)

Subject to

where k, shows quantified inputs/outputs of a unit DMU_k whereas symbols and are fuzzy equality and inequality which means approximately equal to and approximately smaller than. By using a specific α-level, where, then the efficiency of the DMU_k can be achieved by solving the following model:

(2.4)

Subject to

By using the α-level intervals the following model describes DEA model under fuzzy theory:

(2.5)

Subject to

(2.6)

[29] , mentions there are five approaches to solve the DEA model that provides fuzzy inputs and fuzzy outputs, i.e., defuzzificztion, α-level based, fuzzy ranking, possibility measure, and credibility approaches. The first is the defuzzification approach, where the inputs under the fuzzy environment are converted into crisp values through the defuzzification process. The crisp values are then solved by the usual linear program while the α-level approach is solved by using parametric programming. Fuzzy ranking approach produces the bi-level linear programming model, while the possibility approach is to change the FDEA into possibility models by using the possibility measures and the chance-constraint programming. In order to determine the optimal range of the DMUs efficiency, the Equation (2.6) is decomposed to maximize the and, which are formulated as follows:

(2.7)

The Equation (2.6) and the Equation (2.7) are linear programming problems. Let be the centre values, and then let be the efficiency of α-cut. This study uses the most popular FDEA method, the α-level approach, where the values of the input and output variables are converted into interval values. In order to determine the lower and upper bounds of the membership functions of the efficiency scores, this method converts the fuzzy CCR model into the parametric program. According to [30] , the lower and upper limit values are calculated as follows:

where represents the lower limit value, represents the upper limit value, while represents the central value of the variable, while a and b are identified as follows:

where represents the standard error and is calculated by. By applying the α-level approach based on [30] , the fuzzy data with interval values are analyzed by using the FDEA method.

3. Results and Discussions

To evaluate the performance of Indonesian universities, the empirical data are derived from the latest available annual reports from the Higher Education of Indonesia. A total of 25 universities, both public and private that have occupied the top ranking’s based on Webometrics, which is a web-based system of ranking, were selected as DMUs. There are five outputs and four inputs that were analyzed. In addition to world ranking, the indicators of university ranking used by Webometrics were considered as the outputs of this study. There are four indicators, i.e., presence rank (20%), impact rank (50%), openness rank (15%) and excellence rank (15%). The presence rank represents the number of web pages in the primary domain, including all sub domains and directories of the universities that are indexed by search engines. The impact rank shows the quality of the content, which is evaluated through a virtual referendum by counting all the external in links which are received by the universities’ web domain. The openness rank represents the number of files in the form of Adobe Acrobat (.pdf), Adobe PostScript (.ps, .eps), Microsoft Word (.doc, .docx) and Microsoft PowerPoint (.ppt, .pptx), which are online under the universities’ domain. The excellence rank represents the number of scientific publications are indexed by the Scimago Institution Ranking and Google Scholar [31] . This research uses input variables that are directly related to the real situation of the universities; the number of lecturers, the number of students, the number of departments and A-accredited programs by Higher Education of Indonesia [32] .

Table 1 and Table 2 shows the expected values of input and output for measuring the performances of the top 25 universities in Indonesia in 2015 while Table 3 represents

a general description of input and output variables.

The DEA approach is implemented to obtain the efficiency score for Indonesian universities. By using the DEAP software, Table 4 shows the efficiency scores of each university. However, there are two approaches of DEA that was produced as a comparison, i.e., CRS and VRS. The scale indicates a comparison of the two models. The average of efficiency score is 0.749 and 0.839 for the CRS and the VRS models, respectively. Table 4 reports that there are 9 universities or about 36 percent, that perform efficiently. They are Universitas Syiah Kuala, Universitas Lampung, Petra Christian University, Universitas Jenderal Soedirman, Universitas Ahmad Dahlan, BINUS University, Universitas Terbuka, Telkom University, and Universitas Muhammadiyah Yogyakarta. Meanwhile, Universitas Negeri Yogyakarta and Universitas Udayana have relatively high scores, i.e., 0.941 and 0.917 respectively. Based on the description of the discrimination stages that are reported by [33] they should be then able to support the phase of

activity from the existing inputs with only 94.1 percent and 91.7 percent, respectively. In other words, Universitas Negeri Yogyakarta needs to improve its performance by around 5.9 percent to reach an efficient level while Universitas Udayana requires 8.3 percent to achieve an efficient performance.

Moreover, the results also indicate that there are 9 universities that have performances below the average, i.e., Universitas Gadjah Mada, Universitas Indonesia, Brawijaya University, Institute of Technology Bandung, Bogor Agricultural University, Universitas Diponegoro, Universitas Padjajaran, Universitas Airlangga, and Institut Teknologi Sepuluh Nopember. The result is rather intriguing, as they are well-known qualified universities in Indonesia. This can occur due to indicators that were used in this study that only focuses on the university’s website. Furthermore, on the VRS model, there are 13 universities that perform efficiently (52 percent) whereas the rest need to upscale their performances in order to provide efficient levels of performance. These universities with perfect efficiency scores on the CRS model, have the same score on the VRS model. Aside from them, Universitas Negeri Yogyakarta, Universitas Udayana, Gunadarma University, and Universitas Negeri Semarang have efficient performances while the lowest efficiency score of the VRS model is 0.417 (Universitas Indonesia). It is fair to say that the VRS model provides better results than CRS model. Figure 1 below shows the comparison of the traditional DEA efficiency scores between the CRS and the VRS model.

To reduce the probability of errors during the auditor’s assessment in determining the correct input and output variables, this study applied the fuzzy logic to the the DEA method. In order to measure the universities performances by using the FDEA, the input and output variables are analyzed by changing the data into interval data. This study uses the -level approach based on [30] to produce the interval data, in which the standard errors of the 25 universities are applied to define the interval data. Table 5 and Table 6 show the results of the FDEA analysis. The average efficiency scores of the CRS model are (0.706, 0.743, 0.780) while the lowest efficiency scores are (0.110, 0.165, 0.219). The average efficiency scores of the VRS model are (0.802, 0.835, 0.868) while the lowest scores are (0.283, 0.400, 0.497). Figure 2 shows the Fuzzy efficiency scores of the CRS model while Figure 3 shows the fuzzy efficiency scores of the VRS model.

For the CRS model, there are 9 universities that performed efficiently, i.e., Universitas

Syiah Kuala, Universitas Lampung, Petra Christian University, Universitas Jenderal Soedirman, Universitas Ahmad Dahlan, BINUS University, Universitas Terbuka, Telkom University, Universitas Muhammadiyah Yogyakarta. Whereas, the VRS model produced 13 universities with efficient performances, i.e., Universitas Udayana, Univesitas Syaih Kuala, Universitas Lampung, Gunadarma University, Petra Christian University, Universitas Jenderal Soedirman, Universitas Ahmad Dahlan, BINUS University, Universitas Terbuka, Universitas Negeri Semarang, Telkom University, Universitas Muhammadiyah Yogyakarta and Universitas Negeri Semarang. Universitas Gadjah Mada has the lowest efficiency scores for the CRS model, i.e., (0.110, 0.165, 0.219) while Universitas Indonesia has the lowest efficiency scores for the VRS model, i.e., (0.283, 0.400, 0.517). Overall, the VRS model produced better results than the CRS

model for measuring the efficiency of universities by using the fuzzy logic method.

The results of both the DEA and the FDEA methods reveal that the well-known universities have low scores. For example, Universitas Gadjah Mada and Universitas Indonesia

have the lowest scores, even though they are relatively recognized universities in Indonesia. It should be noted that the efficiency scores in this study simply indicate that the universities have efficient performances in producing outputs from the existing inputs, and does not serve as a general overview of the universities performances. Therefore, these results do not indicate that they have poor qualities in general, due to the fact that the indicators that were used in this study only analyzed the website contents of the universities to determine the efficiency scores. This only indicates that they need to improve their efforts in providing useful information for the public via their websites. Even though the use of the DEA and the FDEA methods for measuring the performances of the universities have been widely used, only a few studies examine the performances of Indonesian universities based on their official websites efforts for providing useful information to the public. It is important to note that the Indonesian government has imposed a disclosure of education through the Law Public Disclosure No 14 of 2008, which states that educational institutions at all levels are required to provide the information to the public. The measurements of the performance of the universities are based on Webometrics by using the DEA and the FDEA methods which were very useful for the universities as well as the Indonesian government. Therefore, the efficiency measurement of Indonesian universities needs to be done to determine whether a university has an efficient performance in publishing information. In addition, the efficiency score can be used as a reference to improve their website content or performances.

4. Conclusion

This study investigates the performance of Indonesian universities in 2015 by using the DEA and the FDEA methods. The FDEA was used to reduce the possible errors in determining the variables of input and output in the analysis. There are 25 DMUS with five outputs and four inputs that were analyzed to obtain the efficiency scores. The empirical results for both the DEA and the FDEA methods indicate that the VRS model produces better results than the CRS model. Both results reveal that well-known universities have relatively low scores, which indicate that their performances in providing useful information to the public should be improved. However, the efficiency scores do not describe the universities performances generally. To conclude, there are 9 universities that perform efficiently on the CRS model while there are 13 universities have efficient performance on the VRS model.

Acknowledgements

The authors would like to acknowledge the School of Informatics and Applied Mathematic (SIAM), as well as the Research Management Centre (RMC) of University Malaysia Terengganu for their continuous and diligent support.

Cite this paper

Mahmudah, U. and Lola, M.S. (2016) The Efficiency Measurement of Indonesian Universities Based on a Fuzzy Data Envelopment Analysis. Open Journal of Statistics, 6, 1050-1066. http://dx.doi.org/10.4236/ojs.2016.66085

References