Theoretical Economics Letters
Vol.08 No.14(2018), Article ID:88014,16 pages
10.4236/tel.2018.814195

Economics of Decision Making: Exploring Analytical Hierarchical Process (AHP)

Jitendra Sharma

Institute of Management Technology, Nagpur, India

Copyright © 2018 by author and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

http://creativecommons.org/licenses/by/4.0/

Received: September 14, 2018; Accepted: October 22, 2018; Published: October 25, 2018

ABSTRACT

Decision making in any field, like all other economic issues, involves allocating scarce resources to meet various needs. Since ages decision making has always intrigued the mankind. A host of research study has been conducted in past few decades on economics of decision making. A number of very effective decision tools have been suggested which falls under the category of multiple criteria decision making. The paper presents one such decision making tool called Analytical Hierarchical Process (AHP), a technique for complex decision making used for large-scale, multi-party, multi-criteria decision analysis. AHP converts comparative evaluations to numerical values that can be processed and weighed over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. The objectives of the article are to understand the interrelationship between considered criteria and available alternatives, to grasp the basics of decision making and decision analysis and to use decision making and decision analysis in assessing the scope for cost-reduction (economics of decision making). The outcome of the study is a mathematical solution to the perennial subjective decision making process in the form of a structured methodology culled out from varied disciplines of economics, psychology and mathematics. The applicability of the AHP Model is demonstrated with an illustrative real life example.

Keywords:

Decision Making, Multi Criteria, Analytical Hierarchical Process, Pairwise Comparison, Eigen Vectors, Economics, Cost Benefit Analysis

1. Introduction

Analytic hierarchy process (AHP) was developed by Thomas L. Saaty in the 1970s and has been extensively studied, analyzed and refined since its inception [1]. It is a structured technique for organizing and analyzing complex decisions, based on mathematics and psychology [2]. This technique facilitates a logical and comprehensive structured framework for framing a decision dilemma, for representing and quantifying its criteria, for relating those criteria into overall objectives, and for evaluating alternative options and solutions [3].

AHP has particular application in group decision making [4] and is used around the world in a wide variety of decision situations, in fields such as government, business, industry, healthcare, shipbuilding and education [5]. Rather than prescribing a correct decision, the AHP helps decision makers find one that best suits their goal and their understanding of the problem [6].

The first step in the analytic hierarchy process is to model the problem as a hierarchy. A hierarchy is a stratified system of ranking and organizing people, things, ideas, etc., where each element of the system, except for the top one, is subordinate to one or more other elements. Though the concept of hierarchy is easily grasped intuitively, it can also be described mathematically [7]. The process of AHP begins by converting an existing decision dilemma into a hierarchy of easily deciphered and understood sub-problems, each of which can then be analyzed independently. The entities in the hierarchy can depict any aspect of the decision situation―it can be implicit or explicit, exactly monitored or approximated guess, thoroughly or partially represented, completely or poorly understood―anything at all that fits into the dilemma on hand.

Once the hierarchy is ready, the users then evaluate its different entities by comparing them to one another two at a time, as to what is the impact of this entity on the entity in higher up hierarchy. It is the essence of the AHP that human judgments, and not just the underlying information, can be used in performing the evaluations [8]. In doing this, participants explore the aspects of the problem at levels from general to detailed, then express it in the multileveled way that the AHP requires. As they work to build the hierarchy, they increase their understanding of the problem, of its context, and of each other’s thoughts and feelings about both [9].

2. Literature Review of AHP Use and Application in Recent Times (Post 2000)

A large number of research papers and articles were studied and analyzed specifically in knowing as to how AHP as decision making tool have been used and applied in recent times. As can be seen the literature review focuses on the latest work and application of AHP especially post 2000.

Walailak & McCarthy (2002) propounded an application of the analytical hierarchy process to international location decision-making [10]. Dey & Prasanta Kumar (2003) discussed about how analytic hierarchy process analyzes risk of operating cross-country petroleum pipelines in India [11]. Yasemin (2006) carried out a case study in Turkey for determining key capabilities in technology management using fuzzy analytic hierarchy process [12].

Management Science (2008) reviewed 15 years―1990 to 2005―of progress through various research papers and study in all areas of multi-criteria decision making showed that AHP publications have far outnumbered those in any other area, characterizing their growth as “enormous” [13]. In 2008, the major society for operations research, the Institute for Operations Research and the Management Sciences formally recognized AHP’s broad impact on its fields [14]. Wallenius, Dyer, Fishburn, Steuer, Zionts & Deb (2008) discussed recent accomplishments in the field of multiple criteria decision making and multi attribute utility theory and also focussed on what lies ahead [15]. Ammar, Hafsa & Ouni (2011) utilized analytic hierarchical process for multi-criteria decision making in design of flying voltage source multi-level inverters [16]. Hwang & Yoon (2012) carried out an extensive survey about various methods and application of multiple attribute decision making esp. AHP [17]. Locatelli & Mancini (2012) used an AHP framework for the selection of the right nuclear power plant [18]. Dymova, Sevastjanov & Tikhonenko (2013) attempted a direct interval extension of TOPSIS method [19]. Triantaphyllou (2013) carried out an elaborate comparative study on multi-criteria decision making methods [20]. Saracoglu (2013) utilized fuzzy AHP in selecting industrial investment locations in master plans of different countries in Eurasia region [21].

Liu, You, Fan & Lin (2014) used AHP for failure mode and effects analysis using D-Numbers and grey relational projection method [22]. Chaudhury, Mandal & Das (2015) used AHP for selection of appropriate fluid delivery technique for grinding titanium Grade-1 [23]. Salavati, Haghshenas, Ghadirifaraz, Laghaei & Eftekhari (2016) carried out a case study of Isfahan city by applying AHP and clustering approaches for public transportation decision making [24]. Li, Hu, Zhang & Deng (2016) used AHP for novel distance function of D-Numbers and its application in product engineering [25]. Mallick, Sarkar & Das (2017) proposed a unified decision framework for inventory classification through graph theory using AHP [26]. Munasinghe, Hemakumara & Mahanama (2017) used AHP in GIS application for finding the best residential lands in Ratnapura municipal council area of Sri Lanka [27]. Zong & Wang (2017) followed a D-AHP approach in their seminal work for evaluation of university scientific research ability based on the output of science and technology papers [28].

3. AHP―Methodology and Application for Selection of a Mobile Phone

The intent of the study is to introduce and elaborate the procedure of analytic hierarchy process based multi criteria decision making process. The following steps discuss the methodology and its implementation as applied to a certain product (mobile phone), thus elucidating the important issues and aspects in building up AHP Model.

The first step is to structure the situation (problem) as a hierarchy containing the decision objective, the option available for achieving the objective, and the grounds on which the alternatives are evaluated. The next step is to set up and create the prioritization levels among the various criteria in the hierarchy by making a series of calculations incumbent (based) on pair wise comparison of the criteria. Once, pair-wise comparison is over, it’s time to synthesize and integrate these conclusions in order to arrive at a set of holistic priority for the hierarchy. This will translate the buyer’s decision for all the products under consideration into overall priorities for each individual product. The last step is to have a look at various values and results generated through the process to arrive at a final decision.

The objective of the study: To select the best mobile phone on the basis of chosen parameters (out of the available alternatives). The models of the mobile phones were chosen with certain minimum common features and if they fulfilled all these aspects only then they were considered for study:

• Cost of the mobile: between INR 25,000 to 45,000 (INR―India Rupees: $1 USA = INR 70);

• Operating system―Android;

• 4G;

• Full HD display;

• Dual SIM card;

• GSM + GSM; and

• Minimum 64 GB.

The final four mobile phones were chosen from four different brands/companies, and fulfilled and met all the above expectations. Four alternatives are referred to as: Brand A, Brand B, Brand C and Brand D in the body of the paper (due to confidentiality clause the brand names of the mobile phone could not be divulged and also the intention of the article is not to choose the brand per se but to explain the workings and methods of the decision tool - AHP).

Criterion for selection: (five)

a) Qualitative: Performance and Specifications, Applications and Functions, Features and Style Quotient

b) Quantitative: Camera power in Megapixels and Cost of the mobile

Pair wise comparisons of all criteria―one against the other. In order to bring in their judgments about various criteria in the hierarchy, decision makers compare the criteria in a pair wise manner. The need was to decide which one of the criteria was more important than the others in selecting the mobile phone model/brand. The decision had to be taken and priorities had to be set, as to which criterion was more important for the decision maker in achieving its objective, and how much more important it was than the other criteria.

Scale of pair-wise comparison: 1―3―5―7―9

• 1―Equal importance―two element contribute equally to the objective

• 3―Moderate importance―experience and judgment slightly favor one element over the other

• 5―Strong importance―experience and judgment strongly favor one element over the other

• 7―Very strong importance―one element is favored strongly over the other; its dominance is demonstrated in practice

• 9―Extreme importance―evidence favoring one element over the other is of the highest possible order of affirmation

4. AHP―Implemented Model for Selection of a Mobile Phone

In order to take a final decision regarding the selection of mobile brand/model, various criteria and their rankings of different models/brands in reckoning are considered and a final judgment is achieved through Analytical Hierarchy Process. To bring in judgments about various criteria in the hierarchy, decision makers compare the criteria in a pair wise manner, this is the first step in AHP. The decision has to be taken and priorities have to be set as to which criterion is more important in achieving the objective, and how much more important it is than the other criteria. The next step is to convert these judgments into priorities for each of these criteria. The detailed procedure and computations are show through various tables, and a detailed explanation is given below:

The first Table is the data available to us through the open source/market regarding the cost of each mobile phone and the camera’s power/size in megapixels, and is shown in Table 1.

The very first step in AHP Process is to the comparison of the criteria considered for the study. This is carried out through a subjective assessment of importance of one criteria over the other referred to as ‘pair-wise comparison’ and the result of which is given in Table 2. Further we convert these values in fractional form to decimal form for the sake of ease and calculations. The result of the same is given in Table 2(a). The next step in AHP is to square the matrix of decimal values (i.e. do matrix multiplication of Table 2(b) and Table 2(b)). The aim here is to calculate the first Eigen Vectors by summing up the rows and normalizing each row value by dividing it by the sum total of the column, we keep using a number of iterations for this step until we don’t see much of a difference between the Eigen vectors. In our case, we use three more iteration to get the accuracy needed. The values of Eigen vectors tell us the relative ranking of the criteria in terms of their importance. Refer to Tables 2(c)-(e). Finally after three iterations we got the values of our Eigen Vectors. In this case, the most important criterion is features and style quotient followed by applications and functions, performance and specifications and then camera power in megapixels. Refer Table 2(f).

On the similar lines, the next step in AHP is to focus on all the qualitative criteria under considerations for our study and make a pair-wise matrix for all these criteria―how different brands rate in comparison to one another in terms of fulfilling these criteria. The result obtained is matrices with pair-wise values for different brands/models of mobile phones under study, and they are shown in Table 3, Table 4 and Table 5.

Table 1. Data available for the Mobile Phones considered for the purpose of study. (Source: secondary sources―local market/internet).

(a) (b) (c) (d) (e) (f)

Table 2. Pair-wise comparison of Criteria considered for the purpose of study. (a) Converting Table 2 Values―Fraction to Decimals; (b) Square of the above matrix; (c) First Iteration for computing row sums and normalized values; (d) Second Iteration for computing row sums and normalized values; (e) Third and Final Iteration―Eigen Vectors representing Rankings of the Criteria; (f) Eigen Vector Values representing Rankings of the Criteria considered for Mobile Phone.

Repeating the steps which get mentioned above to find the relative rankings (Eigen Vectors) for each of these criteria. First we convert all these fraction tables into decimal tables―refer Table 3(a), Table 4(a) and Table 5(a). These tables/matrices helps us in computing different Eigen vectors which determines the relative rankings of mobile phones alternatives (brands/models) under each criterion, viz. considering performance and specifications, applications and functions, & features and style quotient.

For Performance and Specifications: [Tables 3(a)-(f)]

From Series of Table 3, we can conclude that as far as performance and specifications as a criterion is concerned, the models/brands can be ranked as 1) Brand D, followed by 2) Brand B, 3) Brand A and 4) Brand C.

For Applications and Functions: [Tables 4(a)-(f)]

From Series of Table 4, we can conclude that as far as applications and functions as a criterion is concerned, the models/brands can be ranked as―1) Brand A, followed by 2) Brand B, 3) Brand D and 4) Brand C.

(a) (b) (c) (d) (e) (f)

Table 3. Pair-wise comparison for Criteria―Performance and Specifications for all Brands considered. (a) Converting Table 3 Values from Fraction to Decimals; (b) Square of the above matrix; (c) First Iteration for computing row sums and normalized values; (d) Second Iteration for computing row sums and normalized values; (e) Third &Final Iteration―Rankings of the Brands on the criterion of Performance and Specifications; (f) Eigen Vector Values representing Rankings of the Mobile Phone Brands for Performance and Specifications.

(a) (b) (c) (d) (e) (f)

Table 4. Pair-wise comparison for Criteria - Applications and Functions for all Brands considered. (a) Converting Table 4 Values from Fraction to Decimals; (b) Square of the above matrix; (c) First Iteration for computing row sums and normalized values; (d) Second Iteration for computing row sums and normalized values; (e) Third &Final Iteration―Rankings of the Brands on the criterion of Applications and Functions; (f) Eigen Vector Values representing Rankings of the Mobile Phone Brands for Applications and Functions.

(a) (b) (c) (d) (e) (f)

Table 5. Pair-wise comparison for Criteria―Features and Style Quotient for all Brands considered. (a) Fraction to Decimals; (b) Square of the above matrix; (c) First Iteration for computing row sums and normalized values; (d) Second Iteration for computing row sums and normalized values; (e) Third & Final Iteration―Eigen Vectors representing Rankings of Brands for criterion of Features & Style Quotient; (f) Eigen Vector Values representing Rankings of the Mobile Phone Brands for Features & Style Quotient.

For Features and Style Quotient: [Tables 5(a)-(f)]

From Series of Table 5, we can conclude that as far as features and style quotient as a criterion is concerned, the models/brands can be ranked as―1) Brand D, followed by 2) Brand A, 3) Brand B and 4) Brand C.

For Camera’s Power in Megapixels:

This can be used directly as the quantitative data is available (Table 1). As mentioned earlier AHP can both use qualitative and quantitative data; in this case, we simply normalize the camera power information, which makes it simple to use the normalized data with other rankings.

From Table 6, we can conclude that as far as the camera power as a criterion is concerned―the models/brands can be ranked as―1) Brand D, followed by 2) Brand A, 3) Brand C and 4) Brand B. Now that we have all the values of rankings/Eigen vector values for all the brands under different criteria [Table 3(f), Table 4(f), Table 5(f) and Table 6] along with rankings/Eigen vector values the criteria preference [Table 2(f)]. We have to find the product of these two matrices as shown in Table 7. The product of these two matrices gives us the final ranking for these mobile phone brand/models, and this is shown in Table 8. So, as we note the best mobile phone brand/model is 1) Brand D, followed by 2) Brand A, 3) Brand B and 4) Brand C. Refer to Table 8.

Considering Cost Information:

But all said and done, we are still missing out on one important piece of information and that is Cost. Let’s now consider the last of the given data in the form of cost. Cost could have been included, but as you are very well aware, in many complex decision-making situation it is always preferable to put cost aside until the relative benefits and priority rankings of the alternatives are evaluated. It has been seen and the research proves it that lumping the cost of the unit under consideration together with the relative benefits and priorities more often than not is a potent mixture for many politically and emotionally charged debates.

This can be used directly as the quantitative data is available (from Table 1). As we did in the case of camera power, we simply normalize the cost information. Refer to Table 9. And lastly we use, the cost to benefit ratio―the benefits are the same as what we derived in the form of the final rankings (Eigenvectors) for different transformers. Numerical priorities, derived from decision makers’

Table 6. Eigen vector values representing rankings of the mobile phone brands for camera power in megapixels.

Table 7. Matrix multiplication―rank values for brands/models on each criterion and rank values for criteria.

Table 8. Rankings of mobile phone brands/models―result of matrix multiplication.

Table 9. Normalized values/Eigen vectors for cost of each mobile phone.

input, are shown in Table 10. The decision to select “the best mobile phone brand/model” out of four given choices, on the basis of five important criteria. It seems even after consideration of cost vs. benefit analysis that―Brand A is the best of the lot with first priority, followed by Brand D, 3) Brand B and 4) Brand C. Features and style quotient are the most important criterion in making the decision, closely followed by applications and functions, performance and specifications and then camera power in megapixels. Refer to Table 11.

5. Conclusions

It can be said that with the article’s focus on only four mobile phone brands/models it was missing out on opportunities to explore other options. The author is also aware that these were by no means the only feasible ones, but to keep decision making to a less complex level, it decided to compare only these four. The decision making though quantitative in nature, involved a lot of qualitative options and hence a lot of subjectivity.

Table 10. Cost (Eigen vectors from Table 9) vs. benefit (benefits are the rank values/ Eigen vectors from Table 8).

Table 11. Final ranks for mobile phone brands/models by analytical hierarchy process (AHP).

(Source: From Tables 2-11―Computed and Tabulated by Author using AHP).

In this paper an attempt has been made to explain and apply an AHP Model. The techniques and concepts which can be used to address the issue fall under: Analytical Hierarchy Process (AHP)―including hierarchical trees, pair wise comparison, criteria weights, matrix algebra, eigenvector values, and cost-benefit analysis for decision making purposes. The paper focuses on “the use of a decision making technique which decomposes the problem into a hierarchy of more easily comprehended sub-problems, each of which can be analyzed independently”. Although the process is fairly straight forward and the alternative options are easy to comprehend, there are other relevant factors to be kept in mind while making decisions. Also, as to how AHP can incorporate both quantitative and qualitative data in its stride, the study provides a very good foundation for quantitative decision making. The paper attempts to inculcate readers to take a holistic view of complex decision making involving a large number many options and criteria. The paper also makes practitioners think critically about utilizing the AHP tool in various scenarios like production planning, scheduling and allocation of scarce resources, prioritization and ranking.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

Cite this paper

Sharma, J. (2018) Economics of Decision Making: Exploring Analytical Hierarchical Process (AHP). Theoretical Economics Letters, 8, 3141-3156. https://doi.org/10.4236/tel.2018.814195

References

  1. 1. Saaty, T.L. (2001) Fundamentals of Decision Making and Priority Theory. RWS Publications, Pittsburgh.

  2. 2. Saaty, T.L. (2008) Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World. RWS Publications, Pittsburgh.

  3. 3. Poyhonen, M.A., Hamalainen, R.P. and Salo, A.A. (1997) An Experiment on the Numerical Modelling of Verbal Ratio Statements. Journal of Multi-Criteria Decision Analysis, 6, 1-10.

  4. 4. Saaty, T.L. and Peniwati, K. (2008) Group Decision Making: Drawing out and Reconciling Differences. RWS Publications, Pittsburgh.

  5. 5. Salem, O., Salman, B. and Ghorai, S. (2017) Accelerating Construction of Roadway Bridges Using Alternative Techniques and Procurement Methods. Transport, 33, 567-579. https://doi.org/10.3846/16484142.2017.1300942

  6. 6. Madurika, H. and Hemakumara, G. (2015) Gis Based Analysis for Suitability Location Finding in the Residential Development Areas of Greater Matara Region. International Journal of Scientific & Tech Research, 4, 96-105.

  7. 7. Saaty, T.L. (2010) Principia Mathematica Decernendi: Mathematical Principles of Decision Making. RWS Publications, Pittsburgh.

  8. 8. Saaty, T.L. (2008) Relative Measurement and its Generalization in Decision Making: Why Pairwise Comparisons are Central in Mathematics for the Measurement of Intangible Factors—AHP. Review of the Royal Academy of Exact, Physical and Natural Sciences: Mathematics (RACSAM), 102, 251-318.

  9. 9. Drake, P.R. (1998) Using the Analytic Hierarchy Process in Engineering Education. International Journal of Engineering Education, 14, 191-196.

  10. 10. Atthirawong, W. and McCarthy, B. (2002) An Application of the Analytical Hierarchy Process to International Location Decision-Making. 7th Annual Cambridge International Manufacturing Symposium, Cambridge, 1-18.

  11. 11. Dey, P.K. (2003) Analytic Hierarchy Process Analyzes Risk of Operating Cross-Country Petroleum Pipelines in India. Natural Hazards Review, 4, 213-221. https://doi.org/10.1061/(ASCE)1527-6988(2003)4:4(213)

  12. 12. Erensal, Y.C., Oncan, T. and Demircan, M.L. (2006) Determining Key Capabilities in Technology Management Using Fuzzy Analytic Hierarchy Process—A Case Study of Turkey. Information Sciences, 176, 2755-2770. https://doi.org/10.1016/j.ins.2005.11.004

  13. 13. Bhushan, N. and Rai, K. (2004) Strategic Decision Making: Applying the Analytic Hierarchy Process. Springer-Verlag, London.

  14. 14. Forman, E.H. and Gass, S.I. (2001) The Analytical Hierarchy Process—An Exposition. Operations Research, 49, 469-487. https://doi.org/10.1287/opre.49.4.469.11231

  15. 15. Wallenius, J., Dyer, J.S., Fishburn, P.C., Steuer, R.E., Zionts, S. and Deb, K. (2008) Multiple Criteria Decision Making, Multiattribute Utility Theory: Recent Accomplishments and What Lies Ahead. Management Science, 54, 1339-1340. https://doi.org/10.1287/mnsc.1070.0838

  16. 16. Ben Faouzi, A., Hammami, H. and Faten, O. (2011) Analytic Hierarchical Process for Multicriteria Decision Making in Design of Flying Voltage Source Multilevel Inverters. European Journal of Electrical Engineering, 14, 719-756.

  17. 17. Hwang, C.L. and Yoon, K. (2012) Multiple Attribute Decision Making: Methods and Applications a State-of-the-Art Survey. Vol. 186, Springer Science & Business Media, Berlin.

  18. 18. Locatelli, G. and Mancini, M. (2012) A Framework for the Selection of the Right Nuclear Power Plant. International Journal of Production Research, 50, 4753-4766. https://doi.org/10.1080/00207543.2012.657965

  19. 19. Dymova, L., Sevastjanov, P. and Tikhonenko, A. (2013) A Direct Interval Extension of TOPSIS Method. Expert Systems with Applications, 40, 4841-4847. https://doi.org/10.1016/j.eswa.2013.02.022

  20. 20. Triantaphyllou, E. (2013) Multi-Criteria Decision Making Methods: A Comparative Study. Vol. 44, Springer Science & Business Media, Berlin.

  21. 21. Saracoglu, B.O. (2013) Selecting Industrial Investment Locations in Master Plans of Countries. European Journal of Industrial Engineering, 7, 416-441. https://doi.org/10.1504/EJIE.2013.055016

  22. 22. Liu, H.C., You, J.X., Fan, X.J. and Lin, Q.L. (2014) Failure Mode and Effects Analysis Using D Numbers and Grey Relational Projection Method. Expert Systems with Applications, 41, 4670-4679. https://doi.org/10.1016/j.eswa.2014.01.031

  23. 23. Ayan, C., Bijoy, M. and Santanu, D. (2015) Selection of Appropriate Fluid Delivery Technique for Grinding Titanium Grade I Using the AHP. International Journal of the Analytic Hierarchy Process, 7.

  24. 24. Salavati, A., Haghshenas, H., Ghadirifaraz, B., Laghaei, J. and Eftekhari, G. (2016) Applying AHP and Clustering Approaches for Public Transportation Decision Making: A Case Study of Isfahan City. Journal of Public Transportation, 19, 38-55. https://doi.org/10.5038/2375-0901.19.4.3

  25. 25. Li, M., Hu, Y., Zhang, Q. and Deng, Y. (2016) A Novel Distance Function of D Numbers and Its Application in Product Engineering. Engineering Applications of Artificial Intelligence, 47, 61-67. https://doi.org/10.1016/j.engappai.2015.06.004

  26. 26. Bivash, M., Bijan, S. and Santanu, D. (2017) A Unified Decision Framework for Inventory Classification through Graph Theory. International Journal of the Analytic Hierarchy Process, 9.

  27. 27. Munasinghe, D.S., Hemakumara, G.P.T.S. and Mahanama, P.K.S. (2017) GIS Application for Finding the Best Residential Lands in Ratnapura Council Area of Sri Lanka. International Research Journal of Earth Sciences, 5, 11-22.

  28. 28. Zong, F. and Wang, L. (2017) Evaluation of University Scientific Research Ability Based on the Output of Sci-Tech Papers: A D-AHP Approach. PLoS ONE, 12, e0171437. https://doi.org/10.1371/journal.pone.0171437