Received 12 January 2016; accepted 8 March 2016; published 11 March 2016

ABSTRACT

In the coming Micro-Communication era, many Sci-Tech periodicals have successively registered WeChat Official Accounts and then how to objectively evaluate the communication effects of the WeChat Official Accounts of Sci-Tech periodicals becomes the biggest concern at present. This thesis analyzed the existing communication effect evaluation system of WeChat Official Accounts of Sci-Tech periodicals and its problems, and put forward the following improving strategies: rebuilding a new evaluation system of WeChat communication effect of Sci-Tech periodicals via the selection of evaluation indexes, the division of assessment index weight and the calculation method of evaluation index, then testing the reliability and efficiency of the new evaluation system, and finally putting forward the corresponding policy recommendations.

Keywords:

Sci-Tech Periodical, WeChat Communication Effect, Evaluation System, Evaluation Index

1. Introduction

According to the definition of Sci-Tech periodical in the official document “Science and Technology Periodical Management Regulations”, Sci-Tech periodical is a serial publication which holds the fixed publication format and reports science and technology as the main content, and an effective media which is used to publish scientific research achievements for professionals and spread the scientific and technical information. Due to the strong professional of the scientific achievements and technological information, Sci-Tech periodicals were limited in the audience scope and audience structure to a certain extent. In response to this phenomenon, the major Sci-Tech periodicals have opened WeChat Official Accounts to enhance their communication effect. By taking “Qiushi Journal Online” WeChat Official Account for example, it was authenticated since April 18, 2013, and within one week, such as 2015.11.29-2015.12.05, the reading number on its recommended articles had come to 50,140 and showed a trend of increasing day by day. It has got more attention on WeChat than paper periodical, which has greatly improved the communication effect.

With the diversification of WeChat’s application functions and the pluralization of information content and form, the previous evaluation system of WeChat communication effect of Sci-Tech periodicals is facing huge challenges. Combined with the audience’s growing appetite for quantity and quality of the technological information and the fierce competition for information resources among those major WeChat Official Account platforms of Sci-Tech periodicals, the defects of current evaluation system are more prominent. In order to ensure the comprehensive and objective of the evaluation system of WeChat communication effect of Sci-Tech periodicals, this thesis tried to correct existing evaluation system, reset new indexes and form the index weight by using new calculation method, and then took the empirical test for the effectiveness of the new evaluation system, and finally put forward policy suggestions to promote WeChat communication effect of Sci-Tech periodicals.

2. Current Situation of WeChat Communication Effect Evaluation of Sci-Tech Periodicals in China

2.1. Academic Research Status

Since the launch of Wechat 1.0 beta on January 21, 2011, the influence of WeChat communication has been continuous expansion and extension, while the academic research on Sci-Tech periodical WeChat communication started in 2013 (Hu Haixia, 2013; Sun Qianjun, 2013) . Through the analysis of WeChat and Weibo transmission effect, Shi Jing and some other scholars summarized four key elements, namely user, graphic, message and interface, which influence the WeChat communication, and gave prospects for the dissemination of new media of Sci-Tech periodicals (Shi Jin et al., 2014) . Zhong Yonggang appealed for great attention on communication of visual language, professional interest of information, and full interaction among users, authors and editors to strengthen the Wechat communication effect of Sci-Tech periodicals (Zhong Yonggang, 2015) . By comparing Chinese and foreign social media application of medical Sci-Tech periodicals, Cheng Qihou pointed out the shortages of Micro-Communication of Chinese Sci-Tech periodicals, and put forward four suggestions for improvement (Cheng Qihou & Zhang Jing, 2015) .

Overall, there are two main directions of this part in current academic research: one is the discussion on WeChat communication model and its influencing factors; the other is the discussion on WeChat communication characteristics and development strategy in various Sci-Tech fields. At present, few research are carried out on evaluation system of WeChat communication effect of Sci-Tech periodicals, besides existing evaluation index still stays at the explicit level and has not yet formed a unified evaluation criterion.

2.2. Actual Application Status

After more than four years’ development, the total registered users of WeChat (more than 600 million) has exceeded the total amount of Weibo total registered users (more than 500 million) (Zhan Xun and Yan Xing, 2014) , and becomes an important medium for information communication in all areas. So far, many institutions, such as Aiweibang, Public Opinion Monitoring Center of People’s Daily Online, New Rank Index, and Index of New Media and so on, have evaluated the Wechat communication effect, carried out the index ranking, and set up corresponding evaluation column for Sci-Tech periodicals. But these independent institutions respectively set up their own evaluation system, which leads to very different results finally. In addition, these evaluation systems mostly tend to select indexes from the aspect of audience while ignore the others (see Table 1), and also not give explicit description of the correlation degree between indexes according to the actual situation of WeChat communication effect in each field (Li Mingde and Gao Ru, 2015) .

It can be seen that there are obviously insufficient of research on evaluation system of WeChat communication effect of Sci-Tech periodicals in both academia and the industry, which also directly affects the credibility of the practical application. To this end, this article will analyze from two perspectives of both audience and

operator and try to construct a more scientific and authoritative evaluation system by using the Delphi Method, Analytic Hierarchy Process Method and Efficiency Coefficient Method so that to achieve the goal of measuring the WeChat communication effect of Sci-Tech periodicals.

3. Reconstitution for Evaluation System of WeChat Communication Effect of Sci-Tech Periodicals

To realize objective and accurate evaluation for WeChat communication effect of Sci-Tech periodicals, firstly, we need to clear the evaluation indexes of it. According to the above status analysis in theory and practice, this article added three indicators―originality degree of articles’ content (Zhang Haidong and Sun Jihua, 2015) , push rate of articles, abundance degree of information form (Song Qifan, 2015) to existing evaluation indexes and further classified all indexes into two categories―the audience aspect and the operator aspect. However, to build a scientific, authoritative and suitable evaluation system of WeChat communication effect for Chinese Sci-Tech periodicals, we conduct it from the following three processes: 1) select diversified evaluation indexes; 2) set reasonable weight of every index; 3) determine scientific calculation method of the indexes.

3.1. The Selection of Evaluation Indexes

Now, the interaction degree between disseminators and recipients is strengthening continuously in “Internet +” era (Sara K. Yeo et al., 2015) . To ensure that the evaluation index data obtained is more targeted, the selection principles of evaluation index should be: 1) give consideration to both operators and audiences (Zhang Yanping, 2015) ; 2) directly related to WeChat communication effect of Sci-Tech periodicals (Ma Yong et al., 2014) ; 3) can be quantified and counted.

Operators, who are the “gatekeeper” of information push, lead the development direction of WeChat. Although WeChat official accounts’ operators of Sci-Tech periodicals can’t put too much subjective philosophy in the process of information transmission, they can promote the WeChat communication effect of Sci-Tech periodicals by improving relevant indicators. Operators could measure the WeChat communication effect of Sci-Tech periodicals from the following five indicators: originality degree of articles’ content, total push frequency of articles, push rate of articles, total number of push articles and abundance degree of information form. The public is final host of WeChat communication of Sci-Tech periodicals, and audience’s behavior and its result is an important measurement of WeChat effective communication of Sci-Tech periodicals. The influence from audiences to WeChat communication effect of Sci-Tech periodicals can be quantitative analysis from following six indicators: total number of reading, average reading number every day, highest reading number of one article, total number of “like”, average number of “like” on one day, highest number of “like” on one article (see Table 2).

3.2. Division of Evaluation Index Weight

Weight setting of primary indexes (A and B in Table 2) comes to the first. Along with the internet permeability to people’s daily life constantly, information communication mode is transiting from one-way closed communication

Note: As the strong “closure” in WeChat communication environment, some indexes are difficult to obtain, such as number of forward, number of fans, number of comments, etc. So these indexes haven’t put into this evaluation system.

to two-way open communication, and the role of disseminators and recipients transformed from immobilized mode to automatic mode. The interactive structure between the two bodies becomes “decentralization”, and the status of the two parties gradually tends to be equal in the process of information dissemination in network (Wang Guoyan and Han Feifei, 2014) . Based on the current development situation, this article used the Fuzzy Comprehensive Evaluation Method to quantize evaluation index weight of A and B, whose result is that weight coefficient for both A and B is quite approaching, and the weight coefficient of index B is slightly higher than the weight coefficient of index A(W_B − W_A < 0.01). In view of this, this article set the same weight for index A and index B as 50%.

Next is the weight setting of secondary indexes (11 indexes in Table 2: X1 − X11). To avoid uncertainty and subjectivism in weight setting, this article used a combined method of qualitative and quantitative analysis. Firstly, we used Delphi Method and questionnaire survey to obtain the evaluation of importance degree of each index from experts and audiences, and on the basis of this evaluation, we use the Analytic Hierarchy Process to quantify the importance degree of every two indicators to the number 1 - 9 and its reciprocal as the scale of the quantitative statistics (Zhang Zhaoguo et al., 2011) . By pair wise comparing the sub indexes for operators (A) and the audiences (B), we got the following judgment matrix:

Finally, we used Arithmetic Average Method to get the weight coefficient (represented by W_i in Table 3, i = 1 - 9) of the secondary indexes (Deng Xue et al., 2012). As you can see in Table 3, the largest eigenvalue (λ_max) of the two matrix are 5.1571 and 6.2853 respectively. Now that the Consistency Ratio of both sides (0.0351 and 0.0453 respectively) is all less than 0.10, it can be concluded that the weighting coefficient of the secondary indexes is reasonable.

3.3. Calculation Method of Evaluation Index

Whether the calculation method of evaluation indexes is objective or not will directly affect the final evaluation

result. Now that the value emphasis and operation way are very different in different fields of WeChat communication, and the information attribute in Sci-Tech journals has distinct difference compared to other media’s, it is not suitable to adopt a unified and absolute evaluation standard, but should take relative evaluation in each field. After selecting different indexes and setting their weight, we used the Efficacy Coefficient Method to calculate the effect of different functions, and built a new evaluation index (Z_i) for measuring WeChat communication effect of Sci-Tech periodicals, whose computational formula is as follows:

,

,

In this formula, X_i stands for the actual value of each index; X_min and X_max respectively stand for the minimum value and maximum value in the whole index ranking of each industry; W_i is the weight of each evaluation index. To translate the Efficacy Coefficient into centesimal system, the two constant terms were set to 60 (as benchmark) and 40.

The calculation method of evaluation index of WeChat communication effect of Sci-Tech periodicals should tend to focus on the authenticity and scientificity of information communication, fully assess the impact of every evaluation indexes on the efficiency value, and make full use of empirical research methods to comprehensively evaluate the effect. From the perspective of operators, the effect evaluation is aim to present the operation state comprehensively of each WeChat public platform in Sci-Tech industry, so it is a comprehensive measure of the function application, service efficiency, supervision level of the Sci-Tech periodicals’ WeChat platform. But from the perspective of audiences, the effect evaluation is aim to understand the information feedback of various types of audience groups, as well as to adjust the communication model of WeChat timely according to audience’s feedback (Gillian Pearson, 2001) .

4. Empirical Test for Effectiveness of the New Evaluation System

After building a new evaluation system, how to test its effectiveness has become the next problem. To this end, we can make a comparative research between the results of the new evaluation system and the evaluation results of the Index of New Media (it contained 276,337 WeChat official accounts till December 7th, 2015, which is far higher than other evaluation agencies). We selected 10 active WeChat official accounts of Sci-Tech journals, whose Composite Impact Factor ranked top 1000 in Annual Report for Chinese Academic Journals Impact Factors (Social Science) (Vol 12, 2014), as samples for analysis. And then we collected relevant data according to the above 11 index sequence (see Table 4).

Putting the above index data of 10 samples into the new evaluation index formula Z_i, we calculated the new evaluation index of WeChat communication effect for each Sci-Tech periodical, and compared the new ranking in descending order with the original ranking as shown in Table 5. It can be seen that the index value under the new standard are decreased significantly, and the numerical gap between these values also decreased because of the centesimal transformation of quantitative values under Effectiveness Function. The previous top 3 index values of WeChat official account of Sci-Tech journals―QS theory, China University Education, Journal of Chinese Distance Education still retain the original ranking; but the new ranking of Economic research changes greatly because its two values―“originality degree of articles’ content” and “abundance degree of information form” are 0 in the form, which dragged down the overall index; other 6 Sci-Tech periodicals’ index ranking just appears slight amplitude adjustment. In general, it has new changes in the results of the new evaluation indexes,

Data source: Annual Report for Chinese Academic Journals Impact Factors (Social Science) (Vol 12, 2014).

but not dramatic changes. From a macro perspective, the new evaluation results can reflect the WeChat communication effect of Sci-Tech periodicals more completely.

5. Conclusions and Policy Recommendations

WeChat public platform of Sci-Tech periodicals is an integration product of traditional media and new media, and plays an extremely important role in ensuring the continuous and stable development of Sci-Tech periodicals while extends the information transmission chain. Based on the defects of current evaluation system of WeChat communication effect of Sci-Tech journals, this article builds a new evaluation system from two perspectives of the audiences and the operators comprehensively, and also realizes the following innovation: 1) it introduces new indicators, increases the number of ratio indicators which make the index values relatively stable; 2) it distributes indicators into two sides―the operators and the audiences, and reflects the same importance; 3) it changes the way of calculation, uses centesimal system, makes it more acceptable subjectively, and also uses the Efficacy Function to give the benchmark points which manifest the basis value of the Sci-Tech journals who open WeChat official accounts and spread information; 4) the index calculation process needs to refer to cross- reference data of peer industry, and the relative but not absolute results are more conducive to peer comparison. Finally, through the empirical test, we conclude that the new evaluation system is effective.

To those difficult problems that appeared in the process of building the new evaluation system, this article put forward the following two suggestions.

The first is expanding the scope of the evaluation objects, and speeding up informatization operation. According to the survey of above 1000 Sci-Tech periodicals, few of them open their WeChat official accounts and most of them are still in the traditional information communication phase at present. Some of them have opened the WeChat official accounts while there are also lots of “zombie accounts”, which makes data collection for evaluation objects become a difficult problem. However, to obtain more accurate evaluation results, we have to expand the scope of evaluation objects and collect data widely, or the final evaluation results could hardly reflect the real communication situation objectively, but also go against the improvement and adjustment of the evaluation index. In addition, in order to optimize the WeChat communication effect of Sci-Tech periodicals, we need to enrich the content and forms of Sci-Tech information to expand the periodicals’ brand influence; and meanwhile, we need to strengthen the communication between the operators and the audiences to stabilize target audience. This is not only beneficial to enrich information communication mode for Sci-Tech periodicals but also important to promote the transformation development of Sci-Tech periodicals.

The second is enhancing the credibility of evaluation behavior, and promoting the standardized management. To make the evaluation system of WeChat communication effect of Sci-Tech periodicals more authoritative, we should establish a professional assessment organization, which can be elected from the bottom up, or be operated by a third party authority, or be original established by the common competent department. Besides, the relative staffs in it should have different knowledge background, different identity, different experience and some other factors, and they are the “opinion leaders” in each field, who can make professional and comprehensive evaluation, and provide the most objective evaluation results to the public finally. It can not only reduce the consumption of manpower, material and financial resources in the process of evaluation, but also effectively purify science communication environment and prevent each of them releasing different index evaluation that causes the loss of evaluation credibility, and further promote the sustainable development of information resources from WeChat official accounts of Sci-Tech journals. It is worth noting that, as the current limitation of the data acquisition and statistical techniques, part of the indicator data such as the number of fans for WeChat official account has been made public, which has certain influence on the comprehensive evaluation. Therefore，in the future research, it is necessary to further improve the evaluation system of WeChat Communication Effect of Sci-Tech periodicals.

Acknowledgements

This work was supported by “Study on the Improvement of the Scientific Quality of Citizens and the Environment of Scientific Communication in Anhui Province” (Soft Science Research Project of Anhui Province in 2015), Project Number: 1502052008.

Cite this paper

FeifeiHan,ZhangzhiGe,RongtingZhou, (2016) Reconfiguration and Empirical Study on the Evaluation System of WeChat Communication Effect of Sci-Tech Periodicals. Advances in Journalism and Communication,04,8-15. doi: 10.4236/ajc.2016.41002

References