ABSTRACT

The current evaluation system of College Students’ innovation and entrepreneurship training program is not conducive to the healthy development of the training project, such as heavy project, light process and fuzzy evaluation subject. In order to improve the implementation of the training program, process oriented evaluation system should be established, concluding indicators selected from initiation process, operation process and final process, and weights are calculated by Delphi method. Process management and classified criterion to finish process are suggested to improve the effectiveness of College Students’ innovation and entrepreneurship training program.

Keywords:

Process Oriented, College Students’ Innovation and Entrepreneurship Training Program, Evaluation System

1. Introduction

Undergraduate Training Programs for Innovation and Entrepreneurship (hereinafter referred to as Daiso training program) is directly hosted by Chinese Ministry of Education, aiming at enhancing the practical capability of university students. However, although most of Chinese universities have paid lots of attention to promoting the program, a so-called “emphasizing on outcomes while neglecting process” phenomenon has become very common but unwanted. As a result, students engaging in Daiso training program tend to pursue quick success and instant benefit, and lack in-depth research thinking and initiative to explore, thus lead to an ineffective result of Daiso training program (Wang & Xie, 2012) . Exploring the causes of this phenomenon, it may be related to its overemphasis on outcome in the current evaluation system of Daiso training program. Therefore, there is an urgent need to establish a process- oriented evaluation system for Daiso training program.

At present, researches on the evaluation system of Daiso training program mainly focused on how to select the evaluation index. Current views can be generally categorized into three types: the first type only considers one participant in project evaluation. For example, Shao advised to build the evaluation system based on the duties of supervisors, such as supervisors’ attitude, experience and capability (Shao et al., 2013) . Qi proposed to evaluate from perspective of student, his evaluation system involved students’ creativity, communication frequency, teamwork, and experiment conductions (Qi & Lu, 2009) . Wang advocated to include experimental conditions into evaluation system (Wang et al., 2013) , while Jing stated that it was also useful to consider universities, enterprises, and the degree of coordination between universities and enterprises, as well as innovation as evaluation indicators (Jing et al., 2014) . The second view believes the evaluation system should be composed of evaluations referring to multiple participants. For example, Du stated that the evaluation system of Daiso training program should include multiple participants such as supervisors, students and universities (Du & Ye, 2013) . Yang argued that the selection of evaluation index should be considered from the establishment of innovation team perspective (Yang et al., 2009) . The third view suggested selecting evaluation index based on operating process of Daiso training program, and including multiple participants to conduct evaluation. Li argued to select evaluation index according to three stages, involving topic selection, implementation and outcomes. And in the topic selection stage, experts should be the raters, while supervisors should conduct evaluation in implementation stage, and in final outcome stage, students, supervisors and experts should be all involved (Li et al., 2010) . Huo and his colleagues (2015) also pointed out that evaluation should not be limited and focused on the application and conclusion reviews, the whole process including the application, implementation and result presentation should be taken into account when selecting key evaluation indicators.

Previous researches have provided a variety of ideas concerning Daiso training project evaluation, as well as rating scales of the evaluation system, which laid a good foundation for further research in Daiso training program evaluation system. We find that most researchers conduct their evaluation focusing on only one participant, such method is simple and straightforward, but cannot help to solve the current “emphasizing on outcomes while neglecting process” problem (Du & Ye, 2013; Yang et al., 2009) . Lee and his colleagues therefore proposed a process-oriented evaluation system for Daiso training programs, which reduced above problem to a certain extent (Li et al., 2010; Huo et al., 2015) , but lacked a comprehensive evaluation system involving different participants, which might lead to a failure in distinguishing the duties between the various participants in each stage, and the rating scale they proposed might turn out to be not very practical. In view of the above-mentioned facts, we believe a process- oriented, quantitative evaluation index system, which serves multiple evaluation bodies, could promote the development of evaluation system of Daiso training programs.

2. Analysis of the Process Evaluation Index

2.1. Analysis of the Process Evaluation Index at Topic Selection Stage

Evaluation in topic selection process refers to reviews on project feasibility conducted by a panel of experts, aiming at determining which projects to finance, and the funding quota, including reviews of project proposals and project application defense.

In topics selection process, quantitative evaluation is conducted focuses on students, supervisors and experimental conditions. Evaluation of students is mainly related to the innovativeness, feasibility and expected practical values of the project proposals, as well as students’ knowledge and skill levels. Evaluation of supervisors focuses on supervisors’ professional guiding capability and responsibility, such as, whether supervisor’s background, field of study and research experience are relevant to the selected topics, and supervisor’s work styles. Evaluation of experimental conditions refers to aspects involving laboratory equipment, materials, facilities and environment, as well as the process management. It especially focuses on whether the existing experimental conditions can meet the requirements of experiments, and whether there are alternative solutions when those requirements cannot be satisfied.

2.2. Analysis of the Process Evaluation Index at Project Implementation Stage

Evaluation in project implementation process refers to the process of students engaging in research projects under supervisor’s guidance. The management party of the project supervises the process by mid-term examination, which is critical for the evaluation of Daiso training program.

In this stage, quantitative evaluation should be conducted involving three angles: 1) Evaluation of the students. Students are expected to conduct research in accordance with the preset goals and time schedule mentioned in project proposal. Evaluation of students should pay attention to students’ problem solving capability, innovative methods adopting, effort commitment, progress of project implementation and experiment records and summary or introspection of training and outcomes. 2) Evaluation of the supervisor. In project implementation process, the tasks of supervisor involves guiding students to master proper research methods, inspiring students to think creatively, encouraging and guiding students to overcome difficulties, and helping students to learn from experience. Accordingly, evaluation of supervisors should take into account the number of supervisor’s guidance, the extent to which that students accept to supervisor’s heuristic teaching, and other aspects of supervisor’s performance such as the degrees of innovative thinking, rationality of resource deployment. 3) Evaluation of the management. The main tasks of management party is to provide necessary space, material supports and process controls in order to ensure the smooth implementation of the project. Therefore, evaluation of management should consider the quality, quantity, timeliness and effective supervision of funds, space and equipment.

2.3. Analysis of the Process Evaluation Index at Results Reporting Stage

Evaluation in results reporting stage is the conclusive evaluation of the quality of Daiso training programs. In results reporting process, students are required to submit a project self-assessment report, project supervisor evaluate the team performance during the whole project, and project review panel conduct an overall evaluation on project results, including both a written review and a presentation evaluation. In particular, the written review adopts a peer review approach in line with the academic fields, while presentation evaluation requires a face-to-face defense concerning the implementation and results of the project.

Evaluation of results reporting process should focus on the performance of team members during the written review and presentation evaluation. Evaluation of written review mainly involves student’s understanding of problem solving plans, the completion of project preset targets, innovation performance during problem solving process, writing skills, and supervisor’s evaluation about the overall performance of students. The key elements of presentation evaluation involve student’s elaboration on problem solving process of main issues, innovativeness of the solutions, degree of independence in research, teamwork capability, value of research results, and performance during the defense.

3. Weights of Evaluation Scales

3.1. Sample and Procedures

In design of evaluation index, researchers commonly use Delphi method to weight different indexes, so as to reflect the relative importance of different indicators. For project evaluation experts, we collected data from experts that have rich experience in guiding, evaluating and organizing of Daiso training programs, their opinions tend to have high reliability and validity. In order to enhance the representativeness, based on the “list of advanced units in 2014 National University Students Innovation and Entrepreneurship Training Program” from Chinese Ministry of Education, we selected four outstanding universities to conduct questionnaire survey (two universities directly under the Ministry of Education: Huazhong University of Science and Technology, Central South University; and two local universities: Hunan Normal University, Yangtze University).

The questionnaire is divided into three parts. The first part is personal information, including gender, age, education and university. The second part concerns the personal opinions on the proportions of topic selection, project implementation and results reporting process in Daiso project evaluation, while the third part requires respondents to weight different indicators. Questionnaires are collected on-site, a total of 40 validated questionnaires are achieved.

3.2. Calculation of the Weights

In this research, weights of indicators are calculated using the weighted average method. Firstly, calculate the average scores. The weight of each secondary index (U_ij, i, j = 1, 2, 3…) is the average score of 40 experts, and the weight of the first level index is obtained by aggregating the secondary index weights (U_i, i = 1, 2, 3…). Secondly, adjust the differences. Given that the second part of the questionnaire has provided the expert’s evaluation of the first level indexes (x_i, i = 1, 2, 3…), the results may differ from the aggregated weights we have obtained. Hence, if there are differences, we calculate and use the average of U_i and x_i as the weight of the first level index. Thirdly, determine the final weight. For convenience, we use integer weights, and excluding indexes that weight less than 5%.

4. Establishing the Evaluation Index System

Index system consists of indexes and weights. Due to the limits of the paper, a evaluation index system we proposed for Daiso training program (science and engineering category) is presented in three tables as follows (Tables 1-3).

Evaluation index in this paper is selected according to three stages, including topic selection, project implementation and result reporting. The overall scale is constituted by 8 first level indexes and 39 secondary indexes. Table 1 is evaluation system of topic selection process, there are 3 first level indexes including 13 secondary indexes, the weight of the 3 first level indexes is 30 percent. Table 2 is evaluation system of project implementation, there are 3 first level indexes including 16 secondary indexes, the weight of the 3 first level indexes is 40 percent. Table 3 is evaluation system of result reporting, there are 2 first level indexes including 10 secondary indexes, the weight of the 3 first level indexes is 30 percent.

The quantitative evaluation of each index can be described by the following formula:

(1)

where in, X represents the total score of the project implementation process, and represent the first level and secondary index weights.

5. Conclusions

The paper obtains the Process Oriented evaluation system of Undergraduate Training Programs for Innovation and Entrepreneurship. Through analyzing the process evaluation index at topic selection stage, project implementation stage and results reporting stage, we get 8 first level indexes and 39 secondary indexes. Then, Delphi method is used to weight different indexes by collecting data from experts. Finally, we get 3 tables by combining these indexes and weights, which is the Process Oriented evaluation system of Daiso training program.

Our research can be used in practice. It can be used to evaluate Daiso training program to find problems and verify who should undertake responsibility, then we know how to do better. If someone doesn’t finish Daiso training program, but his evaluation is excellent, we can provide further support in fiancé, encourage him to put it off, even allow him to successfully end the program in order to encourage innovation.

There are two limitations in our research. The first one is that all indexes are researched by analyzing theoretically not empirically, so its reliability and validity need further confirmation. The second one is the small sample size; weight is calculated by scores rated by 40 experts from four universities. Though we avoid wrongs by choosing experts from different universities which are excellent in Daiso training program evaluation, their representativeness is also a problem.

Cite this paper

Sheng, Y. Y., Yu, Q. S., & Chen, L. Y. (2016) A Study on the Process Oriented Evaluation System of Undergraduate Training Programs for Innovation and Entrepreneurship. Creative Education, 7, 2330-2337. http://dx.doi.org/10.4236/ce.2016.715225

References

NOTES

¹Project supported by Humanities-Society Scientific research program of Hubei provincial education department (Research on promoting transformation of achievement of process-based undergraduate training program for innovation and entrepreneurship, No.16Q086), teaching research program of Yangtze university (research on Evaluation system for process-based undergraduate training program for innovation and entrepreneurship, No.JY2014026), Humanities-Society Scientific research program of Yangtze university (research on Evaluation system for achievements-based undergraduate training program for innovation and entrepreneurship, No.2015csy013).