Lin-Jia Zhao, Wei-Hua Zeng?, Jia-Rui Zhang, Jin-Nan Wangand Hong-Qiang Jiang

1School of Environment, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, China；

2Key Laboratory of Environmental Planning and Policy Simulation, Chinese Academy for Environmental Planning, No. 8 Dayangfang Beiyuan Road, Beijing 100012, China

?

Evaluation Method for Science and Technology Performance on Energy Conservation and Emission Reduction

Lin-Jia Zhao1, Wei-Hua Zeng?1, Jia-Rui Zhang1, Jin-Nan Wang2and Hong-Qiang Jiang2

1School of Environment, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, China；

2Key Laboratory of Environmental Planning and Policy Simulation, Chinese Academy for Environmental Planning, No. 8 Dayangfang Beiyuan Road, Beijing 100012, China

Submission Info

Communicated by Yan Hao

Performance evaluation

Science and technology

Energy conservation and emission reduction

DEA

This study provided an Energy Conservation and Emission Reduction（ECER） evaluation method to identify the role and situation of science and technology （S&T） in ECER. The ECER evaluation method was based on the data envelopment analysis （DEA） model and other statistical methods. A case study was applied to evaluate the S&T ECER performance of 31 provinces of China from 2006 to 2012. And the evaluation results shown that （1） the S&T ECER performance of nearly 40% Chinese provinces were relatively effective and Chinese provinces could be classified into three groups, including good region, general region and poor region. （2）The SO2emission reduction effect was the most in need of improvement to improve the performance efficiency. （3） The S&T investment played an important role in ECER because there was a positive correlation between the S&T investment scale and the ECER performance efficiency.

? 2015 L&H Scientific Publishing, LLC. All rights reserved.

1 Introduction

High pollution and energy consumption are unavoidable problems in rapidly industrializing countries（Casazza et al., 2013）. Science and Technology （S&T）, in addition to cleaner production or terminal treatment approaches, has played an important role in reducing energy consumption and pollution emissions. Therefore, we need to evaluate the energy conservation and emission reduction （ECER） performance of S&T to find ways to improve its efficiency.

Performance evaluation is a commonly used management tool and can provide a basis for management decisions. Performance evaluation refers to the observation, identification and measurement of the performance of people or specific processes as well as the evaluation of the effects of the input and output activities. In recent years, performance evaluation methods have been widely applied in the behavior management of governments and enterprises as well as human resources management, environmental science studies and scientific studies in some special areas （Boiral et al., 2012； Chen et al., 2006； Coelho et al., 2012；ISO, 2013； Jasch, 2000； Omidvari, 2014）. However, most of these studies used qualitative or quantitative weight methods with a small number of indicators. Unfortunately, the use of a small number of indicatorsmay not adequately reflect the real situation, and the weight method is somewhat subjective, which could influence the accuracy of the evaluation results.

In contrast, the data envelopment analysis （DEA） method is a linear programming model for measuring the relative efficiency scores of decision making units （DMUs） regarding multiple inputs and outputs （Chen et al., 2012）. DEA has been used in many fields as an evaluation tool （Cook et al., 2010）. In the field of performance evaluation, Odeck （2000） applied DEA and the Malmquist index to assess the relative efficiency and productivity growth of vehicle inspection services. Ramanathan （2006） employed DEA to study the comparative economic and social performance of selected Middle Eastern and North Africa countries. Eric Wang and Weichiao Huang （Wang et al., 2007） evaluated the relative efficiency of research and development（R&D） activities across countries using DEA and the Tobit regression method. The global warming and environmental production efficiency of the Kyoto Protocol nations have been ranked using DEA （Feroz et al., 2009）. Wei Zhong et al. （2011） used DEA models to evaluate the relative efficiencies of 30 regional R&D investments using data from the First Official China Economic Census in 2004. Picazo-Tadeo et al （2011）assessed farming eco-efficiency using the DEA technique.

All of the above studies have suggested that as a non-parametric analysis method, the DEA method or its combination with other tools is a valid tool for ranking different related entities with multiple input and output characteristics （Angulo-Meza et al., 2002）.

China is one of the largest developing countries in the world, and due to the coal-dominated energy structure and industrial pollution, China is facing serious air pollution problems （Sueyoshi et al., 2015）. Chinese policy makers have already recognized the important role of S&T in ECER in response to increasingly serious environmental problems. Despite the use of terminal treatment technologies, administrative means or market means, such problems continue to grow. Therefore, the role of S&T must now be considered and promoted.

Few studies have evaluated the role of S&T on the ECER. However, as the development statuses of the Chinese provinces are unequal, the current situation must be accurately assessed for policy makers to plan the development direction of S&T in the future.

Therefore, the aims of this study were to provide an evaluation method based on applied DEA method and other statistical approaches to evaluate the S&T performance on ECER and to provide a basis for making policy decisions. Thirty-one provincial administrative regions of China （excluding Hong Kong, Macau, and Taiwan for statistical caliber issues） were used as examples.

The remainder of this study is structured as follows： Section 2 describes the evaluation approach and its application, the data collection and the model parameters； Section 3 presents the results and analyzes the pollution reduction efficiency performance of 31 Chinese provinces； Section 4 includes the discussion and conclusions.

2 Materials and methods

2.1 The ECER evaluation method

The ECER evaluation method is based on the DEA model. DEA is a non-parametric approach that does not require any assumptions about the functional form of a production function or information on the importance of inputs and outputs （Jia, 2012； Lee et al., 2009）. The DEA model is used to empirically measure productive efficiency of decision making units （DMUs） through the DEA score which is modeled and analyzed by various inputs and outputs. The common DEA model is described in many other studies （Heidari et al., 2012；Li et al., 2003； Wang et al., 2007； Zhou et al., 2008）.

The framework of the ECER evaluation method for S&T performance evaluation provided herein is described in Fig. 1. This method can obtain evaluation results through data collection, modeling and analysis processes. Among them, the analysis process includes POST values accounting and another three statistical methods slack variable analysis, cluster analysis and two-dimension distribution analysis.

Fig. 1 ECER evaluation method framework for S&T performance evaluation

In the data collection stage, an evaluation input/output indicator system should be constructed to standardize the data collection format.

During the modeling process, the data quality should be tested. And then, the DEA model parameters should be set according to the evaluation aims. The efficiency of the DEA model is defined by technical efficiency （TE） （Mohammadi et al., 2013） and can be described by the indicator named POST （the Performance of S&T）. TE is a measure by which DMUs are evaluated for their performance relative to the performance of the other. The TE can be defined as follows：

where θjis the TE score of DMU j； x and y represent input and output values, respectively； v and u denote the input and output weights, respectively； i （1, 2, ..., m） and o （1, 2, ..., n） are the number of inputs and outputs, respectively. To solve Eq. （1）, linear programming was formulated as follows （Cooper et al., 2011）：

Maximize：

Subject to：

Here, the TE score, which ranges between 0 and 1, as the value of POST.

The analysis process includes cluster analysis, slack variable analysis and two-dimensional distribution analysis. The cluster analysis results provide a visual ranking of the POST of the DMUs, which can sort the units by their performance. The slack variable analysis results indicate the improvement ways of low efficiency units. The two-dimensional distribution analysis shows the effect of total S&T investment scale.

2.2 Evaluation indicators

The input and output evaluation indicators are shown in Table 1. These indicators are a comprehensive reference of national emission reduction targets, statistics, and existing studies. All data were obtained from Chinese national statistical data of 2004 to 2012. In addition, all of the indicators are quantitative to avoid generating subjective error.

Table 1 Input/output evaluation indicators

There are four inputs and four outputs. The inputs signify the S&T investments, and the outputs signify the results of the investments.

The inputs include human resource and fund input, which are the major form of S&T input. The human resource inputs can be quantified as the environmental protection personnel （EPP） per ten thousand people reflecting the professional environmental personnel density of a province, and the EPP average annual salary reflecting the human resource cost. The fund inputs can be quantified by the R&D investment and pollution control investment as a proportion of the GDP. The former input indicates the total R&D scale and the latter input indicates the environmental protection investment.

The outputs include indicators of the GDP output per unit pollutant （COD, SO2and NH3-N） and energy, which reflect the pollution emission reduction and the energy saving effects.

Large amounts of outputs using small amounts of inputs are indicative of high efficiency. Therefore, for all of the indicators, the inputs are “l(fā)ess is better” and outputs are “more is better” according to the principle of the DEA method （Luo et al., 2012） and are related to allow comparison among provinces.

In addition, S&T performance has the lag characteristic meaning that this year's input may not produce immediate outputs （Chen et al., 2009）. In this study, we supposed that S&T inputs have effects two years later. For example, inputs in 2004 would produce effects until 2006. Therefore, the input data ranged from 2004 to 2010, and the output data ranged from 2006 to 2012.

2.3 Method parameters

We set the 31 provincial administrative regions of China as the DMUs, excluding Hong Kong, Macao, and Taiwan because these areas have different statistical calibers at present.

The DEA models can be divided into input orientation and output orientation. The choice mainly depends on the controllability and treatability of the selected inputs/outputs. If the inputs must not feature major changes or must be maintained at a certain level, then the output orientation is more appropriate and vice versa （Wei, 2004）. In our research, the inputs were relatively fixed. Therefore, we selected the output orientation model.

DEAP 2.1, which was developed by Professor Tim Coelli at the University of Queensland, Australia, was used for calculating the DEA model. In addition, Win4DEAP 1.1.2 was used as a user-friendly interface for the DEAP 2.1 visualization operation （Coelli, 1996）.

2.4 Data quality test

The correlation of the indicators must be analyzed before the other calculations are performed. To reflect the independence of the inputs, the correlation between input indicators should be weak. Similarly, the correlation between output indicators should be strong to reflect their interrelationship （Wu et al., 2007）. Using the 2012 data for an example, correlation analyses （Li et al., 2015） were performed on the inputs and outputs using the Spearman test method.

Table 2 Descriptive statistics on inputs and outputs of Chinese provinces

Table 3 Nonparametric Correlations of the input indicators

Table 4 Nonparametric correlations of the output indicators

Tables 3 and 4 show that only input indicators 2 and 3 had a weak, rather than negligible, correlation, but that each pair of output indicators was strongly correlated. The analysis results indicated that the data met the computing requirements.

3 Results

3.1 POST analysis results

Applying the CCR model （Gattoufi et al., 2004） and multi-stage calculation approach, we collected seven years of input and output data and used DEAP 2.1 software to calculate the POST values of 31 provinces of China. The results are shown in Table 5 and Fig. 2.

In Table 5, the POST values of 31 provinces and six regions are listed according to the Chinese national statistical system classification. Furthermore, the results of Table 5 are summarized in Fig. 2, which depicts the POST value means and ranking trends for the six regions.

The results showed that over the seven years, the POST values changed slightly indicating that the overall situation was relatively stable. Northwest China, which is generally less developed than the rest of China, differed significantly from other regions. Moreover, the rank of East China was highly consistent, and the rank of Northeast China was increasing over time. There was relatively little difference among the other regions.

In the next three sections, we explore the POST values accounting results thoroughly using cluster analysis, slack variables analysis, and two-dimensional distribution analysis. By slack variable analysis, the weak points of DMUs can be found. By cluster analysis, which DMUs have the same POST values may be classified. By two-dimensional distribution analysis, the relationship between POST values and investment scale can be described clearly.

3.2 Cluster analysis results

Using the data from 2006 and 2012, we applied the K-means clustering algorithm to identify the rank of each province nationwide. The cluster results classified the POST of Chinese provinces into three categories, including the good region, the general region and the poor region. The good region represents the provinces where S&T inputs could obtain much better outputs. Thus, these provinces should maintain the inputs to make improving their environmental quality. The general region represents the provinces whose efficiencies were neither better nor worse and could rise or fall. Finding improvement potentials is important for these regions. The poor region was the provinces with lower pollution reduction efficiency and their efficiency need improving.

Table 5 Provincial administrative performance evaluation score

Figs. 3 and 4 clearly show that （1） 18 DMUs （58%） belonged to the good regions, 10 belonged to the general regions, and 3 belonged to the poor regions in 2006. In 2012, 20 DMUs （65%） belonged to the good regions, 8 belonged to the general regions, and 3 belonged to the poor regions. So, the S&T ECER performances of most regions were efficient. （2） The regions with lower POST values were concentrated on Northwest, Southwest, and Northeast China, where are less developed. Northwest China's performance was poor with GS, QH, and NX being ranked in the poor regions in both years and SN being demoted from the good regions to the general regions. （3） In contrast, East China had relatively good performance. Therefore, the environmental managers should pay more attention to S&T development in less developed regions. Furthermore, comparing the 2006 and 2012 data, the cluster analysis results were similar, indicating that China's S&T ECER performance was improved nationwide and was relatively stable throughout the studied years.

Fig. 2 S&T performance scores by region over seven years

Fig. 3 Cluster analysis results of POST values

Fig. 4 National distribution of cluster analysis result of China

3.3 Slack variable analysis results

By slack variable analysis, the DEA model provides insights on which indicators need to be improved and information on how to improve them. In the DEA method, slack variables represent the redundancy of the inputs or outputs and reflect the distance through which an inefficient DMU becomes an efficient one.

Using the 2012 data as an example, we calculated the slack variable of 11 general or poor efficient provinces as listed in the above section. The analysis results are summarized in Table 6.

Table 6 Slack variable analysis of lower efficiency DMUs

Based on Table 6Table 6 Slack variable analysis of lower efficiency DMUs, the worst output indicator of the 11 lower efficiency provinces was GDP output per unit SO2emissions, indicating that the main gap between S&T performance efficient and inefficient provinces was the S&T performance on SO2emission reduction. Hence, the S&T inputs of those provinces have great improvement potential for SO2emission reduction.

The three provinces which had the worst POST values were GS, QH and NX. In addition to the GDP output per unit SO2emissions, all the other three output indicators of these provinces also needed to be improved greatly.

Fig. 5 R&D investments and the POST value distribution

3.4 Two-dimensional distribution analysis results

Because the analytical data were normalized, the above studies considered the relative scale of DMUs but did not include the effect of their absolute total size. However, we must consider the difference of the POST values caused by the different sizes of the DMU inputs. Therefore, two-dimensional distribution analysis （Liu et al., 2013） was used to analyze the relationship between the R&D investment scales and POST values.

A two-dimensional distribution scatterplot for 2012 is shown in Fig. 5 using the R&D investment of 2010 as the X-axis and the POST value of 2012 as the Y-axis. Additionally, the investment data were processed in logarithmic form because the differences of the investment sizes among provinces were large. The dotted line perpendicular to the X-axis is the median of logarithmic investments, and the dotted line perpendicular to the Y-axis is the mean of POST values.

As shown in the Fig. 5 and compared with the cluster analysis results, （1） the provinces with strong S&T and high investment, such as BJ, GD, JS, SD, and SH, had better performance. （2） In contrast, due to their low pollution levels, provinces with input scales that were relatively smaller, such as XZ and HI, could also obtain good ECER performance. （3） Additionally, provinces with weak S&T and low investment, such as XJ, GS, QH, and NX, had worse performance. These findings reflected that S&T input has an important role in promoting the energy conservation and pollution reduction level, and regions where pollution are particularly severe need more S&T input.

3.5 Overview of the evaluation results

To evaluate the performance of S&T on ECER, we used the ECER evaluation method based on the DEA model and other statistical approaches. If the S&T ECER performance of a region was efficient, it indicates that the S&T played an important role in promoting pollution reduction. In contrast, if the performance was inefficient, how to improve it is needed to be analyzed. Based on the above four analysis results, we can evaluate the S&T ECER performance comprehensively.

At the national level, the POST analysis results showed that the S&T ECER performance of most regions was efficient over seven years. Comparing the data of 2006 and 2012, the cluster analysis results indicatedthat China's S&T ECER performance was improved nationwide and was relatively stable within these years.

At the regional level, the POST value of Northwest China, which is generally less developed than the rest of China, was lower than the other regions. The performance of Northeast China increased over time, and East China had relatively good performance all along.

At the provincial level, the provinces with lower POST values were concentrated in Northwest, Southwest, and Northeast China, where are less developed. GS, QH, and NX provinces were ranked in the poor regions in both 2006 and 2012, and SN was demoted from the good regions to the general regions. There were 7 provinces, namely BJ, FJ, JX, SD, GD, HI, and XZ, which had relatively efficient POST values in all seven years. The following 12 S&T inefficient provinces had relatively inefficient POST values in all seven years：SX, LN, HL, AH, HB, HN, GZ, SN, GS, QH, NX, and XJ. The remaining 12 provinces, including TJ, HE, NM, JL, SH, JS, ZJ, HA, GX, CQ, SC, and YN, were considered potential S&T efficient provinces.

4 Discussion

4.1 Relationship between the S&T investment level and the ECER performance

The two-dimensional distribution analysis and Fig. 4 shown that the overall S&T ECER performance of a province had a positive correlation with its S&T investment level. The efficient provinces concentrated in the developed East and South coastal regions. The inefficient provinces were concentrated in West and Northeast China, which are underdeveloped regions.

As shown in Fig. 5, the R&D investment had an important role in promoting the energy conservation and pollution reduction level. Provinces with high S&T levels and large investment scales would have better ECER performance. In contrast, provinces with low S&T levels and small investment scales would have worse performance. Regions where pollution was particularly severe require more S&T input. Moreover, some provinces whose industrial levels were relatively lower also had good performance because they had low pollution degrees and lower S&T investments thus obtain relatively high ECER outputs.

4.2 Policy suggestion

The results of the slack variable analysis showed that the main gaps between S&T performance efficient and inefficient provinces was the SO2emission reduction performance. SO2is an important material that causes air pollution, and the low SO2reduction efficiency may be one of main reasons that have recently caused haze problems in many Chinese cities. So, the S&T inputs of provinces may focus on cleaner energy alternatives and SO2decontamination technology.

Furthermore, Policy makers and environmental managers should strengthen S&T investments in the underdeveloped regions, pay more attention to the S&T development of less developed regions and maintain the inputs in the more developed regions, because the S&T investment and development level have a positive correlation with the ECER performance.

5 Conclusions

S&T is an important driving force for ECER, which is important for industrial sustainable development. Evaluating the ECER performance of S&T is important for policy makers and environmental managers of developing countries because energy and pollution problems are inevitable during the industrialization process and S&T is becoming a major force in ECER. This study provided an ECER performance evaluation method based on the DEA model and other auxiliary statistical methods, such as cluster analysis, slack variable analysis and two-dimensional distribution analysis, to evaluate the S&T performance on ECER.

A case study that evaluated the ECER performance of 31 Chinese provinces from 2006 to 2012 wasapplied. The evaluation results showed that the relative efficiency of China's overall S&T performance on ECER was good and that most Chinese provinces stay at the same level during these years. The Chinese provinces could be clustered as three groups by their POST values. The relatively developed eastern provinces, such as BJ, GD, JS, SD, and SH, had better performance, while the relatively undeveloped western provinces, such as XJ, GS, QH, and NX, had worse performance.

Importantly, performance evaluation can help identifying problems. In the Chinese S&T ECER performance evaluation, （1） SO2reduction efficiency is the main weakness； （2） the S&T investment scale and the ECER performance had a positive correlation, and more S&T inputs could help to increase the pollution reduction efficiency. Therefore, reducing SO2emissions measures and the S&T investments on the underdeveloped regions should be strengthened. In conclusion, the S&T ECER evaluation method may become a good tool for policy makers and environmental managers of China and other developing countries.

Acknowledgement

The research is supported by the Chinese special scientific research project for environmental protection and public welfare （No. 201209037） and the Chinese National Water Pollution Control and Management Technology Major Projects （No. 2012ZX07102-002-05）. We also gratefully thank the anonymous reviewers for their comments and suggestions.

Angulo-Meza, L. and Lins, M. P. E. （2002）. Review of Methods for Increasing Discrimination in Data Envelopment Analysis. Annals of Operations Research， 116（1-4）, 225-242.

Boiral, O. and Henri, J. F. （2012）. Modelling the impact of ISO 14001 on environmental performance： A comparative approach. Journal of Environmental Management， 99, 84-97.

Casazza, M., Gilli, G., Piano, A. and Alessio, S. （2013）. Thirty-years Assessment of Size-fractionated Particle Mass Concentrations in a Polluted Urban Area and Its Implications for the Regulatory Framework. Journal of Environmental Accounting and Management， 1（3）, 259-267.

Chen, A., Tang, W. and Zhang, D. （2009）. Research on system operation performance evaluation. Beijing： Science Press.

Chen, M. Y. and Chen, A. P. （2006）. Knowledge management performance evaluation： a decade review from 1995 to 2004. Journal of Information Science， 32（1）, 17-38.

Chen, P. C., Chang, C. C., Yu, M. M. and Hsu, S. H. （2012）. Performance measurement for incineration plants using multi-activity network data envelopment analysis： The case of Taiwan. Journal of Environmental Management， 93, 95-103.

Coelho, H. M. G., Lange, L. C. and Coelho, L. M. G. （2012）. Proposal of an environmental performance index to assess solid waste treatment technologies. Waste Management， 32（7）, 1473-1481.

Coelli, T. （1996）. Guide to DEAP， Version 2.1： A Data Envelopment Analysis （Computer） Program. University of New England.

Cook, W. D., Liang, L. and Zhu, J. （2010）. Measuring performance of two-stage network structures by DEA： A review and future perspective. Omega， 38（6）, 423-430.

Cooper, W. W., Seiford, L. M. and Zhu, J. （2011）. Handbook on data envelopment analysis. New York： Springer.

Feroz, E. H., Raab, R. L., Ulleberg, G. T. and Alsharif, K. （2009）. Global warming and environmental production efficiency ranking of the Kyoto Protocol nations. Journal of Environmental Management， 90（2）, 1178-1183.

Gattoufi, S., Oral, M. and Reisman, A. （2004）. A taxonomy for data envelopment analysis. Socio-Economic Planning Sciences， 38, 141-158.

Heidari, M. D., Omid, M. and Mohammadi, A. （2012）. Measuring productive efficiency of horticultural greenhouses in Iran： A data envelopment analysis approach. Expert Systems with Applications， 39（1）, 1040-1045.

ISO. （2013）. ISO 14031： Environmental management - environmental performance evaluation - guidelines （ 2nd edition）.

Jasch, C. （2000）. Environmental performance evaluation and indicators. Journal of Cleaner Production， 8（1）, 79-88.

Jia, Y. P. and Liu, R. Z. （2012）. Study of the Energy and Environmental Efficiency of the Chinese economy based on a DEA Model. Procedia Environmental Sciences， 13, 2256-2263.

Lee, H., Park, Y. and Choi, H. （2009）. Comparative evaluation of performance of national R&D programs with heterogeneous objectives： A DEA approach. European Journal of Operational Research， 196（3）, 847-855.

Li, M. and Chen, G. （2003）. A review on the research and application of DEA. Engineering Science， 5（6）, 88-94.

Li, L., Lei, Y. and Pan, D. （2015）. Study on the Relationship between Coal Consumption and Economic Growth： Based on the Data from 1981-2011 of China. Journal of Environmental Accounting and Management， 3（1）, 69-75.

Liu, G. Y., Yang, Z. F., Chen, B. and Ulgiati, S. （2013）. Analysis of the scientific collaboration patterns in the Emergy Accounting field： A review of the co-authorship network structure. Journal of Environmental Accounting and Management， 1（1）, 1-13.

Luo, Y., Bi, G. and Liang, L. （2012）. Input/output indicator selection for DEA efficiency evaluation： An empirical study of Chinese commercial banks. Expert Systems with Applications， 39, 1118-1123.

Mohammadi, A., Rafiee, S., Jafari, A., Dalgaard, T., Knudsen, M. T., Keyhani, A., Mousavi-Avval, S. H. and Hermansen, J. E. （2013）. Potential greenhouse gas emission reductions in soybean farming： a combined use of Life Cycle Assessment and Data Envelopment Analysis. Journal of Cleaner Production， 54, 89-100.

Odeck, J. （2000）. Assessing the relative efficiency and productivity growth of vehicle inspection services： An application of DEA and Malmquist indices. European Journal of Operational Research， 126（3）, 501-514.

Omidvari, M. and Ghandehari, M. （2014）. Urban Environmental Management Performance Assessment by Fuzzy Analytical Hierarchy Processing. Journal of Environmental Accounting and Management， 2（1）, 31-41.

Picazo-Tadeo, A. J., Gómez-Limón, J. A. and Reig-Martínez, E. （2011）. Assessing farming eco-efficiency： A Data Envelopment Analysis approach. Journal of Environmental Management， 92（4）, 1154-1164.

Ramanathan, R. （2006）. Evaluating the comparative performance of countries of the Middle East and North Africa： A DEA application. Socio-Economic Planning Sciences， 40（2）, 156-167.

Sueyoshi, T. and Yuan, Y. （2015）. China's regional sustainability and diversified resource allocation： DEA environmental assessment on economic development and air pollution. Energy Economics， 49, 239-256.

Wang, E. C. and Huang, W. （2007）. Relative efficiency of R&D activities： A cross-country study accounting for environmental factors in the DEA approach. Research Policy， 36（2）, 260-273.

Wei, Q. （2004）. Data envelopment analysis. Beijing： Science Press.

Wu, H. and Liu, S. （2007）. Evaluation on the R&D Relative Efficiency of Different Areas in China Based on Improved DEA Model. R&D Management， 19（2）, 108-112.

Zhong, W., Yuan, W., Li, S. X. and Huang, Z. （2011）. The performance evaluation of regional R&D investments in China： An application of DEA based on the first official China economic census data. Omega， 39（4）, 447-455.

Zhou, P., Ang, B. and Poh, K. （2008）. Measuring environmental performance under different environmental DEA technologies. Energy Economics， 30（1）, 1-14.

3 January 2015

.

Email address： zengwh@bnu.edu.cn

ISSN 2325-6192, eISSN 2325-6206/$- see front materials ? 2015 L&H Scientific Publishing, LLC. All rights reserved.

10.5890/JEAM.2015.03.006

Accepted 6 May 2015

Available online 1 October 2015