2017 ESA Annual Meeting (August 6 -- 11)

COS 133-7 - The steering effects of research evaluation: Lessons from Brazil

Thursday, August 10, 2017: 10:10 AM
B114, Oregon Convention Center
Mark Neff, Environmental Studies, Western Washington University, Bellingham, WA
Background/Question/Methods

Almost universally, governmental spending on science is justified by assertions that the resulting research will yield benefits for the taxpayers who ultimately fund it. Prominent scientists and science policy makers in the United States have argued for the past half century that these benefits accrue most efficiently when scientific efforts are self-governed, with scientists and their communities making decisions about research questions and methods via internal mechanisms. This ideal has diffused to other settings, and the metrics of scientific productivity and merit implicit within that model have followed. Brazil, the Latin American country with the largest research budgets and outputs, has institutionalized these norms in their science policies. I present the results of a study that seeks to document the impacts of Brazilian research evaluation schemes on the content and process of ecological research in that country. I derive my conclusions from an analysis of policy documents and of 30 semi-structured hour-long interviews with Brazilian scientists, research managers, and knowledge users. This study is one component of an overall project designed to highlight strengths and weaknesses of science policies in a variety of national settings; the Brazilian approach to research evaluation offers lessons that can inform science policies in other contexts.

Results/Conclusions

Brazilian science policies are designed with the intent of boosting the country’s prominence and reputation in international science. They have been successful in that regard: Brazilian research output has steadily climbed international rankings. With that increased productivity, however, come several unintended consequences. Individual graduate programs are evaluated every three years by a central agency: CAPES (Coordenação para o Aperfeiçoamento de Pessoal de Nível Superior/Coordination for the Advancement of Graduate Education) with the intent of ensuring and improving their quality. CAPES tri-annual evaluations have the unintended effect of asserting a strong conservative bias on ecological research, reifying traditional disciplinary boundaries, and systematically discouraging engagement with stakeholders and work that spans multiple field seasons or might not yield sufficient publications and citations. Students have strong incentives to conduct research with fast turn-around times, encouraging conflation of “fast” and “good” science. Brazil has novel mechanisms lacking in other Latin American settings that foster development of journals of national and regional interest, but ecologists do not fully accept the merits of these publication outlets. To the extent that Brazil succeeds in fostering journals of national and regional interest, it can advance its own interests while benefiting other Latin American nations.