Volume 12, Issue 1, June 2017

In the last decade, research assessment has become a crucial dimension of academic life, in Italy as elsewhere in Europe. Today, career advancements, research fund granting or department ranking, are increasingly dependent on research assessment as one of the key indicators. Individual and collective evaluations are thus mixed up to achieve the declared objectives of more efficient use of public funds, on the one hand, and more innovative and advanced research, on the other.

This IPS issue hosts the contributions presented at the Round Table on research assessment in a comparative perspective, held at the SISP annual conference, in September 2016. The IPS community will certainly benefit from the experience of the distinguished international scholars who participated in the Round Table, namely Matthew Flinders (PSA), Rudy Andeweg (ECPR and Dutch association for political science), Catherine Paradeise (Professor emerita UPEM-LISIS) and Daniele Checchi (member of GEV-13 for Research Quality Evaluation (VQR) 2004-2010). SISP President Simona Piattoni, who organized this Round Table, is the guest editor of this issue. We thank them all for having contributed to the on-going debate, providing food for thought from the European academic environment.

This is not the first time that IPS has focused on research evaluation and VQR. The very first issue coordinated by the current IPS co-editors was devoted to “Evaluating the Evaluation. The pros and cons of ‘VQR’ in social and political research”, in IPS Volume 8, Issue 1 of June 2013, which addressed controversial issues such as the conceptualization of quality (i.e. quality as an objective fact or as a social construction), the ex post adoption of evaluation criteria, the fact that “the number of citations may reflect the level of subordination of scholars who cite, instead of indicating the degree of innovation, originality and explanatory power of the cited publication”, etc.

Four years later, the third VQR assessment edition has ended, and research assessment as a tool has been generally accepted by Italian scholars (to be precise, fewer than 10% of them abstained from the VQR). Yet, the debate on research assessment in social sciences is still faced with several dilemmas: qualitative versus quantitative evaluation, quantitative indicators privileging single products or journal quality (e.g. journals with IF), the definition of internationalization of research, which does not simply mean publishing in English, etc. Alongside that, periodic research assessment has an impact on academia both in individual and systemic terms, on an individual’s academic profession and on departments in terms of ranking and fund distribution.

It remains to be seen whether an article is quoted because of the popularity of the author, the relevance of the topic, the impact of the journal or the quality of the article itself. Paradoxically, once peer-review has become widespread, recruiting peer-reviewers has become a hard task, because peer-review is not regarded as a rewarding activity, it is a ‘hidden’ prestige and not many scholars are happy to remain behind the scenes. Moreover, peer-review methods rely upon the subjectivity of reviewers. The introduction of a peer to peer-review might be useful to evaluate the evaluators.

The comparative perspective provided by this IPS issue helps to foster the Italian debate on research assessment and to better understand the criticisms of research assessment, which can be useful in academic advancement and should not risk becoming an individual or systemic hindrance.

Stefania Panebianco and Francesco Zucchini, IPS Co-Editors

SPECIAL FOCUS ON:

The impact of Research Assessment on the Profession and the Discipline of Political Science: a Roundtable

Guest Editor: Simona Piattoni, University of Trento, and President of the Società Italiana di Scienza Politica (SISP)

Proceedings of the roundtable held at the SISP Annual Congress, Department of Social and Political Sciences, University of Milan, 15 September 2016

Several European countries – UK, Sweden, Spain, Norway, Netherlands, Italy, Ireland, Hungary, Germany, France, Finland and Belgium – conduct by now periodic research assessment exercises. In particular, the Italian university system has already conducted three such rounds of assessment, the last two (2004-2010, 2011-2014) according to particularly formalized procedures. And yet the very idea of assessing the scientific production as if it were a specific output of the university systems still meets with considerable resistance and skepticism. Many doubt that such assessment has any consequence at all (while it does have a financial repercussion on the distribution of funds from the Ministry of Education, which departments can use to expand their teaching staff) and most fear that it simply lends itself to constructing rankings of little scientific significance, but potentially great political import. The controversy is particularly intense in the social sciences and humanities, therefore in political science, an area in which the so-called bibliometric indicators are more difficult to apply and assessment is mostly qualitative. We think that the time has come for a collective reflection on the pros and cons of such exercises and on the potential repercussions that they may have on the academic profession and the discipline of political science.

We have invited four distinguished colleagues from four different university systems – UK, Netherlands, France and Italy – to discuss about these issues and to bring their particular experience and points of view to fruition of the Italian political science community. We asked them to comment on and report their experience on the following aspects of research assessment:

  1. Have research assessment exercises in your country been met with enthusiasm and collaboration or with suspicion and resistance? What were the arguments pro and against? What was the return rate?
  2. Which aspects have been pinpointed as being particularly problematic: a) use of quantitative indicators (such as single product/journal impact factor); b) internationalization (often coinciding with “publishing in English”); c) publishers’ prestige; etc.?
  3. Which consequences – monetary or otherwise – have these exercises had on: a) single scholars; b) departments; c) universities?
  4. Which aspects have been reformed/improved from one round to the next? Have the problems encountered in early rounds been amended in successive rounds?
  5. What impact have these exercises had on the academic profession in political science? Have they prompted a higher rate of international submissions? Have they improved overall production rates? Have they encouraged publications of journal articles as opposed to monographs?
  6. What is your overall assessment of research assessment in your country?

Roundtable participants:

  • Prof. Matthew Flinders (Political science, University of Sheffield, UK), Chair of the Executive Committee of the Political Studies Association (PSA).
  • Prof. Rudy Andeweg (Political science, Leiden University, Netherlands), Chair of the Executive Committee of the European Consotrium for Political Research (ECPR).
  • Prof. Catherine Paradeise (Sociology, Université Paris Est-Marne-la-Vallée), Professor emerita UPEM-LISIS (Laboratoire Interdisciplinaire Sciences Innovations Sociétés).
  • Prof. Daniele Checchi (Economics, University of Milan), member of the Group Expert Evaluator in Economics (GEV-13) for the Research Quality Evaluation (VQR) 2004-2010.

Moderator:

  • Prof. Simona Piattoni (Political science, University of Trento), President, Società Italiana di Scienza Politica, member of the Group Expert Evaluator in Political and Social Sciences (GEV-14) for the Research Quality Evaluation (VQR) 2011-2014.

The Impact of Research Assessment on the
Profession and the Discipline of Political Science

Simona Piattoni (University of Trento)

At the 2016 SISP annual meeting, in Milano, we held a roundtable on the topic of research assessment in comparative perspective. Leading European scholars, both expert of evaluation and with significant experience at the helm of their respective national associations, took part in this roundtable […]

The Tragedies of Political Science: The Politics of Research Assessment in the United Kingdom
Matthew Flinders (University of Sheffield)

The Tragedy of Political Science is the title of a 1984 book by David Ricci that made a bold argument concerning the evolution of the discipline. Ricci’s thesis, put simply, suggested that as political science had become more ‘professionalised’ throughout the twentieth century […]

Two Decades of Political Science Research
Assessment: the Dutch Experience

Rudy B. Andeweg (Leiden University)

Today, there is a general acceptance of, or at least resignation about, Dutch Research Assessments, whether in political science or in other disciplines. Research assessment exercises started in the Netherlands in 1993, and are held every six years. […]

The French HER system and
the issue of evaluation

Catherine Paradeise (Université Paris Est-Marne-la-Vallée)

The French higher education and research system is based on a double divide of missions, statuses and recruitment procedures. On one side, over 30 public research organizations – whose permanent scientific staff varies from about 300 to over 12.000 – differentiate themselves from higher education (HE) institutions. […]

The Italian Research Assessment Exercises
Daniele Checchi (University of Milan)

talian universities have so far experienced three assessment exercises (2001-3, 2004-10 and 2011-14). The fiscal law approved in December 2016 dictates that from now onwards the reference periods will be quinquennial, reducing the discretionary power so far exercised by the Ministry of Education in designing the exercise. […]

Book Reviews Edited by Carla Monteleone and Stefania Panebianco

Manuela Caiani and Simona Guerra (eds.), Euroscepticism, Democracy and the Media
Reviewed by Ornella Urso (Scuola Normale Superiore, Pisa)

Paolo Chiocchetti, The Radical Left Party Family in Western Europe, 1989-2015
Reviewed by Carlo Baccetti (University of Florence)

Stefania Panebianco (ed.), Sulle onde del Mediterraneo. Cambiamenti globali e risposte alla crisi migratoria
Reviewed by Eugenio Cusumano (Leiden University)

Gianluca Passarelli (ed.), The Presidentialization of Political Parties. Organizations, Institutions and Leaders
Reviewed by Luca Verzichelli (University of Siena)

%d bloggers like this: