Here’s a blog post by Boris Barbour about living with DORA. I guess we’re seeing some of the ideas in practice, but is it really going to be better?
Less frequent evaluation — I guess this makes sense for everyone who’s currently undergoing annual evaluations for the sake of it. Evaluations when you need them seems logical. But then this means we’re moving even more to an in/out system: once you’re in, you’re in, and you’re free to do whatever. Didn’t get the grant, well you’ll have to live with a smaller research team or more teaching. What about those who are not “in”, who don’t have a tenured position or a clear perspective (a.k.a. “track”) of obtaining one? You’re going to be “out” no matter.
What’s your biggest achievement? — The Swiss National Science Foundation is using this, asking for the three biggest achievements. I’m confused about this. In the context of a job application, isn’t that the cover letter, where you’d highlight just that? Again, this seems fine for those who are “in”. Not getting any publications out, then you can highlight your influence on public policy or something like that. It’s a “pick your own” evaluation thing. What about those who are not “in”? You’re going to try and second guess which of your achievements the evaluators may find relevant. You have all the incentives to inflate your achievements, and none to be humble about what you contributed to the team. If we consider fraud to get publications out, why not consider it here?
Looking at how they write about their own research on social media can indeed be personal and revealing, but if we know that this is (possibly) evaluated, we’re changing the incentive structure.
An interesting thought I found was the idea to ask letters of recommendations from those we mentored and supervised in the past. This is the only idea where the assessment of “quality” does not rely on the candidate’s ability to “brag” and present their achievements in a particular light, or the evaluator being intrinsically familiar with the work and thus able to assess the “quality” directly.
Didier Ruedin 