Yesterday, HEFCE released the REF2021 Decisions on staff and outputs. There’s a lot to chew over here, including information about whose outputs can be submitted where (portability), and how many outputs and impact case studies are required for a Unit of Assessment. But I want to focus on a rather niche issue that arose in REF2014. If I’m reading this right, the issue seems to be worse under the new system.
To see the problem, let’s think about a fictional Unit of Assessment (UoA) with 10 members of staff, all of whom are full-time and none of whom have mitigating circumstances — no early career, carer or parental leave, etc. (This simplifies the point I’m making, but the point holds just as well for more normal departments, which have part-time staff and staff with mitigating circumstances.) Each member of staff in this fictional UoA has four outputs that will be rated 3* and one that will be rated 4*. (Again, a simplification, but again the point generalises and in fact gets worse for departments with lower than this fairly high performance.) Now suppose that they decide to hire five research superstars from very prestigious institutions abroad on fractional contracts of 20% — these researchers will typically be drawn from the US, since it is possible for academics to retain full-time employment in the US whilst undertaking 20% employment in the UK. And let’s suppose that each of these researchers has one output that will be rated 3* and four that will be rated 4*. Now consider the percentage of 3* and 4* outputs this UoA will receive in REF2014 and in REF2021:
- In REF2014, each standard member of staff would submit three 3* outputs and one 4* output, while each fractional superstar would submit four 4* outputs. Thus, they would have (10×3) = 30 outputs at 3* and (10×1) + (5×4) = 31 outputs at 4*. So, in REF2014, they’d have roughly 50% of their outputs at 3* and 50% at 4*.
- In REF2021, in contrast, they need (2.5×11) = 27.5 outputs in total. Each member of staff must submit at least one output, and no member of staff can submit more than five. So this fictional UoA would submit each of the 4* outputs by their standard members of staff, giving them 10 outputs; and they’d submit the remaining 17.5 outputs from amongst the twenty 4* outputs produced by their fractional superstars. So, in REF2021, they’d have 100% of their outputs at 4*.
Something similar happens if there are 20 standard members of staff in the department and 5 fractional superstars:
- This time, in REF2014, they would have (20×3) = 60 outputs at 3* and (20×1) + (5×4) = 40 outputs at 4*. So, in REF2014, they’d have 60% of their outputs at 3* and 40% at 4*.
- In REF2021, in contrast, they need (2.5×21) = 52.5 outputs in total. So they’ll submit the 4* output from each of their standard staff, giving them 20 outputs at 4*. And they’ll submit the 20 outputs at 4* from their fractional superstars. And they’ll make up the rest from 3* outputs, so they’ll submit 12.5 outputs at 3*. So, in REF2021, they’d have around 25% of their outputs at 3* and 75% at 4*.
To understand what worries people about this possibility, we’re often asked to consider what the department’s REF score would look like if, instead, of employing five superstars who are already employed and highly remunerated, they were to hire two permanent staff, perhaps at Lecturer (Assistant Professor) or Senior Lecturer (Associate Professor) level. Let’s suppose that the two staff they hired had the same REF profile as the standard staff — so four 3* outputs and one 4* output.
- Then, in REF2014 and REF2021, the 11FTE and 21FTE fictional departments would both have had 75% of their outputs at 3* and 25% at 4*.
And of course both of these are significantly lower than their score with the fractional superstars in either REF2014 or REF2021.
Richard Pettigrew's homepage 