Of car crashes and performance!
Car crashes usually are a matter of seconds in duration except when they involve politics, writes Jon Duveen. The School and College Examinations 2020 car crash has evolved over almost 5 months and has yet to be completed.
With the lockdown and the closure of schools across the United Kingdom on Monday 23 March, students were left with unfinished courses and no clear idea of what would happen to their results. Education is a devolved matter so resolving the issue was left for each of the four governments for England, Wales, Scotland and the six counties of ‘Northern’ Ireland. However, in the case of A levels and equivalent qualifications taken by 18 year olds in schools with sixth forms and in local colleges, the examination system is run across three of the four jurisdictions, with Scotland having its centuries’ old separate system (where students finish at 17 and University degrees are four years long).
The governments seemed to endorse the concept of using teacher assessments to assess the students in these courses. For many teachers, parents and students this seemed a reasonable and fair procedure.
However, when, after Easter, the details of the assessment model being used in England began to emerge more clearly, many teachers began to have second thoughts. They were being asked not only to make a prediction of what grade each student would achieve but to also rank students in each grade level for each course. Many teachers began to feel worried by this. Why rank the students if the teachers had been asked to give a predicted grade? Were the predicted grades going to be accepted? How would the ranking of the students be used?
With the exposure in the newspapers around Saturday 8 August that students would mainly be graded by a statistical analysis, many teachers fears became reality. This analysis would be based on the performance of schools (or colleges) over the past years. All teachers know that that each year cohort of students is different in many ways from all other previous cohorts. To ignore these differences, or try and assign some numerical or algebraic value to these differences without explaining to schools or students the assumptions behind these calculations will only make both the students and schools feel cheated.
When the results for the Scottish school qualifications (Nationals, Highers and Advanced Highers) were announced on Tuesday 4 August about 25% of the teacher assessments were marked down through a statistical process called ‘moderation’. There was a huge outcry from the students, parents and teachers. Particularly challenging for the SNP government was the revelation from the Government’s Scottish Qualifications Authority (SQA) that more results from schools in the most deprived areas had been marked down than those from schools in more affluent areas.
While the SNP government initially tried to hold the line, the impact of demonstrations by school students and the tabling of a motion of no confidence in the Education Secretary began to have an impact. The SNP government was forced by popular protest to make a major U-turn within a week and agreed to use the teacher assessment as the grade for the examinations and also to set up an independent review into the handling of this year’s school results. This satisfied the Scottish Green Party which holds the balance of power in the Scottish Parliament and so the Education Secretary’s job was saved.
In England, however, we were being continually ‘reassured’ by Gavin Williamson, the Tory Secretary of State for Education, and Boris Johnson, that the system in use across England for A level examinations was robust and accurate. But many statisticians were questioning the methodology being used. How accurate could the analysis be when for most schools the number being examined in any subject at A-level were low, typically 10 to 15 in a class?
On Thursday 13 August the A-level results in England were released, with students finding that 40% of the results had been downgraded from the grades submitted by their teachers. Clearly something had gone wrong but for the UK Tory government this was just something the students would have to swallow. After all we couldn’t have pupils being ‘over-promoted into jobs that are beyond their competence’ to quote Gavin Williamson.
For the A level examination boards operating in England, which are all privately owned companies, teacher assessment was almost an irrelevance. What they wanted was the ranking of the students for each course in each school. The boards used a three stage process to arrive at the grades.
First, they examined the spread of the results for each course in each school over the past few years which gave them a spread of grades to be used for these examinations. Next they looked at the ranking produced by the teachers in each school for each course and applied that ranking to the spread produced in the first stage. Only in the final stage did they use the teacher assessments to ensure that they had not made any ranking errors. So effectively the teacher assessments played no real role in assessing the grade given to a student.
For the privately owned exam boards and for Ofqual, the state regulatory body for the school assessments in England, it was more important to ensure that this year’s results were not out of line with previous year’s results than it was to recognise that we were in a unique situation that required different solutions.
The outcome of this whole car crash is that students feel cheated of the results they should have had, and all the effects this will have on university entry and their future careers, and teachers feel conned by the government.
The origins of this car crash go back many years to when Michael Gove, then Secretary of State for Education, reversed previous education policies and decided that all GCSEs and A-levels, in England, should mainly be assessed by a final exam and there should be little or no role for course work in the assessment. As a result the use of AS levels as a stepping stone after one year’s study towards the full A Levels, a reform introduced by the Labour government, was abolished. All A-level courses were to be for the full two years’ duration. However, assessments for BTEC vocational and technical subject were to be left alone even though they had a considerable coursework component to them, in effect confirming their second class status in the eyes of the Tories.
The outcome of this is that when the schools exams were cancelled, schools and colleges in England had no standardised assessment marks to show the exam boards how well the students had performed. This also had the effect of removing teacher’s direct influence on the examinations, another step in the UK Tory government’s attack on the professionalism of teachers.
Another issue of concern is what the Department for Education in England has been doing for the past five months. It cancelled the exams and then seemingly ignored the problem of assessment until just before the results for A-levels were announced. Any competent Minister would have made sure that the system for deciding on the grades was fair and transparent to all those concerned.
But the opposite has happened. There is clear evidence that there is a bias in the results against those students in working class communities. Why wasn’t this picked up before the results were announced and corrected? Why was no formal “Equality Impact Assessment” carried out on the data before its release? Since there was no actual marking of scripts for these exams why did it take so long to get the results out to students?
The next stage in this sorry saga will come next Thursday, 20 August, when the GCSE results for 16 year olds are released and, again, about 40% of the students are expected to be marked down in their grades compared to the teacher’s assessments.
So what can we learn from this car crash?
Do we need the current system of school assessments?
This year has seen the cancellation not only of the GCSE and A-level exams but also of the Standardised Assessment Tests (SATs) taken at ages 7 and 11 and of baseline assessments. One effect of this has been to reduce the pressure on the children and students and this alone should be grounds for a rethink on school assessment.
Any assessment system should be based on assessing what children and students can do and what knowledge they have not on the basis of what they cannot do. Any system also a needs to be based on making sure a child’s needs are met when they are identified. These are not the principles on which the present system is based. We need a new system that parents, education workers, trade unions and students are actively involved in designing.
Abolish private schools
Analysis of this year’s results showed that students in small groups, 5 or less, in less popular subjects, e.g. German or music, obtained results that were closer to those submitted by their teachers. It is no secret in which sector the majority of small classes are to be found, and it is in private schools, not the state sector.
About 20% of A-level entries from private schools are in subjects where the school entered 5 or less students. The corresponding data for state schools was 5%. Ofqual’s data also showed that 49% of students entered through private schools in England received a grade A or above. The corresponding data for state schools was 20%.
Clearly the statistical analysis and algorithms of Ofqual and the exam boards have a bias against the state sector in education. With all this data, we have to have to ask why the Labour Party is not calling for the abolition of the public schools or at least their nationalisation and integration into the state sector of education?
Students in England should be allowed to appeal their grades?
Gavin Williamson, the Secretary of State for Education, announced this morning, Saturday 15 August, a partial climb-down over the appeals’ process. His concession was to agree that the Government would pay for any appeals. However, this does not deal with the central issue of ignoring the grades submitted by teachers in this unique situation. Nor does it address the grounds for the appeal. Appeals still have to be submitted by the school. Why cannot students appeal directly to the exam boards over their results? Do we really think that 18 year old students are not capable of putting forward a clear argument as to why their grade is incorrect? If that is really the case what have we been doing in education for the time that these students have been in the system?
University and Higher Education entry should be based on ability to benefit, not elite selection
A level grades are important because they are the primary selection method for university entry. The UK Tory government has constantly pressed for a small group of universities to become increasingly selective and follow the elitist practices of Oxford and Cambridge Universities that most of the Tory cabinet came through. This is surrounded in the language of international ‘excellence’ through stratification, rather than the equality and mass participation in higher education needed for a better society.
In practice due to the correlation between highest qualifications achieved and home life due to parental income and prior education, the university selection system discriminates against working class people. The Tories increasingly push for more selectivity and have opposed the recent practice of some universities seeking more progressive intakes in making “unconditional” entry offers based on predicted grades by school teachers.
In effect the Tories have now banned universities doing this in England, despite the supposed “arms’ length” of University admissions from state control. We need a university system that does not admit on pure grading alone, but on a range of wider factors, including ability to benefit by people from a working class or deprived background. This also means challenging the use of tuition fees of £9,250 per year and student loans and returning to a system of free tuition and grants for the most needy students whereever they choose to study.
What should the government do about the results in England?
With 40% of the student’s results being downgraded, there is clearly a need for the UK government to act and the Scottish option would be the best model. But the Tory’s actions so far have been minimal. A chance to use their mock results is really no option for the students. Mock exams are not standardised in any form and many schools have their mocks in December or even in late March and so will not reflect the ability of the students. The only action that makes sense in the present crisis is to recognise in England as in Scotland that we are in exceptional circumstances and use the teacher assessments as the only criteria for awarding grades in this year’s exams. All other courses of action will penalise a large number of students and have serious repercussions on their career paths. This may be a step too far for a government that has continually denigrated teachers but is the only course of action that parents and students will see as fair.
Education not safe in Tory hands
Just looking at the current crisis in examination results should be enough to say that the Tories are not putting the interests of the students and children first. Why then aren’t Keir Starmer and the Labour Party calling a vote of no confidence in the Secretary of State for Education or even the Government? They moved a vote of no confidence in the Scottish Education Secretary even though he actually changed the policy and implemented the Labour Party’s (and Green Party’s) proposals. Why isn’t the Labour Party leadership making Tory education policy an issue in the local and mayoral elections scheduled in England for next May?
My own trade union the National Education Union (NEU), as the main teachers union in England, has had considerable success in making education an issue in the last two general elections and will I’m sure continue to do so in these upcoming elections. The University and College Union (UCU), which represents college teachers in England, have also campaigned hard on college and university education, as has the Educational Institute of Scotland (EIS) north of the border. United and coordinated action by education unions is key to winning these issues.
When the crisis of school and college funding, of university funding, of student loans, of the state of many school and college buildings, of asbestos in the buildings, of the marginalisation of elected local authorities, of Ofsted, of the funding of special needs, etc, is added to the car crash of the examination results, there are enough reasons to say that the education system is not safe in the Tories hands. Parents, students and education workers now need to fight to gain control of the education system.
There are two issues here:
1) What to do about this year’s exams.
2) What the problems encountered say about the education system in Britain,
On the second point, I agree with much of what John says: the system is designed to favour those who come from richer and educationally advantaged backgrounds and not for those who would benefit more from (higher) education. This is not the immediate question, so I won’t comment further on that.
Once you’ve accepted that the system (exams, league tables, private schools, pressure on teachers from their management for results, racism and class division in schools) still exists and this year’s cohort needs to have exam grades, then it is a major challenge to do that fairly. Just as an example, the article above says that Ofqual’s system has favoured private schools, because they have many more pupils in cohorts of 15 or under. It then says that Ofqual should just have used teacher assessments when that is actually what they did (in modified form) for these small cohorts.
What the private school results show is that teacher predicted grades are not that reliable, hardly surprising when the reputation and income of the school and the teacher’s reputation in some measure rests on them.
It is known that over the whole cohort applying for university, predicted grades have historically been overestimated in 40% of cases (affecting 76% of students). This is one of the reasons that the University and Colleges Union has suggested that school and university terms be altered so that offers of places can be based on actual results, not predicted grades.
In my opinion, the racism and class bias in exam results and predicted grades is introduced in schools and over-reliance on them for this year’s results would probably have led to a worse situation than exists now. That is not to say that the Ofqual system for larger cohorts is perfect by any means – especially the rigidity with which it is applied at grade boundaries.
The link below is to a report showing historical data about predicted grades – 76% of applicants had 1 or more grade over-predicted and 8.5% had them under-predicted: 16% were spot on. Note that this is applicants, not exams, in which about 40% were over-predicted, coincidentally (?) the figure that came out of the Ofqual results. The UCU research also shows that disadvantaged pupils were more likely to have over-predicted grades, except if they were of high ability, when their grades were more likely to be under-predicted.
Finally, I think the article misses the fact that Ofqual did use individual pupils’ past (GCSE) results in its calculation. This, of course, has its own problems. In some subjects, there is a massive difference in intellectual challenge between GCSE and A level – where in the former learning by rote can get you a good grade. On the other hand, GCSEs occur in the middle of adolescence, with huge the differences in the levels of maturity and emotional or mental stability at that age, so poor performance then is not necessarily a good predictor of later attainment.
What Ofqual did (so far):
Ofqual technical report (section 8 gives the details of their approach):
Phil confuses the teacher predicted grades that are annually put into UCAS applications in time for the 15 January deadline – and therefore usually written before Xmas with less knowledge of progress/achievement, with the internal grading process and predictions requested by the exam boards after the cancellation of the exams on 18 March. In fact many Schools and all colleges are used to having a rigorous system of internal moderation of final assessments for qualifications like BTEC, that in this instance parallel the request for A levels, but with the noticeable exception of the request for ranking that Jon draws attention to in this flawed exercise. The Royal Statistical Society drew attention to the different types of predictions in their seemingly ignored letter to Ofqual as early as 9th April. Jon also draws out the fact that since there was no actual marking of exam papers being undertaken by the Boards, the results and any moderation algorithm could have been published much earlier in time for results to be reviewed/appealed before University clearing began – whereas in fact the exam boards and Ofqual waited until the last minute of the traditional results publication day before publishing a 300 page technical report alongside ‘definitive’ results.