My new book explaining in detail how the FOI system in the UK works and how to use it successfully to get information from public authorities is now published – available from Amazon here or direct via Rhododendron Publishing here.
Freedom of Information: A practical guidebook Read More »
As Rachel Reeves ponders her forthcoming budget and how to balance raising money against economic growth, one of her self-imposed constraints is her pledge not to raise the rate of VAT. However the impact of taxes also depends greatly on the thresholds from which they apply, even though this tends to get a lot less attention in public debate (as is certainly the case for income tax).
So what about the annual turnover level at which businesses have to register for VAT?
Data I have recently obtained from HMRC under the freedom of information law shows the dramatic impact of the VAT threshold in restricting the growth of some of the UK’s small businesses.
In 2021/22 the UK had 21,752 businesses with annual turnover in the range £84,000-£85,000, just below the then threshold. But there were only 10,096 businesses just over the limit, in the range £85,000-£86,000.
In other words the number of businesses clustered just under the VAT threshold was more than double the number just above, as businesses curtail their activities to remain outside the VAT registration system.
The graph above clearly shows the cliff edge in the data.
Many small businesses are desperate to keep their annual turnover under the VAT level, so that they avoid the bureaucracy and costs of registration and they don’t have to charge VAT to customers, which would make them less competitive. However the consequence is that they then won’t grow further into larger, more successful operations.
For some businesses the VAT threshold functions as a ceiling constraining their growth.
Research by Warwick University in 2022 concluded that earlier data of this kind reflected genuine curtailment of business activity rather than false reporting to HMRC.
This is the latest data available from HMRC, which says that more recent information is still being processed. The current VAT threshold is now £90,000, as the figure was increased by the Conservative government before the general election.
The UK’s VAT threshold is high compared to other European countries which tend to impose VAT registration on businesses at a much lower level. While the UK policy saves many small businesspeople from the compliance burden of VAT, the significantly lower thresholds elsewhere make it less likely that enterprises will be found bunched and held back just under the relevant level of turnover.
I also wanted to get a breakdown of the data by sector of the economy, to see which kinds of businesses were most affected. HMRC said it could provide this for 2019/20, as it had previously extracted the information involved, but that more recent breakdowns would probably exceed the FOI cost limit.
According to these 2019/20 figures, the most dramatic effect is in the construction sector.
This data shows 4,445 construction businesses with an annual turnover of £84,000-£85,000, but only 1,425 in the range £85,000-£86,000. So the number of construction businesses appearing to have kept themselves just below the limit is over three times the number who grew a little more and just exceeded it.
The chart shows the impact for construction and some other economic sectors with large numbers of small enterprises.
These FOI releases from HMRC constitute the latest and most thorough official evidence of what the tax expert Dan Neidle of Tax Policy Associates has called ‘the VAT growth brake’.
The full HMRC spreadsheets can be downloaded here:
The VAT cliff edge: How the threshold impedes small businesses Read More »
Ever since the Freedom of Information Act came into force nearly 20 years ago, some unhappy public bodies have protested loudly about the resulting ‘administrative burden’. But what is less appreciated is how numerous authorities actually seem to find the law useful – to obtain information themselves.
A few weeks ago Redcar and Cleveland Council issued a statement asking that the new Labour government ‘significantly revises the scope of the act to reduce the burden on councils’.
The particular grievances itemised included the usual targets of FOI applications emanating from businesses and journalists, but the council also focused its ire on ones which came from ‘local/national government’.
Intrigued by this, I asked the council (under FOI, naturally) for details of recent requests from local and national government. And it turned out that there were more than I expected, covering a wide range of council responsibilities. See the list below.
In my view these are entirely reasonable and legitimate requests. If FOI enables councils to find comparable information from their counterparts, which assists with policy development, service provision, budgeting and external contracting, that is not a reason to curtail the law – it is another reason to stress how useful FOI is in contributing to the general public interest.
Incidentally, there weren’t actually any applications from national government departments, despite what Redcar and Cleveland Council claimed. There were however numerous requests from MPs and Peers (or their staff), which the council bizarrely lists as coming from ‘National Government Departments’, but that is obviously not the same thing.
Since January 2022, Redcar and Cleveland Council has received the following 29 FOI requests from public authorities (26 from other councils and three from national quangos):
From one council to another Read More »
Freedom of information requests can be rejected for a range of reasons, but some are much more likely to be overturned by the Information Commissioner’s Office than others.
The details of this are made clear by my analysis of a dataset recently released by the ICO covering nearly 22,000 decisions issued by the information rights regulator since FOI came into force.
For example, the ICO has upheld nearly half the complaints received from information requesters against FOI refusals linked to protecting commercial interests. But it has upheld only one in six objections to refusals based on international relations.
This table shows, for each of the legal grounds for dismissing FOI requests, the number of complaints about that reason which the ICO has ruled on and the percentage which it has upheld (ie backing the requester and overriding the public authority).
| Subject matter (section of FOI Act) | Number of complaints | Percentage upheld |
| The economy (29) | 27 | 56 |
| Relations within UK (28) | 17 | 53 |
| Commercial interests (43) | 1010 | 47 |
| Future publication or research (22/22A) | 213 | 44 |
| Health and safety (38) | 119 | 42 |
| Policy formation (35) | 622 | 38 |
| Already accessible (21) | 332 | 36 |
| Effective conduct of public affairs (36) | 967 | 35 |
| Audits (33) | 38 | 34 |
| Confidential information (41) | 605 | 34 |
| Law enforcement (31) | 860 | 30 |
| Vexatious or repeated (14) | 1498 | 23 |
| Investigations (30) | 318 | 21 |
| Personal data (40) | 3097 | 18 |
| Monarchy and honours (37) | 181 | 18 |
| Defence (26) | 41 | 17 |
| National security (24) | 299 | 17 |
| International relations (27) | 292 | 16 |
| Legal privilege (42) | 507 | 16 |
| Otherwise prohibited (44) | 406 | 14 |
| Cost (12) | 1491 | 12 |
| Court records (32) | 108 | 8 |
| Security bodies (23) | 304 | 7 |
| Parliamentary privilege (34) | 12 | 0 |
Or in chart form:
So during FOI’s two decades of operation, the ICO has been much happier to overrule public authorities on matters like commercial interests and policy formation than on topics like defence, security and international affairs.
My analysis uses three spreadsheets with details of ICO rulings which were recently disclosed via the What Do They Know website, in response to a request from Alison Benson. The spreadsheets list the ICO’s formal decision notices from the first one in 2005 until last month.
The ICO maintains that it provided this material voluntarily ‘on a discretionary basis’, arguing that the information would be already available through its routine publication of decision notices.
However the supply of these three files makes the statistical analysis of ICO rulings much more practical than by trying to process all the individually published decisions. The ICO’s release of this dataset is therefore a positive and welcome step in terms of its own transparency.
Environmental information
Note that my analysis excludes environmental information, which falls under a different law, the Environmental Information Regulations. The EIR exceptions do not exactly correspond to the FOI exemptions, so the data cannot be combined.
The numbers of EIR cases are fewer than for FOI, but a similar pattern emerges. Thus the ICO has more frequently overruled public authorities when they base an EIR refusal on commercial confidentiality or the internal nature of communications, rather than when authorities rely say on protecting the course of justice.
Delay
It is also possible to analyse aspects of the dataset in more detailed ways. Here is one example.
This table shows the 15 public authorities against whom the ICO has most often upheld complaints about delay in processing FOI requests (under section 10 of the FOI Act), and how many times this has happened since 2005.
| Public authority | Upheld complaints about FOI delay |
| Home Office | 303 |
| Ministry of Justice | 173 |
| NHS England | 162 |
| Cabinet Office | 161 |
| Dept of Health and Social Care | 84 |
| Metropolitan Police | 82 |
| Dept for Work and Pensions | 79 |
| Foreign Office | 74 |
| Sussex Police | 74 |
| BBC | 60 |
| Ministry of Defence | 58 |
| Dept for Education | 54 |
| Wirral Council | 43 |
| Croydon Council | 39 |
| Information Commissioner’s Office | 35 |
On this measure the public authorities with the biggest record of delay since FOI was implemented are the Home Office, the Ministry of Justice, NHS England and the Cabinet Office.
Ironically the authority which comes 15th on this list of shame is the ICO itself! This is clearly a very bad record for an organisation which should be setting a good example of prompt compliance with the law, but at least as a regulator it has been willing to point out its own failings.
Notes: 1) My analysis amalgamates bodies which at some point since 2005 had some change of name or scope but remained essentially the same organisation (eg NHS England with NHS Commissioning Board; Department of Health and Social Care with Department of Health). 2) The ICO is thoroughly and annoyingly inconsistent when naming authorities (eg sometimes using ‘Metropolitan Police Service’ and sometimes using ‘Commissioner of the Metropolitan Police Service’. I hope I have spotted all such instances and combined the figures accordingly, but it is possible I have missed some.
FOI: Which complaints are upheld by the ICO? Read More »
Which predictive model for the results of the election was best – or the least bad?
I say ‘least bad’, because in what may seem like the frequent tradition of the British polling industry, they all overstated how well Labour would do.
However there was also a huge gap between the least bad and the much worse. In a close election discrepancies of this extent would have pointed during the campaign to very different political situations, creating the impression that the forecasting models were contradictory chaos. This level of variation is somewhat disguised by the universal prediction of what could be called a ‘Labour landslide’, now confirmed as fact (even if it isn’t as big as they all said it was going to be).
Labour seats
Let’s look at the forecasts for the total number of Labour seats. This determines the size of Labour’s majority and is the most politically significant single measure of how the electorate voted.
| Actual result for Labour seats | 412 |
| Britain Predicts | 418 |
| More In Common | 430 |
| YouGov | 431 |
| Election Maps | 432 |
| Economist* | 433 |
| JL Partners | 442 |
| Focal Data | 444 |
| Financial Times | 447 |
| Electoral Calculus | 453 |
| Ipsos | 453 |
| We Think | 465 |
| Survation** | 470 |
| Savanta | 516 |
I have listed the models which predicted votes for each constituency in Great Britain and were included in the excellent aggregation site produced by Peter Inglesby. (If that means any model is missing which should have been added, my apologies.)
Note that what I am comparing here are the statistical models which aimed to forecast the voting pattern in each seat, not normal opinion polls which only provide national figures for vote share. These competing models are all based on different methodologies, the full details of which are not made public.
The large number of such models was a new feature of this election, linked to the growing adoption of MRP polling along with developments in the techniques and capacity of data science.
On this basis the winner would be the Britain Predicts model devised by Ben Walker and the New Statesman. Well done to them.
This model is not based on a single poll itself, but takes published polling data and mixes it into its analysis. This is also true of some of the others around the middle of the table, such as the Economist and the Financial Times.
On the other hand polling companies like YouGov and Survation base their constituency-level forecasts on their own MRP polls (Multilevel Regression and Post-stratification), combining large samples and statistical modelling to produce forecasts for each seat.
The closest MRP here is the More in Common one, with YouGov narrowly next. However the bottom of the table are also MRP polls rather than mixed models – We Think, Survation and Savanta. (It should be noted that the Savanta one was conducted in the middle of the campaign and so was more vulnerable to late swing).
Constituency predictions
However a different winner emerges from a more detailed examination of the constituency level results. This is based on my analysis using the data aggregated on Peter Inglesby’s website.
Although Britain Predicts was closest for the overall picture, it got 80 individual seats wrong in terms of the winning party. This was often in opposite directions, so at the net level they cancelled each other out. It predicted Labour would win 33 seats that they lost, while also predicting they would lose 26 seats which the party actually won.
In contrast YouGov got the fewest seats with the wrong party winning, just 58. So well done to them. And I’m actually being a bit harsh to YouGov here, as this is counting the 10 seats they predicted as a ‘tie’ as all wrong – on the basis that (a) the outcome wasn’t a tie (haha), and (b) companies shouldn’t get ranked with a better performance via ambiguous forecasts which their competitors avoid. If you do not agree with that, which might be the more measured approach, you can score them at 53.
The two models that did next best at the constituency level were Elections Maps (62 wrong) and the Economist (76 wrong). The worst-scoring models were We Think and Savanta which both got 134 seats wrong.
This table shows the number of constituencies where the model wrongly predicted the winning party.
| Model | Errors at seat level |
| YouGov | 53 |
| Election Maps | 62 |
| Economist | 76 |
| Britain Predicts | 80 |
| Focal Data | 80 |
| More in Common | 83 |
| JL Partners | 91 |
| Electoral Calculus | 93 |
| Financial Times | 93 |
| Ipsos | 93 |
| Survation | 100 |
| Savanta | 134 |
| We Think | 134 |
(I’m here adopting the slightly kinder option for YouGov in the table).
This constituency-level analysis also sheds light on the nature of the forecasting mistakes.
There were some common issues. Generally the models failed to predict the success of the independent candidates who appealed largely to Muslim voters and either won or significantly affected the result. On the one hand it is difficult for nationally structured models to pick up on anomalous constituencies. On the other it is possible that the models typically do not give enough weight to religion (as opposed to ethnicity).
On this point there’s increasing evidence of growing differences in voting patterns between Muslim and Hindu communities. It’s striking that 12 of the 13 models (all except YouGov) wrongly forecast that the Tories would lose Harrow East, a seat with a large Hindu population where the party bucked the trend and actually increased its majority.
The models also failed almost universally to predict quite how badly the SNP would do – ironically with the exception of Savanta, the least accurate model overall.
On the other hand there were also wide variations between the models in terms of where they made mistakes. In all there were 245 seats – 39% of the total – where at least one model forecast the wrong winning party.
The seats that most confused the modellers are as follows.
Seats where all the 13 modellers predicted the wrong winning party: Birmingham Perry Barr, Blackburn, Chingford and Woodford Green, Dewsbury and Batley, Fylde, Harwich and North Essex, Keighley and Ilkley, Leicester East, Leicester South, Staffordshire Moorlands, Stockton West, plus the final seat to declare: Inverness, Skye and West Ross-shire***.
Seats where 12 of the 13 modellers predicted the wrong winning party: Beverley and Holderness, Godalming and Ash, Harrow East, Isle of Wight East, Mid Bedfordshire, North East Hampshire, South Basildon and East Thurrock, The Wrekin.
Overall seats v individual constituency forecasts
So which is more important – to get closest to the overall national picture, or to get most individual seats right?
The statistical modelling processes involved are inherently probabilistic, and it’s assumed they will make some errors on individual seats that will cancel each other out. That’s the case for saying Britain Predicts is the winner.
But if you want confidence that the modelling process is working comparatively accurately, that would point towards getting the most individual seats right – and YouGov.
Note that this analysis is based just on the identity of the winning party in each seat. Comparing the actual against forecast vote shares in each constituency could give a different picture. I haven’t had the time to do that more detailed work yet.
Traditional polling v predictive models
The traditional (non-MRP) polls also substantially overstated the Labour vote share, as the MRP ones did, raising further awkward questions for the polling industry. However, there’s an interesting difference between the potential impact of the traditional polls compared to the predictive models which proliferated at this election.
Without these models, the normal general assumption for translating vote shares into seats would have been uniform national swing. (This would have been in line with the historical norm that turned out to be inapplicable to this election, where Labour and the LibDems benefitted greatly from differential swing patterns across the country.) And seat forecasts reliant on that old standard assumption would then have involved nothing like the massive Labour majorities suggested by the models.
Although the predictive modelling in 2024 universally overstated Labour’s position, it did locate us in broadly the correct political terrain – ‘Labour landslide’. We wouldn’t have been expecting that kind of outcome if we’d only had the traditional polling (even with the way it exaggerated the Labour share).
To that extent the result was some kind of vindication for predictive modelling and its seat-based approach in general, despite the substantial errors. The MRP polls and the models that reflected them succeeded in detecting some crucial differential swings in social/geographic/political segments of the population (while also exaggerating their implications).
However, it’s also possible that the models/polls could in a way have been self-negating predictions. By forecasting such a large Labour victory and huge disaster for the Tories, they could have depressed turnout amongst less committed Labour supporters who then decided not to bother going to the polling station, and/or they could have nudged people over into voting LibDem, Green or independent (or indeed Reform) who were until the end of the campaign intending to back Labour.
Notes
*Note on Economist prediction: Their website gives 427 as a median prediction for Labour seats, but their median predictions for all parties sum up to well short of the total number of GB seats. In my view that would not make a fair comparison. Instead I have used the figure in Peter Inglesby’s summary table, which I assume derives from adding up the individual constituency predictions.
**UPDATE 1: Note on Survation prediction: After initially publishing this piece I was informed that Survation released a very late update to their forecast which cut their prediction for Labour seats from 484 to 470. The initial version of my table used the 484 figure, which I have now replaced with 470. However, despite reducing the extent of their error, this does not affect their position in the table as second last.
Other notes: (1) I haven’t been able to personally check the accuracy of Peter Inglesby’s data, for reasons of time, but I have no reason to doubt it. I should add that I am very grateful to him for his work in bringing all the modelling forecasts together in one place. (2) This article doesn’t take account of the outcome in Inverness, Skye and West Ross-shire, which at the time of writing was yet to declare.
***UPDATE 2: The eventual LibDem victory in Inverness, Skye and West Ross-shire was not predicted by any model, which all forecast the SNP would win. This means that this has to be added to my initial list of those which all the models got wrong, which therefore now totals 12 constituencies.
Election prediction models: how they fared Read More »
Suppose you are the lucky owner of a very valuable object which is ‘pre-eminent’ for its historic or artistic interest.
When you die, that might result in a substantial inheritance tax payment. Except that this can be completely avoided, if HMRC agrees that the item constitutes a national heritage asset, and the inheritor is willing to let the British public come and look at it.
And if you are not already the owner of such an artefact, but you can afford it, you could buy one – as a handy method of reducing the tax liability of your estate. Naturally there are legal and financial advisors who will help you do this.
The objects exempted from tax under this law range from a Rembrandt self-portrait to a ‘pair of Chelsea Derby candlestick figures, each of a scantily draped winged cupid kneeling with arm around a floral encrusted bough, rococo scroll base with gilt enrichment, 6 3/4in. high (both with sconces missing, some damages)’.
A full list is published by the government, in a database currently containing over 36,000 entries. Some are on public display – HMRC has told me that about 8,000 are on loan to museums or galleries. But for the others, which is the large majority, how often does anybody actually make use of their legal right to go and see them?
No overall statistics are available to answer this question. However, according to information I have just obtained from HMRC under FOI, there were just 5,521 searches of the database in the last financial year (2023/24).
Obviously the number of actual visits will doubtless be much fewer than the number of database searches, many of which will not lead to any further action. Even though it is possible visitors might see more than one item at a time, it seems very likely indeed that most of these ‘national assets’ – saved for the nation at public expense – are never appreciated by any member of the general public, and certainly not by significant numbers of them.
You can find out what is available, and when and how it can be viewed, by searching the database. In many cases public access must be allowed without a prior appointment for at least a few days each year. Outside these open days, an appointment may be required.
That’s the theory. The practice might be somewhat trickier. As the tax consultancy Ross Martin states: “It seems that few people try and see some of the objects. On a practical level, it is very difficult to gain access to some of the assets. Access in most cases is handled by private client law firms and the links given to open days can be uninformative. Be prepared to be ruthlessly persistent if you wish to see an object or a collection.”
HMRC also informed me that ‘we do from time to time get contacted by members of the public directly to make us aware of any access issues that they have experienced’. But it does not have a central record of receiving any formal complaints about this.
In the 1990s the campaigning comedian Mark Thomas organised coachloads of visitors to attempt to see various artworks involved for a television programme. The law has since been changed, and access should have become easier.
HMRC estimates that this inheritance tax exemption/loophole (according to your personal preference), together with a similar rule for land and buildings, reduces government tax receipts by about £60 million annually.
Like many laws the Freedom of Information Act has apparent anomalies, which may or may not have been intentional.
It seems very odd, for example, that the FOI process doesn’t let you find out about complaints and other issues which council trading standards departments are pursuing with businesses. I’d expect even people who don’t much like FOI to think that kind of consumer protection information should be publicly available.
But it isn’t, because the Enterprise Act 2002 stops councils from releasing it. After some early legal disputation it was ruled that this legislation trumps the disclosure requirements of the FOI Act. To illustrate, here’s an ICO decision notice about a case relating to a window installation company.
Another anomaly is that obtaining environmental information is not covered by the FOI law, but by a separate set of rules, the Environmental Information Regulations. These are similar to the FOI regime, but not identical, and in my opinion both public authorities and people requesting information are not sufficiently alert to the differences.
These two anomalies are connected, in that I have recently successfully argued that while the Enterprise Act can block the disclosure of material under FOI, it can’t be used to prevent the release of environmental information. The EIR do not allow the legal basis for that kind of refusal.
So Hertfordshire Council have now been forced to send me copies of trading standards emails sent to Tesco about price displays under the planned Scottish deposit return scheme for single-use drinks containers.
(Businesses which operate across multiple locations can deal with just one council as the ‘primary authority’ for trading standards purposes. Tesco’s primary authority is Hertfordshire, where its corporate head office is based. This arrangement extends to Scotland, as – unlike the deposit scheme itself – consumer protection is not a devolved policy area.)
Over several months Hertfordshire Council went through a number of different and implausible arguments while it tried to resist giving me this documentation. It first proclaimed that due to the Enterprise Act disclosure would prejudice the administration of justice; it then moved to saying it would damage the interests of the information provider (ie Tesco); it finally decided to assert that a deposit return scheme for bottles and cans was nothing to do with the environmental issue of recycling – an argument dismissed by the Information Commissioner’s Office, which ruled in my favour.
The emails I have received show that in 2022 the council was telling Tesco that shop price labels would have to state the full price for the relevant bottled and canned products including the deposit, not a price separately without the deposit.
However the implementation of the Scottish scheme (which was beset by controversies) has since been postponed, so this is no longer a pressing concern. As matters now stand, the UK, Scottish and Welsh governments are pledged to introduce a UK-wide deposit return scheme in October 2025. If this goes ahead then the issue of how prices are displayed in order to be fair to consumers will doubtless be widely raised.
Further reading: I give a detailed account of the numerous significant differences between FOI and EIR, and how they affect the process of obtaining information, in my book.
Anomalies detected, but every little helps … Read More »
My attempt to find out what the House of Lords Appointments Commission had to say (if anything) about the award of peerages to Charlotte Owen and Ross Kempsell by Boris Johnson has just been rejected by the Information Commissioner’s Office.
I will now be appealing this to the First-tier Tribunal, on the grounds that in my opinion it is in the public interest for this material to be revealed, despite the view of the ICO.
Last July I made a freedom of information request to HOLAC for the material it held about the two individuals we now know as Lady Owen of Alderley Edge and Lord Kempsell, after their somewhat unexpected appointment to the House of Lords in Johnson’s resignation honours list.
After HOLAC declined to send me anything, I complained to the ICO. My arguments can be summarised as follows:
HOLAC argued:
The ICO has upheld HOLAC’s stance. We will now find out what the First-tier Tribunal makes of the rival arguments. It is likely to take several months before the Tribunal decides the case.
I was interested to see last month that the UK Governance Project, a high-powered independent commission with a distinguished membership, drew attention to the problem of lack of transparency at HOLAC. It recommended that HOLAC should always have to publish a citation setting out the basis on which it has approved an individual for appointment.
Charlotte Owen, Ross Kempsell and the secrecy of HOLAC Read More »
Pupils are over 20 per cent more likely to be absent from school on Fridays compared to Wednesdays.
The average rate of absence last term in England’s state-funded schools was 7.5% on Fridays. This compares to 6.7% on Mondays, the next most common day for school absence, and the lower figures for the middle of the week: 6.3% for Tuesdays, 6.2% for Wednesdays and 6.4% for Thursdays.
I have derived these figures by analysing the detailed school attendance data collected and published by the Department for Education.
The issue of school attendance is moving up the political agenda, as levels of absence are now much higher than before the covid pandemic.
The government has today announced what it calls ‘a major national drive to improve school attendance’, with measures targeted at tackling persistent absence. The Labour party is also focusing on the issue this week.
This weekly pattern of absence being highest on Fridays, and second-highest on Mondays, with better attendance mid-week, is a widespread feature of the current school system.
From my analysis of the DfE’s data, it applies in both primary and secondary schools, and also in all regions of England.
It is seen when looking both at authorised and unauthorised absences from school. This includes applying to absence due to illness, which is the most common reason recorded for pupils not attending school.
It was also evident throughout the autumn term, as can be seen in this chart (with a particular peak on the Friday before half-term).
The DfE’s data on school attendance can be downloaded here.
In a previous post I examined how school attendance can be affected by when in the year pupils are born.
For school pupils, does when in the year they are born affect how often they are absent from school?
My analysis of government data suggests that secondary school pupils born in September to December have a somewhat higher absence rate than those born in May to August – which is actually the opposite of what I expected.
Absence from school is now significantly higher compared to before the covid-19 pandemic, and tackling this has been made a target of government educational policy.
Since 2022 the Department for Education (DfE) has been collecting centrally some remarkably detailed and up-to-date data on attendance records for individual pupils from many schools in England, and publishing regular summaries.
The data collated by the department makes it possible to quickly analyse a wide range of factors and potential connections with absences.
Since month of birth is definitely related to other aspects of school life, such as how well pupils do in exams and in sport – the so-called ‘relative age effect‘ – I decided to explore any link with school attendance. Through a freedom of information request I obtained pupil attendance data from the DfE for the school year 2022/23, broken down by type of school, school year and month of birth.
This table shows the percentage of school sessions missed by pupils in selected year groups. It shows that for pupils in years 1 and 2 (aged 5/6 and 6/7), it was the summer-born pupils who had higher rates of absence. This was what I expected, given the well-documented school problems often faced by summer-born children.
But for pupils in years 8 to 11 (aged 12/13 to 15/16), it was those born in September to December who were more likely to be absent.
However the differences within the year groups are not massive, so this pattern (while clear) shouldn’t be overstated. For the intervening ages the data showed very little variation within each year group, so I haven’t presented the figures here. I haven’t obtained data for the reception year.
All this data relates to pupils at about 85% of state-funded schools in England, those which take part in the DfE scheme for automatically submitting daily attendance information.
The following graph shows the same data presented in the form of a line chart.
Persistent absence is a particular problem. This is defined as when pupils are absent for over 10% of school sessions. Analysing the data on persistent absence discloses a similar pattern.
This is indicated in the table below (which involves data from primary and secondary schools, but not special schools).
Generally rates of absence increase as pupils get older and move into higher year groups. Perhaps this trend could help to explain the fact that in secondary schools it’s the older pupils within the year group who tend to be absent more often.
But this can’t be a complete explanation – for example, the frequency of persistent absence is higher for year 10 September births (32.4%) than for the older pupils born in August and in year 11 (30.7%), and similarly for various other data points.
So it looks like there may be some kind of relative age effect involved here, if probably quite mild.
Bear in mind that this is just one year’s data, the period in the wake of the pandemic could be atypical, and there is also the possibility of random variation.
As another potential factor, some illnesses have been associated with when people are born within the year. However, this would not explain the jumps in this data between August and September births.
The DfE data distinguishes authorised and unauthorised absences, but this does not help much in explaining the pattern identified here.
It’s important to note that there are other characteristics which clearly have a bigger impact on school attendance, including levels of disadvantage (poorer pupils are more likely to be absent) and ethnicity (Caribbean and White ethnic groups have higher absence rates than Indian, African and Chinese groups).
The data spreadsheet supplied to me under FOI by the Department for Education is here.
For background on the government’s impressive automated collection of real-time school attendance data, you can watch a recent talk by Caroline Kempner, the DfE’s head of data transformation, given at one of the regular Institute for Government ‘Data Bites’ events (from 37’25” in the video).
It was hearing this presentation which prompted me to do this analysis.
Absence from school and month of birth Read More »
This article was originally published in the following journal: Freedom of Information, Volume 20, Issue 1 (published by PDP Journals)
The requester and the requested Read More »