My new book explaining in detail how the FOI system in the UK works and how to use it successfully to get information from public authorities is now published – available from Amazon here or direct via Rhododendron Publishing here.
Freedom of Information: A practical guidebook Read More »
An edited version of a presentation I gave last month for staff at the Information Commissioner’s Office about my perspective on FOI and the ICO. Many thanks to the ICO for asking me to address them.
Thank you very much for inviting me. It’s a pleasure to have this opportunity to share my thoughts on freedom of information from the user’s perspective.
That perspective goes back about 20 years or so, because I’ve been making FOI requests since the law came into force in 2005, primarily as a journalist. Before that I made requests under the previous system, the Code of Practice on Access to Government Information, a much weaker system than the FOI law we have today.
During my time at the BBC, I specialised in using FOI, both submitting requests myself and advising and training other journalists on how to use the law effectively. I left the BBC around four years ago, and since then I’ve written a practical guidebook for FOI requesters.
So over the past two decades I’ve seen a lot of FOI in practice, and when people ask me for an overall verdict on FOI – is it working well, or not? – I have sometimes described it as a ‘tale of two giraffes’.
Of the two giraffes photographed in this slide, one is in London Zoo, and the other is in Whipsnade Zoo in Central Bedfordshire. Three years ago I was doing some research into zoos, and I asked for the latest inspection reports on these two zoos, which should be a very simple FOI request.
Central Bedfordshire Council sent me the latest inspection report on Whipsnade within an hour. It was absolutely the way FOI should work.
Westminster Council, the council responsible for London Zoo in Regent’s Park, didn’t reply. After 20 working days. I sent them a chasing email. They didn’t reply to that. I had to complain to the ICO, and the ICO – thank you very much – did the usual thing of sending an email saying you must reply within 10 working days, and eventually five months after my request I got the report from Westminster.
So what one public authority had been able to do within less than an hour, it took another public authority over five months and the intervention of the regulator.
In other words, sometimes FOI works very well in practice, sometimes it works very badly. And obviously there’s also a lot of circumstances in between those two extremes. But the key point is that the experience of FOI as a requester is that it’s very varied. There’s both a positive side and a negative side.
The positive
Taking the positive side first, and looking back over the last 20 years, one of the very good things is the way that FOI has become well established as a tool that a lot of people use successfully.
This is a chart which I’ve compiled from government statistics. I don’t think anybody else, by the way, is analysing these annual statistics in the same way that I do, keeping track of them over every year going back to 2005. This shows how in terms of central government departments, for the first few years the number of requests increased quite steadily until about 2013. Since then the numbers have been fairly steady over the past 10 to 12 years or so, maybe 35,000 FOI requests annually to government departments.
As you know we’ve only got statistics for central government bodies. So this is only a very small sample of FOI requests. Yet even so it’s one way of showing that FOI has become well established.
However, that fact obviously hasn’t gone down well with some people. For example, Tony Blair, who said that introducing FOI was like giving people a mallet to hit him over the head with, and that it was one of his two biggest mistakes. He famously wrote these words in his memoirs about the introduction of FOI by his own government: “Freedom of Information. Three harmless words … You idiot. You naïve, foolish, irresponsible nincompoop … I quake at the imbecility of it.”
He’s not the only prime minister who disliked FOI. David Cameron complained about it. Theresa May was more measured in her memoirs, but also had criticisms. And Liz Truss said in her memoirs: “Constant questions about who you’ve met and why … innocent meetings are privy to the press and often deliberately misconstrued.” She does, however, from her point of view, get the chance to blame Tony Blair for it.
What I think about these remarks is that actually they show in a way that FOI is working, at least in some cases. FOI should sometimes be uncomfortable to people who hold positions of power. That’s part of the point of it. If people in power were always very happy about the way that FOI worked, then it wouldn’t really be doing its job. So these quotes from former prime ministers don’t put me off FOI, to my mind they strengthen the case for it.
It’s really quite striking how FOI has reached the level of use and acceptance where it would be pretty politically impossible for a government to seek to weaken the law significantly.
We can see what happened when sometimes they have tried to take certain steps in that direction. For example, the Cameron government set up a review, a Commission on Freedom of Information, which included people like Jack Straw and Michael Howard, former politicians who were thought to be very sceptical about FOI. I think the government at the time was probably hoping that this review would report and pave the way for some restrictions on FOI to be introduced.
But actually when the review looked at how FOI was working in practice (and I’ve got to say full credit to people like Jack Straw, who I know from my own conversations with him had had doubts about FOI), it concluded that the law was generally working well, that it enhanced openness and transparency, and there was no evidence that it needed to be altered. It came down against the suggestion that was around some time ago of introducing charges for FOI requests. In fact, the review even said that in some cases the right of access should be increased.
FOI has become well established, it’s difficult to change in principle, and it works well in practice for many people.
So what is the point of FOI? It’s not to hit Tony Blair on the head with a mallet. As far as I’m concerned, primarily as a journalistic user, its purposes are to find out the truth, to hold power to account, to expose wrongdoing, and to give the public useful information.
There are journalists who’ve made a lot of use of FOI, and others who do so more occasionally, who are trying to achieve these kinds of aims. If we look at a recent list from a website that aggregates news stories, which I searched for freedom of information stories, the selection here is about London’s dirtiest streets, Thames Water (that must be an environmental information request), banks and fraud reports, stories about universities, others about councils, a wide range of themes.
When I was at the BBC we did go to some lengths at some times to collate together on the website a lot of FOI stories that had been done across the organisation, and again what this illustrates is the range of different issues covered, a lot of them very local – health stories, stories about a particular NHS trust, often very much of use to people in the local community, police stories, home affairs, politics stories about what councils are doing, transport stories. One could go on, but it shows the tremendous wealth of topics that FOI enables journalists to find information about in the public interest.
So to give some context about journalistic use of FOI, what do journalists get from FOI requests?
First of all, FOI might directly get the news story – the FOI disclosure in itself is the story, the headline.
But that’s not always the case. Sometimes for us FOI is a piece in a jigsaw, it’s an element in the story. That’s sometimes because you have a case study and FOI gives you overall statistics or context. Sometimes it’s the reverse, you might have some overall context already, but with the use of FOI you can get information about particular examples that help to fill out the story.
Sometimes FOI obtains background information. It doesn’t end up getting reported directly, but it points you in certain directions. It enables you to decide what you’re going to research, it informs questions to put to people in interviews, it helps with how to find out stuff from contacts, and so on.
And of course sometimes you get nothing useful whatsoever. All of us who’ve put in a number of FOI requests have sometimes found that the outcome was nothing of any use.
To give some examples of all of this, here is a story I did some time ago when I was at the BBC about Prince Charles, as he was then, urging Tony Blair to meet campaigners against genetically modified food. Charles doesn’t like GM food and he was lobbying Blair as prime minister about the issue. This fits into the example of the disclosure as the news story.
I have to say on this example that it took me two years to get this information out of the Cabinet Office, and it took four rulings from the ICO at different points in that process. I’m grateful to the ICO for that, but it also shows the kind of attitude sometimes of the Cabinet Office. For example, first of all neither confirming nor denying that they held the information, then trying to insist it was not environmental information, and so on.
To give a different kind of example, here’s a story where the BBC had some case studies about people who were scammed into paying money to get COVID vaccines. It was the kind of thing that a lot of scammers piled into, trying to get money out of people by sending them bogus messages. But is it a big problem? Well, FOI enabled the BBC to get some kind of national overall context and say there’s over a thousand reports, people have lost nearly £400,000 to scammers through this scam, so it is a significant thing. That’s an example where FOI is a piece in the jigsaw, you’ve got the case studies and then you’ve also got overall statistics.
Next is a different kind of example of a story I did once, and this is one of my favourite examples of the success of FOI, for reasons which I’ll explain.
This is about which makes and models of cars are most likely to fail MOT tests. When I first requested this data from a Department for Transport agency, I was told that this information would not be released to me under the commercial Interests exemption, presumably because it would be damaging to the commercial interests of people who sold cars that didn’t do well. Again, I appealed to the ICO, and the ICO again ruled in my favour, forcing the Department for Transport agency to release this information, which I regard very much as in the public interest, as useful to potential car purchasers, knowing which cars are most likely to do well in MOTs and which aren’t.
This information is now released annually routinely as open data. In other words, what for 18 months the Department for Transport told me was so confidential it could not be disclosed is now released routinely and proactively. It’s a very good example of the benefits of FOI, transforming information that was previously secret into something which is released as a matter of routine.
Another instance of this is food hygiene reports, where again we had councils refusing to release food hygiene reports, and where again the ICO stepped in and ruled that it’s in the public interest for the public to know which food outlets have good food hygiene and which don’t. And this has now become information which the Food Standards Agency releases routinely as a matter of course. We have ‘scores on the doors’, so you can see on the outside of a restaurant how it’s doing.
That transformation has enabled journalists to do this kind of story – how do restaurant chains compare? To give the quick summary, if you want to go to a place with good food hygiene consistently across the country, when we did this study Nando’s had a satisfactory score everywhere. So you can’t say you don’t get useful information out of this kind of talk.
Another example I want to share illustrates the kind of information I think of as practically useful. This was a graphic I generated based on data from individual hospitals, showing A&E waiting times by hour of day and day of week. Each line represents a different hospital. The pattern is clear: if you turn up at midnight, any day of the week, you’re likely to wait much longer than if you arrive around eight o’clock in the morning.
That’s useful information, and yet it wasn’t being published at that level of detail by NHS England, even though it was collecting the data. I had to request it through FOI. Without the law, I wouldn’t have been able to obtain it.
The negative
So that’s some of what I see as the positive potential of FOI, and what I and others have achieved with it, but on the other hand FOI doesn’t always work well. Journalists frequently complain about delays and obstruction. Here’s a headline from the Press Gazette, the news trade website: ‘Freedom of Information in the UK sinks to new low’. This story is from 2024, last year, but you could probably find a version of that story in the Press Gazette every year. Because FOI still all too often doesn’t function as it should.
I first want to make a general point. Part of the problem is a broader shift in official attitudes. In the past there were more people in positions of power, contrary to the views of Tony Blair, stressing the benefits of openness and transparency. In the early years after the law came into force in 2005, there was rhetorical support from government for openness. Lord Falconer, the minister responsible for FOI at the time, attacked the ‘culture of secrecy’ – you don’t hear government ministers now saying that kind of thing.
Then under the Cameron government, the Cabinet Office minister Francis Maude was very keen on open data – which is not the same as FOI, but is part of a wider transparency agenda.
Again we don’t hear that from ministers today, that rhetoric about the benefits of transparency and openness. And this climate does have an impact on how public authorities behave, and in some respects they are more resistant to releasing information.
Here’s an example. Early on in the FOI period I obtained documents showing how the Metropolitan Police Special Branch had monitored the Anti-Apartheid Movement in the UK over 25 years. They were released in 2005. There is absolutely no way the Metropolitan Police would release that information today. In the early years of FOI they were generally more committed to transparency, and it’s an example of how things have deteriorated.
Sometimes I’m accused of being too cynical, so let’s take a look at what the chief executive of the Environment Agency, Philip Duffy, said at a conference last year: “I see these letters and these FOI requests and I’ve got great volumes of them, and I see local officers going through contorted processes not to answer when they know the answer and it’s embarrassing.”
This is a clear statement from the chief executive of the Environment Agency about how his officials behave, trying to avoid releasing embarrassing information. This is obviously a completely unsatisfactory state of affairs. It’s not only the Environment Agency, in some ways what we have here is Philip Duffy being more honest about how FOI sometimes operates than other public authorities often are.
Another example: a slide from internal Cabinet Office training on FOI. It says: “If you don’t want to appear in tomorrow’s newspapers, consider carefully what you send out.” That’s basically telling people don’t release stuff which is embarrassing, and it’s not what the law says. There’s no FOI exemption in the Act for “Stuff you don’t want to appear in tomorrow’s newspapers”. That should be a totally irrelevant consideration, yet here we have it appearing in the Cabinet Office guidance to staff on how to handle FOI requests.
I do think journalists in particular often have a problem with how their FOI requests are handled. To give just one example of the sort of processes that come into play, the Financial Conduct Authority guidance to staff on FOI explicitly states that approval for journalists’ requests must be “obtained from press office”. It’s very far from being the only body where this is the case, very far indeed, and I see two problems with this. First of all, it inevitably increases delay, because the reply has to be checked by the press office. But secondly, for people who work in the press office, their job is to protect the reputation of the organisation where they’re working. That is entirely separate from FOI considerations and they should not be involved in approving the answers to FOI requests.
This is another statistical analysis I’ve done, of how often internal reviews change initial refusals. Again, as far as I’m aware, I’m the only person who is analysing the statistics issued about central government FOI requests to this extent. This chart shows how often internal reviews overturn, partly or entirely, an initial refusal from a government department that information should be withheld.
It’s only because we have statistics for central government that we can do this. We don’t know what the position is for all sorts of other public authorities. But what we can see here is that maybe 20 to 25 percent, and perhaps increasing, of internal reviews overturn an initial refusal, partly or entirely.
You can look at this in two ways. One is to say that internal reviews are actually doing their job, scrutinising the initial decision. But I also think that this rate is surprisingly high, suggesting that very often at the initial stage people are refusing FOI requests in ways that they should not be. This isn’t even the ICO getting involved. This is just somebody else in the same organisation saying you should have released it, and I think it points towards an initial level of obstruction in some cases.
Now let’s look at how often the ICO overturns decisions made by public authorities. This is an analysis I did based on a dataset going back to 2005 which the ICO released last year. I don’t know to what extent this kind of analysis is done internally within the ICO or not.1
What I find really striking about this is how for some of these exemptions, the ICO overturns them so often. Taking the ones at the top, the economy and relations within the UK are not so often used, but commercial interests is a very frequent exemption used by public authorities to turn down requests. Yet of the cases that have been appealed to the ICO, basically in half of them the ICO has overturned it.
And there are other exemptions – future publication, policy formation and so on – where the ICO is very frequently overturning their use. What that really points to is a systemic issue that these exemptions are being overused by public authorities. And what the ICO in my view needs to do is draw attention to that and say there is a widespread problem of the overuse of certain exemptions.
I’m aware that the ICO has issues about resources and its level of casework. So if authorities could be stopped from overusing those exemptions, then that will be a way of reducing the level of casework that the ICO gets, and this is another reason why the ICO should be vocal in flagging the systematic overuse of certain exemptions.
The ICO
As I’m talking to the ICO, I should tell you my view of the ICO. This is what I wrote in my book: “Despite problems of resource constraints and significant delay (and the fact that I sometimes disagree with its decisions)” – we’ll come onto that later – “its existence is one of the strengths of the UK’s FOI system”.
When I’ve done training for journalists, this is one of the points that I make, that it’s a strength of the UK system to have an independent body which can overrule public authorities. That’s not the case in some other countries, and the ICO’s role is very valuable.
Let’s take some aspects of its FOI work in more detail. On delay, I very much welcome what the ICO has done over the past couple of years or so to speed up its consideration of complaints about FOI.
In contrast there was an extreme example back in 2009, when the ICO ruled in my favour on releasing the minutes of a Thatcher cabinet meeting about the Westland crisis, where it took the ICO four years to reach that decision. That’s ridiculous. It’s an extreme case, but when the ICO takes that sort of time, then public authorities have tremendous incentives to boot issues into the long grass, turning down requests for information on the basis that by the time the ICO rules, no one will care about it. A lot of progress has been made in speeding up the ICO’s procedures, and it’s vital that it doesn’t deteriorate again.
The fact that the ICO now prioritises certain FOI complaints with significant public interest is also a welcome and valuable step which enables the most important cases to be dealt with most quickly. Maybe the threshold for prioritisation is a bit too high and probably more cases should be prioritised, but in general it’s absolutely right to have done this.
I’m also very pleased by the way that the ICO is now taking serious regulatory action against recalcitrant public authorities. For example, the enforcement notices and practice recommendations served on numerous police forces recently and rightly, because the performance of the police in FOI terms has often been very bad, as has also been the case for various government departments, certain NHS bodies and so on.
It’s much more effective for the ICO to take this kind of regulatory action, issuing enforcement notices, than the kind of situation that we used to see beforehand.
Take this example that I wrote about the Cabinet Office. You would have this flow of decision notices from the ICO referring to Cabinet Office delays as being unacceptable, unhelpful, extreme, protracted, considerable, unsatisfactory, excessive, prolonged, severe. One decision notice after another would say this, but nothing ever changed by just issuing decision notices where it says in passing that delays are unacceptable.2
In contrast taking proper regulatory action and issuing enforcement notices is the best way to achieve change from the most recalcitrant public authorities, and I’m very glad to see that the ICO has been doing that.
However, I’m not going to say that everything about the ICO is completely marvellous. I’m pleased about reducing delays, I’m pleased about the prioritisation, I’m pleased about the enforcement action. But I do think there is an issue about the quality of certain ICO decisions, and there are decisions where the ICO does not really address properly the balance of the public interest.
This includes recent cases where I’ve been successful in appealing ICO decisions to the First-tier Tribunal.
One example involves the House of Lords Appointments Commission and the citations provided for some of Boris Johnson’s appointments to the Lords. In my opinion it’s overwhelmingly in the public interest that the public knows what are the official reasons given for why people are put in a legislative assembly. The tribunal agreed with me on that point3 that the citations should be public. So the House of Lords Appointments Commission was forced to reveal them.
I was disappointed, however, that I needed to appeal to the tribunal in order to achieve this. This is a case where the ICO should absolutely have said it’s in the public interest for this to be out in the open, and people could then assess for themselves the reasons that Boris Johnson gave as to why people should be members of the House of Lords with the right to decide on laws everyone else has to abide by.
That’s one example, and another occurred just last month when the First-tier Tribunal ruled that I should be given information in a Cabinet Office case where the ICO had ruled against me on a public interest test.
The tribunal said: “The tribunal shares the appellant’s view that there is a strong public interest in upholding standards of good decision-making, transparency and integrity in public administration … In the tribunal’s assessment, the withheld material is relatively benign and does not contain candid or controversial exchanges. The tribunal finds that the Cabinet Office’s concern over inhibiting future frank advice is overstated and not supported by the content of the material in question.”4
Here we have another example of the ICO in my view not taking a strong enough stance on where the balance of the public interest lies, and then the tribunal has to step in. That means extra work for me as the requester to appeal to the tribunal, but it’s also extra work for the ICO, which then ends up trying to defend an untenable position at the tribunal, costing a lot of time and money, as well as being against the public interest.
I know there are other cases where the ICO has taken a position which has annoyed the Cabinet Office. But I think it doesn’t do that often enough, and the ICO should be more prepared to stand up to the Cabinet Office on cases like these.
Finally, I wanted to make one other point, and this goes back to what I was saying earlier about the public debate about open government. I’d like to see the ICO now speaking out more about the broad advantages of transparency. We need to be hearing the voice of the ICO more often in public discussion.
To give one specific example, a government policy document issued last year states that it’s going to extend the FOI Act to private companies that hold public contracts, and that the government will take this forward “in due course”. So it’s publicly committed, but we’ve no idea when it will actually do this, or even if it will really happen. It’s not in the current legislation.
This is the kind of thing that in my view the ICO should be talking about now. It should be stepping into this discussion and saying this is an important and valuable change that is needed, because so many services are contracted out in the way that our public sector now operates.5
I’ve left time for questions. Thank you very much indeed.
——————————————————-
1 During the Q&A session afterwards I was told that the ICO does do this analysis itself.
2 On the day after I delivered this talk, the ICO announced it was issuing a practice recommendation against the Cabinet Office due to its FOI delays.
3 ICO staff are keen to point out that the tribunal did not agree with me on everything and upheld some other parts of their decision.
4 The material in question can now be found here, if you want to decide for yourself whether its release will damage free and frank discussion in the civil service in future.
5 In the Q&A I was reminded that the ICO has issued a report on this topic, but the point I made was that this was done back in 2019.
Twenty Years of FOI: A User’s Perspective (My presentation to the ICO) Read More »
What will be the political impact of the additional challenges that MRP polls may face at the next general election?
Last week’s local election results suggest that we are entering a new phase of multi-party politics across Britain.
Setting aside the local and national policy consequences, the impact will also be to make our elections harder to forecast and to complicate the task of the polling companies.
Yesterday I attended a meeting organised by the British Polling Council to discuss ways of tackling the industry’s failings at the last general election, when the polls significantly over-predicted how well Labour would do.
The next general election is likely to present further difficult challenges for the pollsters, particularly for MRP polls which proliferated in 2024. I think this could have important political ramifications.
The MRPs apply statistical modelling to survey data in order to produce individual constituency forecasts based on the local demography, and thus predict how many seats each party will win. Despite the headline on this piece, it stands for Multilevel Regression and Post-stratification, rather than Maybe Right, Perhaps.
Last year the MRPs made a positive contribution to understanding the pattern of public opinion by correctly showing that Labour and the LibDems would benefit from different swings in different seats, rather than the traditional norm of roughly uniform swing across the country. They were therefore a very useful expansion of polling techniques.
However the MRP polls all exaggerated the level of Labour success, as I have previously analysed. This systematic error across the industry stemmed largely from the voting intention polling figures which were fed into the statistical models.
If, as seems probable, the current electoral fragmentation continues until the next general election, then predicting constituency winners will surely get harder, for the following reasons.
All this will make forecasts more sensitive to problems with unrepresentative survey samples and any flawed assumptions or procedures in the statistical modelling. It is also likely to produce greater differences in constituency predictions between the various pollsters.
We already saw in 2024 how the MRPs can profoundly affect a campaign. Their forecasts for each seat were relied on by a number of tactical voting websites, and in local electioneering political parties made great use of those estimates which were convenient. The MRPs also probably had a substantial impact on the morale of some party activists, both positively and negatively.
All these points were made at the BPC event yesterday. For example, Martin Baxter from Electoral Calculus described how he received irate complaints about some local parties quoting out-of-date seat analyses in their election literature, but despite his efforts there was nothing practically he could do to stop them.
I expect next time there will be greater variation and inconsistency, with more opportunities for parties to cherry pick and publicise forecasts that suit them. We’ll also see more instances of different tactical voting organisations issuing contradictory advice. Sounds like a recipe for chaos and confusion. And perhaps more calls for polling to be banned during campaigns.
However I should note that one factor will help the MRP statistical modellers at the next election. As Prof Chris Hanretty pointed out yesterday, they won’t have to cope with the complication of new constituency boundaries.
As well as these challenges, the MRPs will also face the fundamental issue that the political polling industry in general does – the accuracy or otherwise of voting intention data.
The major problematic factors in 2024 considered by the BPC’s member companies, as I have reviewed in the past, were late swing, ‘shy Tories’, difficulties with reaching over 75s and the less politically engaged, and religion/ethnicity.
No one knows of course whether next time there will be much late changing of mind by the electorate. Some of the other concerns may be dealt with by more sophisticated demographic modelling and more ingenious or determined ways to survey the kind of voters who aren’t enthusiastic about being polled. But the problem of ‘shy Tories’ may get trickier to handle.
Pollsters historically and internationally have faced a frequent (but not universal) difficulty of under-stating backing for right wing parties (as can be seen in this chart presented by Prof Will Jennings). In the UK most pollsters try to manage this by weighting samples according to how people voted in the past.
Yet at a time of increasing volatility in the electorate, with chunks of public opinion churning around in all sorts of different directions, this is becoming much more awkward than in an era predominantly of neat two-party uniform swing.
This may also leave pollsters with dilemmas, as was illustrated at the meeting by Robert Struthers of BMG Research. Given how age was very strongly associated with voting patterns in 2024, it would surely make sense to take account of mortality and adjust for Tory voters (who tended to be much older) being more likely to die between then and the next election. But if you are already worried that your polling is under-stating Tory support, this would only take you further in the wrong direction.
Prof Patrick Sturgis also raised what could become a growing problem in the world of online survey research, which is that of questionnaires being completed by bots or organised bogus respondents, so that financial or other incentives can be claimed. This could be exacerbated if the fakers increasingly purport to be the hard-to-reach groups that pollsters may be upweighting in analysing samples.
It’s expected that the presentations (which I thought were impressively interesting and candid) given by the polling companies at yesterday’s meeting will be placed on the BPC website, to add to the analyses which are already there. Well done to the BPC, which aims to increase transparency in the UK’s political polling industry, for arranging the event.
The pollsters are continuing to grapple with all these issues. In particular they are awaiting the release of delayed data from the large-scale academic British Election Study, which may shed further light on what went wrong for the industry in 2024.
Maybe Right, Perhaps Read More »
I’ve been analysing recently published data from the Electoral Commission about local campaign spending by candidates at last year’s general election.
Newly released figures on campaign spending at the last election confirm the suggestion that Labour sharply limited its electioneering efforts in seats which the Liberal Democrats might win from the Tories.
My analysis of Electoral Commission data shows that in potential LibDem target seats, Labour spent on average about £4,600 less than it did in other comparable constituencies.
This chart demonstrates that in most places where the LibDems could be hopeful of victory, Labour generally spent a fairly small proportion of the maximum legally allowed for local campaigning at the 2024 election. This was in stark contrast to its general pattern of expenditure.
On average Labour candidates in these LibDem targets used only 26% of the legal limit, whereas Labour’s average in other seats was 68%.
While it could be argued that some of these seats were low Labour priorities as probably unwinnable, my statistical analysis shows that Labour put much less effort into campaigning in LibDem targets even after controlling for how well or badly Labour was positioned after the previous election in 2019.
Looking at seats then held by the Conservatives, and after taking account of the swing Labour would have needed to take the seat in 2024, Labour’s local campaign spending was £4,597 lower on average in these possible LibDem targets.
Party strategies
This fits with the claims made during the 2024 campaign that Labour was diverting its resources away from these seats, sometimes to the annoyance of disgruntled party activists.
In their local campaigning the LibDems concentrated resources very tightly indeed on their priority constituencies. Since these were predominantly ones held by the Conservatives who were clearly in massive electoral difficulties, they did not spend much money in other seats which were Labour or Conservative-Labour marginals.
These strategies suited both parties nationally who could each successfully focus their energies on maximising Tory losses and not competing against each other, without the need for any official agreement which would have been highly difficult and controversial.
I’ve defined LibDem targets here as constituencies where after 2019 they were the party in second place after the Conservatives (along with the much smaller number of places they held). This is a clear and reasonable demarcation.
However, it would not be identical to the list decided on by their party HQ, which added or dropped seats in line with fluctuating political circumstances and local factors. In any case this is more about which seats Labour would regard as sensible LibDem targets than the LibDems themselves. Nevertheless, even if there is a discrepancy of a few locations, that would not invalidate the very strong pattern this analysis identifies.
The local data published last month also reveals a very interesting contrast between Labour and Conservative spending patterns, which reflected the different political situations they faced when going in to the election.
This is indicated in the next two charts, which compare the party’s spending in each constituency in 2024 to its strength there after the previous general election in 2019.
The ^ shape in the Labour graph shows the party tended to spend most in marginals, with significantly less expenditure in many hopeless seats and safe ones.
For the Conservatives, however, they had to concentrate resources on the seats they already held and were not able to assume any were safe. So their spending was much higher in places where they’d done better in 2019.
This expenditure refers to money parties use for electioneering in each constituency, including leaflets, posters, letters, meetings and office costs.
The legal limit in each place depends on the size of the electorate and whether the constituency is urban or rural. In 2024 the maximum allowed was generally in the range of £16,000 to £21,000.
Overall Labour spent £7.2 million on local electioneering, while the Conservatives spent a little less, £7 million. For the LibDems the total was £3.2 million, for Reform £1.3 million, and for the Greens £700,000.
This is separate to money used for national campaigning. The legal cap on that in 2024 was £34 million for parties standing candidates throughout Great Britain. The Electoral Commission has not yet released the details of the national sums spent by the major parties.
There is some evidence that spending on constituency campaigning does matter. I analysed how many extra votes each party got for an extra £100 of local spending, after controlling for the party’s vote share in that constituency in the 2019 election.
The figures are given in the table below. However it is difficult to read much into this, as parties would also be devoting more effort to places where they felt things were going well, especially the LibDems, Greens and Reform. So the overall direction of causation is far from clear.
* This is the number of extra local votes in 2024 for that party associated with an increase of £100 in its local spending, after controlling for its votes in 2019 to take account of its position going into the election. (In the case of Reform, I used the figure for how the Brexit Party did in 2019, although they didn’t stand in many seats).
The art of the deal, LabLib style Read More »
On Tuesday June 3 in London I’m giving a talk on how to use freedom of information productively to a Meetup for the London Business Analytics Group. It’s a free and friendly event, open to everyone. So please do come along if you’d like to know more (and do tell others about it) – details here.
Forthcoming talk about using FOI Read More »
I’ve found archive documents which show, contrary to recent reports, that Denmark icily rejected the UK’s demand to have right of first refusal if Greenland was ever to be sold
In the past couple of weeks some news reports have suggested a peculiar difficulty with President Trump’s self-proclaimed intention to acquire Greenland from Denmark – an agreement from a hundred years ago which apparently gives Britain ‘first dibs’.
Supposedly the UK has the right of first refusal if the Danes ever decide to sell the icy but strategically located and mineral rich island territory (which of course they insist they won’t). The articles, which have appeared internationally, all seem to be based on statements made by a former Danish government representative to Greenland.
In fact, I’ve located the relevant documents in files at the UK National Archives and can report that it’s not true.
The reality revealed by these papers is that in 1920 the British government did indeed tell the Danish government that they would only recognise full Danish sovereignty over Greenland in return for the right of first refusal if the island was ever sold.
But the Danes completely dismissed this suggestion. Britain then backed down and agreed recognition, while merely asserting that if ownership was transferred then the British government ‘must reserve their right to be consulted’. To which the Danes sent no response.
Intriguingly, I also discovered that I’m not the only person who’s just been to the National Archives in Kew to search through obscure binders from a century ago containing paperwork about cryolite mining, musk oxen, trading outposts, and other aspects of life in Greenland, along with diplomatic dealings.
My access to several folders I had ordered in advance was postponed for several hours because, according to archive staff, they had been taken to be read by someone ‘from the government’.
So what do the documents disclose?
In an era when it wasn’t uncommon for colonial nations to trade land (and control over its population) for money, a Foreign Office memo shows the issue had previously arisen.
By the time of the First World War, the Danish government was extremely keen to obtain formal international recognition for its authority over all of Greenland.
In 1916 the Danes got the United States to accept this, with the Americans thereby renouncing any claims of their own, as part of a treaty in which the US purchased from Denmark what then became called the US Virgin Islands.
In the wake of the war Denmark wanted all the leading victorious powers to agree. From the internal discussion in the archive records it is clear that there was some unease within the British state, particularly the Colonial Office (CO). This was largely because of potential future implications for Canada, a neighbour to Greenland and at that time a dominion within the British empire.
The Danes wanted to raise the issue at the post-war Paris Peace Conference in 1919, but were persuaded not to. In 1920 they formally asked for recognition.
In line with the wishes of the Canadian government, who had been consulted, the Foreign Office informed the Danish ambassador in London that the UK would only recognise their complete sovereignty over Greenland on one condition: that ‘should Denmark ever wish to dispose of the island, she would grant the right of pre-emption to the British Empire’.
This didn’t go down well with the United States government, who were notified. The US ambassador to London John W Davis wrote to the Foreign Secretary Lord Curzon to say that his government ‘is not disposed to recognise the existence in a third government of a right of pre-emption to acquire this territory if the Danish Government should desire to dispose of it’.
More importantly, the guarantee which the UK wanted didn’t prove acceptable to the Danes.
The Danish ambassador replied, stating: ‘The King’s government direct me to observe that there is absolutely no intention of selling Greenland and that there is no prospect that such a sale will ever become actual. Accordingly the King’s Government on grounds of principle regret that such a guarantee cannot be given as such an undertaking might imply the possibility at some future date of the question of sale being raised in some form.’
The British then felt they had no alternative but to retreat to a much weaker backstop position, which was agreed with the Canadian government. This said only that ‘His Majesty’s Government must reserve their right to be consulted should the Danish Government contemplate the alienation of this territory’. (I couldn’t find a copy of the original message in the archives, but did come across this later officially typeset version).
‘Alienation’ here is used in the legal meaning of transferring property rights, rather than the current everyday one of causing disaffection. As it happens, in the latter sense the Danish state has over the decades sometimes gone a considerable way towards alienating parts of the local population, but that’s another story.
Greenland is now semi-autonomous and will hold elections next month, in which the question of full independence from Denmark will be a leading issue. The island is currently dependent on substantial financial support from Denmark, but global warming could bring increased economic opportunities with mining and international transportation.
It looks like the Danes never replied.
One Foreign Office official wrote that this was presumably ‘an oversight’. But possibly the Danes didn’t think that this final British missive merited a response.
Two years later in 1922 the question arose of whether these exchanges between the governments should be published. This stemmed from an enquiry from the Norwegian government, which was contesting Danish sovereignty over the island in a dispute about fishing and hunting rights.
The suggestion that these confidential messages should be released did not meet with enthusiasm within the Foreign Office, where an official wrote that ‘this correspondence seems in general unsuitable for publication’. There was particular concern not to reveal that the US objected to the British demand for first refusal.
In the file there follows some discussion of whether to release some much more limited information: just the initial Danish request for formal recognition of sovereignty and the final British response. However from the records I could find, it’s not clear what eventually happened and whether any of this was actually revealed to the public at the time.
But in any case it is all revealed here and now. And it’s a very good thing that this paperwork is carefully preserved and made available by the National Archives, so that these questions can today be researched and answered. I hope the person ‘from the government’ found what they were looking for.
My thanks to Liz Evans, one of the independent researchers based at the National Archives, for her specialist expertise and assistance in locating some of these records.
No, Britain doesn’t have first dibs on Greenland Read More »
After a freedom of information dispute lasting 18 months, I have finally obtained the official reasons Boris Johnson provided with his resignation honours list for giving peerages to Charlotte Owen and Ross Kempsell.
The reasons cited for nominating Owen to membership of the House of Lords do come across as very thin, inadequate and lacking in evidence of relevant achievements. They leave her peerage as a mystery rather than properly justifying and explaining it.
The justification which the former prime minister supplied to the House of Lords Appointments Commission states that Johnson “entrusted Charlotte with engaging the parliamentary party on his behalf and she was essential to maintaining his relationship with them”. He added that “Charlotte was later tasked with helping the new Chief Whip in his role and used her unique knowledge to become a bridge between the Prime Minister and the Chief Whip”.
Johnson’s citation for her also states that “Charlotte led on many sensitive and key projects including advising the Prime Minister and the Chief Whip on suitability for Ministerial appointments during the reshuffle”. No other details of these “sensitive and key projects” are given.
The first part of this citation fits with the notion that she was a junior official with very little responsibility and experience, whose sudden elevation to a peerage was peculiar.
The unevidenced claim in the citation that she led on “many sensitive and key projects” including on advising on ministerial appointments will, I think, come as a surprise to a lot of people working in Downing Street at the time.
The citation added that Owen wanted to advocate for young women who have fallen victim to image based sexual abuse, an issue she has indeed taken up since being in the House of Lords.
The released documentation also reveals that her nomination was supported by the former Tory cabinet ministers Grant Shapps and Chris Heaton-Harris, although not what they said.
The citation for Ross Kempsell is more detailed, giving an account of his work and achievements as the head of the Conservative research department and a special adviser in Downing Street.
The House of Lords Appointments Commission was eventually forced to release this material due to a lengthy freedom of information case I have been pursuing with them.
I am very pleased that the documentation has now been revealed, but it shouldn’t need an argument over 18 months for the public to find out what reasons are officially provided for allocating certain people important political powers.
Members of the House of Lords debate and vote on laws that control the British public’s lives. As a basic principle the public is fully entitled to know what reasons are given for why they have been appointed to rule over us.
The two peers are now known as Lady Owen of Alderley Edge and Lord Kempsell, and have both been active in the Lords after their somewhat unexpected appearance in Johnson’s resignation honours in June 2023.
At the time of the announcement they were 29 and 31 respectively. Owen’s ennoblement caused a great deal of consternation and puzzlement, as there was no evidence of any achievement of hers that could explain it.
She was a junior staffer working in Johnson’s Downing Street operation. Press reports described her as possessing “no views, no achievements, no experience” and as “the most junior person in political history to have received a peerage”, while also alleging discrepancies in her career history.
The disclosure follows a ruling last month from the First-tier Tribunal, which upheld my arguments that releasing this information is in the public interest.
It is the culmination of the dispute between myself and HOLAC, which initially rejected my FOI request in July 2023 for the information held about these two nominees. After HOLAC insisted on its position, I complained to the Information Commissioner and then appealed to the Tribunal.
At the Tribunal hearing, where I represented myself against HOLAC’s extensive legal team and cross-examined Clare Brunton, the secretary of HOLAC who is also head of the honours secretariat in the Cabinet Office, I argued that the process for giving individuals a status as legislators is a vital matter of the public interest.
Those appointed can approve or reject proposed laws, as well as being able to take part in parliamentary debates, directly question ministers, and so on. This entails the need for maximum transparency, so that the process is both legitimate and is seen to be legitimate, and the public can see for themselves whether appropriate procedures are followed.
The First-tier Tribunal’s judgment stated: “We attributed considerable weight to the public interest knowing the PM’s reasoning … Life peers are Members of Parliament with the rights, obligations and influence associated with such an appointment thus enhancing further the weight of the public interest in the PM’s citations.”
The Tribunal dismissed HOLAC’s arguments that releasing the information would damage the honours system and be a breach of confidence.
The full citations disclosed to me are here.
If you are interested in taking an FOI case to the First-tier Tribunal, there is a chapter devoted entirely to this with detailed and thorough advice in my book, Freedom of Information – A practical guidebook.
Why Johnson said Charlotte Owen should get a peerage Read More »
The general election result last July was certainly a ‘Labour landslide’, but it wasn’t the even bigger, ginormous landslide which the polls predominantly predicted.
We were saved from the normal cliched headline ‘Polls Apart’, because the polls were all together on one side of reality, overstating Labour and understating the Tories.
I’ve been examining the reasons provided by those polling companies who have publicly tried to explain how these forecasts went wrong. They focus on the following factors: late swing, religion, turnout, ‘shy Tories’, and age.
The most recent company to publish its analysis was YouGov, which did so just before Christmas, also announcing that it would adopt a new methodology from January.
The election polls significantly overestimated Labour and underestimated the Conservatives, as shown in a chart from Will Jennings.
While this pattern has often happened, in terms of the difference between the two parties, this was their biggest miss since 1992, exaggerating the gap on average by 7 percentage points.
The constituency prediction models known as MRP polls were also all awry in the Labour direction, as demonstrated in the dataset collated by Peter Inglesby.
Of course some polls were much nearer to the actual outcome than others, as the companies that did reasonably well and got closest are naturally keen to stress, and as I myself have analysed in the past. But the industry as a whole clearly systematically over-predicted Labour, and that’s not good for the world of opinion research.
This is despite the fact that one can argue this was a tricky election to get right, with an increasingly volatile electorate, a very large swing, an important new party, the impact of independents, and changes in how demographic characteristics such as education and class link to voting behaviour.
If the result had been close, the level of polling error involved would have created a sense of chaos and surely have become a crisis for the industry. However the problem has been disguised by the fact that the only point at issue was the extent of the landslide, and so it did not disturb the central narrative of the election.
Pollsters are constantly seeking to improve their methods, and indeed the MRP models last July were a positive contribution to getting the overall impact correct, confirming the value of innovation. Companies have been reviewing their performance and what went wrong.
The British Polling Council (BPC) is collating relevant research from its members on its website. So far work from six organisations has been added. As well as YouGov, the others are BMG, Electoral Calculus, Find Out Now, More in Common, and Verian. It’s possible that more BPC members will add further submissions in due course.
I’ve been reading them to see what prevailing points emerge on an industry-wide basis.
It is important to note that given the companies have different methodologies, this implies there could also be variations in what each got wrong. But the fact that they were all out in the same pro-Labour/anti-Tory direction suggests that as well as any individual aspects there is also something significant which is shared.
Although there is no unanimity, their findings do reveal some common themes. (None of them discuss the issue of ‘herding’, the claim that error can be exacerbated if some companies sometimes take decisions in such a way that they stay in line with the crowd – a charge which is very unpopular within the industry).
Late swing
Three companies – BMG, More in Common and YouGov – attribute the error partly to ‘late swing’, due to people changing their mind about how to vote at the last minute after final opinion surveying ended. A cynic might say that this is the most convenient excuse for the industry, as it is the least challenging to the accuracy of their methods. Maybe, but the fact that it is convenient doesn’t necessarily mean that it is wrong.
Beyond the data presented, I have to say I also find this plausible based on anecdotal evidence, with the forecasts of a huge Labour victory nudging some intending supporters into eventually switching to vote for someone else, such as the Greens. In this sense the polls ironically could have been their own enemies, almost a kind of partially self-negating prophecy.
However Electoral Calculus finds no evidence of late swing, and in any case none of the companies thinks it can approach the full explanation, which still leaves a methodological challenge for the industry.
Religion/ethnicity
The pollsters seem to have failed to reflect the increasing fragmentation of the ethnic minority electorate, with some Muslim/Pakistani & Bangladeshi voters abandoning Labour, often for independent candidates who campaigned about the situation in Gaza, while Hindu/Indian voters drifted towards the Tories. This factor is referred to by BMG, More in Common and YouGov. It is clear that election analysis can no longer crudely treat voters of Asian heritage (let alone all ethnic minorities) as if they are one political bloc.
More in Common suggests that Muslims who currently take part in online market research panels are probably not representative of the overall Muslim population, being more likely to be second or third generation immigrants, and less likely to be born outside the UK or not speak English. The company says it will probably modify its weighting scheme.
Similarly YouGov says it will incorporate a more detailed ethnicity breakdown into its modelling in future.
However, the numbers of voters involved, while crucial in certain seats, mean that this could also only be a very partial factor nationally.
Turnout
Taking account of likelihood-to-vote is a notoriously difficult problem for pollsters, who employ a range of strategies to estimate how many of each party’s proclaimed supporters will actually go to the trouble of casting a ballot. Three companies – BMG, Electoral Calculus and YouGov – include the overpredicting of Labour voters’ turnout as a factor in the 2024 error.
YouGov argues the cause stemmed from panels which over-represented people who would actually vote, especially for low turnout demographic groups. The company says that from now on it will base turnout modelling purely on demographic data, rather than respondents’ self-reported likelihood-to-vote.
This sort of problem has been a general industry issue in the past, of over-sampling the more politically engaged (who tend to be keener to take part in this kind of survey).
However it is awkward for pollsters to get turnout adjustments correct. There is no guarantee that what worked best last time will be best next time, as the commitment of different groups to implement their asserted voting intentions may depend on the political circumstances of the moment. Ironically again, the forecast Labour triumph last July might have pushed some of the party’s less determined supporters into not bothering to go to the polling station on the big day.
Shy Tories
This has also been a traditional difficulty for the polling industry, where those of a Conservative outlook are somewhat less willing to express their allegiance – possibly because they feel in some sense disapproved of or intimidated (this is sometimes called ‘social desirability bias’), or perhaps alienated from polls. Again, the extent to which it happens can also depend on the political atmosphere of the time.
Over-estimating the Left and under-estimating the Right is not just a UK polling problem – it has cropped up as a fairly consistent (but not universal) pattern across many countries, as can be seen in the Deltapoll slide in this piece by Mark Pack.
The industry has tried to counteract this skew through various means of political weighting, such as using previous voting behaviour.
Electoral Calculus states there is indeed suggestive evidence of a ‘shy Tory’ effect in 2024, with people who refused to answer voting intention questions or who replied “don’t know” being more likely to be Tory voters. This is also consistent with the findings reported by BMG and by More in Common about ‘undecided’ voters who were then pressed.
YouGov suggests that its past vote weighting fell down in 2024 because at the previous election in 2019 the Brexit Party endorsed the Conservatives in many seats. The result was that its panels had too many 2019 Tories who actually preferred the Brexit Party and then voted Reform in 2024, and not enough firmly committed Conservatives. Their paper does not raise the issue of whether it is staunch Tories who are most likely to avoid voting intention opinion research, but it seems to me that this conclusion is compatible with their evidence.
Find Out Now (which only produced one unpublished poll during the 2024 campaign) argues against the ‘shy Tory’ hypothesis. But in my opinion their data only counters the hypothesis that online research panels under-represent Tories in general, as opposed to the hypothesis (advanced by Electoral Calculus) that Tories may be reasonably represented in panels but are disproportionately likely to refuse or reply “don’t know” when faced with a voting intention question in a survey.
More in Common also states that there is possible selection bias affecting online panels as the recruitment processes appeal to the ‘overly opinionated’.
Age
Age was very strongly associated with how people voted last July, with Tory support concentrated in the older section of the electorate.
The report from Verian (the polling company which came closest to the actual result on percentage vote shares) focuses entirely on the issue of age, and concludes that those companies whose samples contained a smaller proportion of over-65s (after weighting) tended to be less accurate. But its presentation adds that other biases would also have played a role.
Find Out Now raises a different possibility on age, that it failed to locate Conservatives who were younger and less politically engaged (a group that is hard for pollsters to reach).
Summary
At this stage we are left with the suggestion that perhaps four or five factors may have contributed together to the polling miss, and none explain it alone.
There can be a problem with this kind of analysis, dubbed the “Orient Express” approach by Electoral Calculus, where multiple possible causes are examined and all those which affect the error are deemed part of the solution. In other words, as in the Agatha Christie story, if everyone/everything is responsible for what happened, then eventually no one/nothing is actually held responsible, and nothing is done.
On the other hand, looking at the underlying fundamentals, it seems to me that predictive opinion polling is a difficult business given the level of precision required and the volatility of today’s voters. There are many sources of potential error (apart from normal sampling variation), arising from which people get contacted, whether they reply or tell the truth or change their minds later, how the electorate is modelled, and how the answers from different groups are weighted to aim at representativeness. And errors that arise are difficult to eliminate methodologically, as they depend on political circumstances which vary from one election to the next, and also on the communications technology for conducting research which is constantly evolving and in different ways for different social groups.
Inevitably therefore pollsters are bound to make some mistakes (and not all will make the same ones). When they are lucky, the errors may cancel themselves out, more or less, and nobody notices them. When the pollsters are unlucky, the errors largely or entirely mount up in the same direction.
Further, more thorough analysis will be possible once detailed data becomes available from the academic British Election Study and its extensive voter research.
The British Polling Council, to which all the main pollsters belong, is also planning to hold a public event to discuss these issues, probably in April.
Where did the polls go wrong? Read More »
The public can now expect to find out the reasons Boris Johnson officially provided for nominating Charlotte Owen and Ross Kempsell to the House of Lords, due to a lengthy FOI battle I have pursued.
A Tribunal has ruled that the confidential recommendations put forward by the former prime minister for two of his most controversial peerage appointments must be disclosed.
It has just ordered the release of the secret citations Johnson sent to the House of Lords Appointments Commission (HOLAC) for the peerages awarded to Charlotte Owen and Ross Kempsell in his resignation honours list in June 2023.
This is the latest stage in an 18-month freedom of information dispute between myself and HOLAC, which turned down my FOI request in July 2023 for the information held about these two nominees.
The First-tier Tribunal, which hears information rights cases, has now backed the view that releasing the citations Johnson submitted to justify the nominations is in the overall public interest.
It has also instructed that the identity of some public figures who were indicated as supporting Charlotte Owen’s appointment should be revealed.
I’m very pleased by the decision, which is a boost for transparency and democracy. Members of the House of Lords debate and vote on laws that control the British public’s lives. As a basic principle the public is fully entitled to know why they have been appointed to rule over us.
According to HOLAC, citations provide ‘the reasons for nomination’ and a statement of ‘personal and professional background and attributes’.
The two individuals involved are now known as Lady Owen of Alderley Edge and Lord Kempsell, and have both been active in the Lords after their somewhat unexpected appearance in Johnson’s resignation honours.
At the time of the announcement last year they were 29 and 31 respectively. Owen’s ennoblement caused a great deal of consternation and puzzlement, as there was no evidence of any achievement of hers that could explain it.
She was a junior staffer working in Johnson’s Downing Street operation. Press reports described her as possessing “no views, no achievements, no experience” and as “the most junior person in political history to have received a peerage”, while also alleging discrepancies in her career history.
The Tribunal’s judgment published today states: “We attributed considerable weight to the public interest knowing the PM’s reasoning … Life peers are Members of Parliament with the rights, obligations and influence associated with such an appointment thus enhancing further the weight of the public interest in the PM’s citations.”
The Tribunal dismissed HOLAC’s arguments that releasing this information would damage the honours system and be a breach of confidence.
The decision means HOLAC has until 22 January to release the material.
I brought the case to the Tribunal to appeal against the Information Commissioner, who in March this year had upheld HOLAC’s rejection of my request. It shows that it is worthwhile challenging weak decisions from the IC.
The Tribunal has now overruled the Commissioner, following a one-day hearing in October. However it did not back all aspects of my appeal, coming down against the release of the detailed minutes of the HOLAC meetings which discussed Owen and Kempsell.
At the hearing, where I represented myself against HOLAC’s extensive legal team and cross-examined Clare Brunton, the secretary of HOLAC who is also head of the honours secretariat in the Cabinet Office, I argued that the process for giving certain individuals a status as legislators is a vital matter of the public interest.
Those appointed can approve or reject proposed laws, as well as being able to take part in parliamentary debates, directly question ministers, and so on. This entails the need for maximum transparency, so that the process is both legitimate and is seen to be legitimate, and the public can see for themselves whether appropriate procedures are followed.
Earlier this month the Labour government announced that in future the citations to support individual nominations for political peerages will be published. This is also a welcome move towards much-needed greater openness in the appointments system for members of the House of Lords.
If you are interested in taking an FOI case to the First-tier Tribunal, there is a chapter devoted entirely to this with detailed and thorough advice in my book, Freedom of Information – A practical guidebook.
FOI Tribunal orders release of Owen and Kempsell peerage citations Read More »
As Rachel Reeves ponders her forthcoming budget and how to balance raising money against economic growth, one of her self-imposed constraints is her pledge not to raise the rate of VAT. However the impact of taxes also depends greatly on the thresholds from which they apply, even though this tends to get a lot less attention in public debate (as is certainly the case for income tax).
So what about the annual turnover level at which businesses have to register for VAT?
Data I have recently obtained from HMRC under the freedom of information law shows the dramatic impact of the VAT threshold in restricting the growth of some of the UK’s small businesses.
In 2021/22 the UK had 21,752 businesses with annual turnover in the range £84,000-£85,000, just below the then threshold. But there were only 10,096 businesses just over the limit, in the range £85,000-£86,000.
In other words the number of businesses clustered just under the VAT threshold was more than double the number just above, as businesses curtail their activities to remain outside the VAT registration system.
The graph above clearly shows the cliff edge in the data.
Many small businesses are desperate to keep their annual turnover under the VAT level, so that they avoid the bureaucracy and costs of registration and they don’t have to charge VAT to customers, which would make them less competitive. However the consequence is that they then won’t grow further into larger, more successful operations.
For some businesses the VAT threshold functions as a ceiling constraining their growth.
Research by Warwick University in 2022 concluded that earlier data of this kind reflected genuine curtailment of business activity rather than false reporting to HMRC.
This is the latest data available from HMRC, which says that more recent information is still being processed. The current VAT threshold is now £90,000, as the figure was increased by the Conservative government before the general election.
The UK’s VAT threshold is high compared to other European countries which tend to impose VAT registration on businesses at a much lower level. While the UK policy saves many small businesspeople from the compliance burden of VAT, the significantly lower thresholds elsewhere make it less likely that enterprises will be found bunched and held back just under the relevant level of turnover.
I also wanted to get a breakdown of the data by sector of the economy, to see which kinds of businesses were most affected. HMRC said it could provide this for 2019/20, as it had previously extracted the information involved, but that more recent breakdowns would probably exceed the FOI cost limit.
According to these 2019/20 figures, the most dramatic effect is in the construction sector.
This data shows 4,445 construction businesses with an annual turnover of £84,000-£85,000, but only 1,425 in the range £85,000-£86,000. So the number of construction businesses appearing to have kept themselves just below the limit is over three times the number who grew a little more and just exceeded it.
The chart shows the impact for construction and some other economic sectors with large numbers of small enterprises.
These FOI releases from HMRC constitute the latest and most thorough official evidence of what the tax expert Dan Neidle of Tax Policy Associates has called ‘the VAT growth brake’.
The full HMRC spreadsheets can be downloaded here:
The VAT cliff edge: How the threshold impedes small businesses Read More »
Ever since the Freedom of Information Act came into force nearly 20 years ago, some unhappy public bodies have protested loudly about the resulting ‘administrative burden’. But what is less appreciated is how numerous authorities actually seem to find the law useful – to obtain information themselves.
A few weeks ago Redcar and Cleveland Council issued a statement asking that the new Labour government ‘significantly revises the scope of the act to reduce the burden on councils’.
The particular grievances itemised included the usual targets of FOI applications emanating from businesses and journalists, but the council also focused its ire on ones which came from ‘local/national government’.
Intrigued by this, I asked the council (under FOI, naturally) for details of recent requests from local and national government. And it turned out that there were more than I expected, covering a wide range of council responsibilities. See the list below.
In my view these are entirely reasonable and legitimate requests. If FOI enables councils to find comparable information from their counterparts, which assists with policy development, service provision, budgeting and external contracting, that is not a reason to curtail the law – it is another reason to stress how useful FOI is in contributing to the general public interest.
Incidentally, there weren’t actually any applications from national government departments, despite what Redcar and Cleveland Council claimed. There were however numerous requests from MPs and Peers (or their staff), which the council bizarrely lists as coming from ‘National Government Departments’, but that is obviously not the same thing.
Since January 2022, Redcar and Cleveland Council has received the following 29 FOI requests from public authorities (26 from other councils and three from national quangos):
From one council to another Read More »