The spreadsheet gives data separately for each provider of urgent and emergency care in England for 2021/22. For patient arrivals in each hour of the week, it shows the average duration of attendance there until discharge or admission – ie, until leaving the hospital or being admitted as an inpatient.
The overall A&E pattern is very much that there are longer waits in the late evening and overnight, shorter waits in the morning, with the afternoons/early evenings in the middle.
Source: Analysis by Martin Rosenbaum from NHS Digital data
In this chart each row going across is a different provider of emergency/urgent care in England (I have excluded those with only partial data or which are not 24-hour services), and each column is an hour of the week, going from 0000-0059 on Monday to 2300-2359 on Sunday. The red cells show longer average waiting times, the green cells shorter waits, and the yellow ones intermediate times.
It makes clear that for almost all providers, patients who arrive just before midnight or in the hours afterwards experience the longest waits on average, while those who arrive in the morning have the shortest waits.
This pattern is the same on every day of the week, including weekends. The very longest waits of all tend to be overnight from Monday to Tuesday.
The exceptions to this are mainly urgent treatment centres rather than A&E departments – their busiest times are often late afternoon or early evening. They appear congregated towards the top of this chart due to the ordering of the NHS provider code system.
Some providers show much greater variation in waiting times across different points of the week than others do.
Overall national statistics about busy times of the day in A&E are published routinely, but as far as I am aware this dataset broken down by different local providers and hour of the week has not been released before.
In a period when there is increasing concern over waiting times for emergency and urgent care, it is important and valuable localised information.
Information Commissioner John Edwards launching his new strategy
John Edwards is clearly adapting to life in the UK, seven months after he arrived here from New Zealand to take up the role of Information Commissioner.
Launching his three-year strategic plan for the Information Commissioner’s Office yesterday, he succeeded in saying ‘day-ta’ most of the time, although he did manage to come out with at least one ‘dar-ta’.
While data protection (however pronounced) is certainly the main focus of his attention and indeed the dominant activity within his budget, the new Edwards plan is also important for those concerned about the FOI side of the ICO’s work.
To his credit Edwards has recognised publicly as well as privately that the substantial delays in the ICO’s handling of FOI complaints are not acceptable. In his speech he said that the administering of FOI law ‘requires fundamental change, and that change has to start in my office’.
He and his team have also shown a welcome willingness to engage with the FOI community and discuss (in his words) ‘how we fix a system that clearly needs fixing’.
As the ICO leadership itself recognises, things are now at the stage where requesters need to start seeing practical results rather than just hearing expressions of intention.
As part of its plan the ICO announced new objectives for its work on handling complaints under FOI law and the Environmental Information Regulations (EIR). These include: • ensuring that less than 1% of its caseload is over 12 months old • reaching a decision on 80% of complaints within six months • ensuring that 66% of tribunal appeal decisions go in the ICO’s favour
The current serious casework delays are illustrated by new up-to-date statistics which the ICO also released yesterday, with a commitment to do this regularly.
This is a valuable improvement, because in recent periods the ICO has only been publishing an inadequate series of datasets which are outdated, patchy and limited to closed cases.
The latest figures (for end of June 2022) show that 7.2% of active cases were over 12 months old. The ICO had reached a ‘first decision’ within six months on only 67.7% of cases.
The new dataset of open casework reveals that the ICO is still investigating one complaint relating to Brent Council nearly 26 months after it was made in May 2020.
There are 25 FOI/EIR cases that were still unresolved over 18 months after they were received. Of these, one of them actually involves a complaint about the ICO itself.
But the public authority which features by far the most frequently in these long-delayed cases is the Cabinet Office, which accounts for nearly half of them – 11 of the 25 involve complaints about its refusal to supply information requested. It is not clear whether this is because the Cabinet Office handles especially tricky issues or whether it is particularly slow and difficult about cooperating with ICO investigations, or indeed both.
As for the ICO’s performance objective for when its decisions are appealed by dissatisfied requesters or authorities, I have done a very quick analysis of tribunal cases so far this year, and the target of 66% appears pretty close to the current success rate. My rough calculation produced a figure of 68% (it is based on the summary outcome listed on the tribunals website without checking the details of individual cases).
ICO staff accept that some of their decision notices will inevitably get overturned, but there is always a risk that faster processing of cases can lead to more reversals later.
The ICO says its new approach will include more active prioritisation of significant cases and greater attempts at speedy resolution of disputes without the issuing of a formal decision notice.
For the ICO to improve its FOI casework handling is important and will be a relief to frustrated requesters.
However it is essential as well for the ICO to tackle the crucial systemic issues of poor performance on FOI by numerous public authorities. This week the ICO also released a new framework for regulatory action.
While it is understandable that ICO staff wanted to have this revised policy in place before implementing a tougher stance, it is unclear for the moment how much tenacity will be deployed in carrying out its series of measures and enforcing better standards.
One early promising sign could be the practice recommendation it delivered this week to the Department of Health and Social Care, reprimanding the department over its failure to properly search non-corporate communication channels when responding to FOI requests.
In the Q&A session at the launch I raised one particularly egregious example of FOI deficiency, the fact that in the latest set of government statistics the Department for International Trade had breached the legal deadline for replying to requests an extraordinary 55% of the time.
I asked Edwards what more evidence he would need before taking enforcement action against a badly performing authority and he replied simply: ‘None’.
If this does indicate a much firmer attitude is being adopted, that then is good news and will be strongly welcomed by requesters pursuing their legal rights to information.
I understand the ICO has taken up the issue directly with the DIT. But how persistently determined it will actually be in tackling this sort of blatantly inadequate FOI record remains to be seen.
Edwards announced that he had persuaded government to set up a cross-Whitehall senior leadership group to drive improved compliance on data protection within the civil service. There is some hope within the ICO that this could in future be extended to cover systemic issues of government FOI compliance too.
Unlike his predecessor Elizabeth Denham whose extent of contact with the Cabinet Office had become minimal by the end of her tenure, Edwards says he’s happy with the level of engagement he has been getting from the Cabinet Office.
Edwards also stated that the ICO will be publishing its internal staff training materials on both FOI and data protection, and this is expected to happen soon.
The ICO has initiated a consultation exercise on the three-year plan. If you want to express your views, you can do so here.
Source: Cabinet Office FOI statistics, my projection
My prediction is that today, somewhere in Whitehall, a civil servant will receive a request for information which is the half-millionth freedom of information application to UK central government departments since the FOI law came into force in 2005.
Their published quarterly figures don’t include the cumulative numbers since FOI started, but I’ve been keeping track of the running total, as shown in my chart above.
At the time of the latest published data, for quarter 1 of 2022, the cumulative number of requests to government departments stood at 491,838.
The Cabinet Office won’t be releasing their next set of statistics, covering Q2 of 2022, until September, but I’ve made a projection. Assuming the number of requests in Q2 of 2022 is 7.8% down on last year, the same factor by which requests in Q1 of 2022 were down on the corresponding quarter of last year, then my estimated cumulative total after the end of quarter 2 of 2022 is just over the half million mark, at 500,076 FOI requests submitted to departments since 2005.
On this basis the half-millionth request would just squeeze into the current quarter, today!
If you’re the recipient, congratulations. I feel it should be accompanied by some kind of prize, but sadly no one, not even you, will actually know.
So that is half a million questions. The number of decent answers? Well, that’s a different issue altogether.
Last week’s local election results appear to confirm how a candidate’s chance of getting elected to Scotland’s councils is dramatically influenced by a factor which is nothing to do with their abilities – alphabetical order of surnames.
This arises from the voting system used for Scottish council elections, the Single Transferable Vote (STV), where voters number candidates in their order of preference.
Parties will stand more than one candidate in a multi-member ward if they think they have a chance of getting more than one elected.
But of course lots of voters, who may have strong preferences between the parties, don’t particularly care about preferring one candidate from within a party to another.
It’s well established that under STV many voters have a tendency to number candidates from the same party just in the order they find them on the ballot paper, which is a major advantage for those listed first. In Scotland that is alphabetical order by surname.
To illustrate the striking extent of this I have looked at what happened last week in two Scottish councils, Aberdeen and West Lothian (the first and last councils alphabetically, in a limited attempt to avoid alphabetical bias in my selection).
I examined all the cases in these two councils where a party stood two or more candidates in one ward.
In West Lothian, there were 14 examples. In 13 of these, the candidate who came first alphabetically from that party got more first preference votes than the candidate listed second alphabetically, sometimes by huge margins.
The candidates listed first alphabetically for a party averaged 1,669 first preference votes; the candidates from the same party listed second alphabetically only averaged 745 first preferences – less than half as much.
The result was that the candidates listed first alphabetically for a party had a 100% success rate at getting elected; the candidates from the same party listed second alphabetically only had a 64% success rate of election.
In Aberdeen, there were 16 examples. In 14 of these, the candidate who came first alphabetically from that party got more first preference votes than the candidate listed second alphabetically, again sometimes by huge margins.
The candidates listed first alphabetically for a party averaged 1,223 first preference votes; the candidates from the same party listed second alphabetically only averaged 554 first preferences – again, less than half as much.
The result here was that the candidates listed first alphabetically for a party had an 88% success rate at getting elected; the candidates from the same party listed second alphabetically only had a 56% success rate of election.
Obviously it would be ideal to do this analysis for all the 32 local authorities in Scotland. But given the different locations and formats in which all the results are published, that would be a very laborious exercise which is too time-consuming for me to do right now. If there was one single national database of all Scottish local election results in a convenient format for exporting data then it would be a lot more feasible! (I also haven’t examined the impact in the very different political circumstances of Northern Ireland).
It seems clear that the current position in Scotland represents a form of institutionalised systemic discrimination. A council seat is often a step towards building a powerful political career on a bigger stage.
In the past the Scottish government has considered various means of ameliorating this situation but has not implemented any change. Potential options would include randomising the ballot paper order or listing candidates in reverse alphabetical order on half the ballot papers.
Parties could counteract the effect if they had loyal, disciplined voters who would order candidates as instructed, with different instructions issued to different subsets of voters. Roughly equalising the number of first preferences would help to get more than one of their candidates elected.
There has been some evidence of alphabetical voting affecting results in English and Welsh elections, but this is to a much lesser extent because of the different voting systems. Alphabetical voting is also an international phenomenon.
And alphabetical bias also exists in other contexts – here’s an interesting paper on its impact in an academic discipline where co-authors of papers were listed alphabetically.
By the way, when drafting this piece I noticed I had automatically defaulted to providing the Aberdeen data before that for West Lothian, so I went back and reversed that. But I did leave Aberdeen first in the chart.
The acceptance of alphabetical order as an apparently natural and unproblematic method may have a deeper and more insidious grip on our minds, and more important consequences, than we may consciously realise.
The Commons Public Administration Committee is calling for a ‘cultural shift’ across government, with ‘greater advocacy for the core principles’ of FOI and action to improve outcomes for FOI applicants.
In a new report which is highly critical of the Cabinet Office, the committee concludes: “The Cabinet Office needs to work harder to ensure a strong and enthusiastic tone from the top that supports FOI.”
The MPs also rebuke the Cabinet Office for a ‘lack of transparency’ on the workings of its FOI Clearing House, which advises other government departments on handling information requests. And they criticise the Cabinet Office’s refusal to allow an audit of its FOI operations by the Information Commissioner’s Office as ‘misjudged’.
The committee wants to see better timeliness in the freedom information process, through government conducting internal reviews of requests within the 20 days recommended in the Information Commissioner’s guidance.
These are some of the conclusions in the newly published report from the House of Commons Public Administration Committee after its inquiry into the Cabinet Office and FOI, focused around the controversial Clearing House unit.
The report also draws attention to evidence the committee received about ‘poor FOI administration in the Cabinet Office and across Government which appears to be inconsistent with the spirit and principles of the FOI Act’.
The committee chair and Conservative MP William Wragg said: “As FOI policy owner and coordinating department, the Cabinet Office should be championing transparency across government, but its substandard FOI handling and failure to provide basic information about the working of the coordinating body has had the opposite effect.”
The report contains some stinging rebukes for the Cabinet Office, saying that ‘transparency is needed to restore trust’. It calls for the quarterly publication of Clearing House casework data, broken down according to the referring department, and analysed for the timeliness of replies.
The committee’s MPs were clearly particularly annoyed by the fact that the Cabinet Office now releases less information about the casework of the Clearing House than was issued in the past by the Ministry of Justice when it had responsibility for FOI and the Clearing House.
The committee is also concerned about the overall approach of the government to openness. It is worried about ‘a slide away from transparency’ illustrated by the decision to exclude the new Advanced Research and Invention Agency from FOI. The MPs call on ministers to show proactive leadership on championing FOI and set a ‘stronger tone’ on promoting its benefits.
The MPs are unhappy about a split of responsibilities, in which the Cabinet Office oversees FOI policy but DCMS funds the ICO. They argue this should be resolved by either the Cabinet Office taking over the funding or DCMS being given charge of FOI policy.
The committee’s inquiry was prompted by a Tribunal case last year, arising from an FOI request made by Jenna Corderoy, a reporter for openDemocracy. This succeeded in revealing details about FOI requests passed to the Clearing House by other departments after much resistance from the government.
Last August the government said it would set up its own internal review of how the Clearing House functions. Nothing more happened until yesterday – funnily enough, the day just before the Commons committee report was to be published – when it revealed that this review will be led by Sue Langley, the lead non-executive director at the Home Office.
● Is the role of the Clearing House proportionate and effective?
● Is there sufficient information available to the public about the operation of the Clearing House?
● How is the ‘applicant-blind’ principle [that the identity of the applicant is not relevant to the handling of an FOI request] understood and adhered to across government?
● Are there other areas of FOI practice across government with scope for improvement (within the scope/line of sight of the Clearing House)?
It is expected to be completed by the summer parliamentary recess (scheduled to start on 21 July), and the Cabinet Office says it will publish a summary of the findings.
The Cabinet Office will also have to publish a formal response in due course to the report of the Commons Public Administration Committee.
I submitted written evidence to the committee, which is here, and was invited with some other journalists to give oral evidence, which is here.
From the internal briefings prepared for the new Information Commissioner John Edwards, it’s clear there has been rather a lot of angst among his senior colleagues about how he conducts his personal Twitter feed.
His account’s tone was casual and folksy when he was New Zealand’s Privacy Commissioner, combining some serious policy points with jokey remarks, funny videos and cultural criticism, while slipping into the occasional political controversy. As I said when I wrote about it before, I like its friendly informality, but I’m not at all surprised that his new cautious-minded colleagues seem deeply wary about the ‘risks’.
The induction documents recently released under FOI reveal how the ICO management assembled a battery of arguments and research to try to persuade him to adopt a more corporate approach for his tweeting.
They fear that his unofficial posts could be seen as the ICO’s organisational view and policy – perhaps they’re thinking of the time he described Facebook as ‘morally bankrupt pathological liars’.
They’re worried that his personal tweets could be ‘misconstrued’ – perhaps they saw the time he pointed out that a New Zealand opposition politician sent emails in comic sans.
They’re concerned that he could be seen as endorsing individuals or sub-tweeting – linking to a Guardian article that’s seven years old to explain to him the tricky etiquette of sub-tweeting.
They certainly don’t want him conversing with other Twitter users or getting involved in Twitter spats, recommending he should ‘reduce or limit interaction with other Twitter users’.
They warn him that ‘UK media have form of going through old tweets to try and find content that could be used critically’ and so maybe he should delete some of them, though obviously I worked through his back catalogue ages ago.
They would like him to change his Twitter bio so that it directs enquiries to @ICONews, but that hasn’t happened – yet.
And naturally they want him to focus his feed on ICO work and events, and tweet about this in a timely way. They’ve done their research and thus can draw attention to the anodyne content of the personal Twitter feeds of various other heads of UK public agencies – who are largely content to retweet their organisations’ press statements, with a little extra stuff on the wonderful work of whichever outpost they have just visited, perhaps a picture of some conference they’ve spoken at, and minimal interaction with anyone else. This familiar flow of earnest tedium is presented as ‘good practice’ in corporate social media.
Still it’s not all negative – they point out that the @ICONews Twitter account has ten times the followers that Edwards has personally, and therefore they ‘can help to grow’ his follower base.
In New Zealand they’re called BIMs – Briefings to Incoming Ministers – and it’s routine for these induction documents explaining background and policy options for the new office-holders to be published. It’s one example of how NZ outdoes the UK on government openness in policy formulation.
One point first: Anyone familiar with how FOI works in practice will note that the ICO has disclosed some internal discussion points which many UK public authorities would have sought to keep secret, claiming it would harm the free exchange of views (until the ICO overrules them several months later if the requester bothers to complain). Of course they’ve acted rightly in line with the public interest and set the correct example by releasing this material, but still credit to them for doing so. (Some information has been withheld, to protect frankness of discussion, their investigations, international relations, and legal advice.)
While much of the content is about data protection, and there’s some revealing analysis on the ICO’s overall strategy, I’ll focus on my interest, which is the FOI side of their work.
The ICO is clearly very sensitive to two common criticisms. Firstly, that it concentrates on data protection and privacy issues at the expense of FOI. Secondly, that it has failed to tackle the substantial and apparently growing delays affecting its FOI appeals casework – many complainants are now told they will have to wait nine months just for their appeal to be allocated to a case officer.
This can be seen in the briefing paper on developing FOI strategy. It argues against a simple focus on casework and enforcement with limited involvement in policy debate, largely because that would play into fears about the ICO neglecting FOI and encourage the view that FOI responsibilities should be given to another organisation.
The briefing papers also contain numerous references to the knock-on problems stemming from the ICO’s inability to stay on top of its FOI appeals caseload (which involves complaints about the handling of FOI requests by other public bodies).
The FOI strategy paper acknowledges that the delays in complaint handling ‘do understandably draw criticism’, although it argues the ICO is ‘hampered’ due to available resources. It also accepts that the ICO’s current enforcement powers are ‘arguably underused’.
A draft communications plan for the new commissioner’s ‘first 100 days’ is revealing when it expresses worry about initiating a campaign for Edwards to speak out about the ‘continued value’ of FOI. The fear is of resultant publicity risks, namely ‘poss criticism around a significant casework backlog’.
I have to confess that on this point the ICO comms team have got people like me bang to rights. When the ICO press office drew attention to Edwards talking about the benefits of FOI in a media interview, this is indeed the theme I picked up on.
This briefing also notes concern that it could open up questions of charging for FOI requests, a topic Edwards surprisingly raised at his select committee appointment hearing, to the consternation of openness campaigners. He later told me that it was an ‘off the cuff response’ linked to his experience in New Zealand. But the paper also argues this would be an ‘opportunity to set record straight and keep civil society stakeholders at bay for a while’.
Another campaign idea considered is one to ‘highlight how FOI has informed public debate’ and ‘remind people how to make FOI requests’. But the document again points out a possible downside for an organisation which can’t handle its current caseload: generating more FOI requests would ultimately put ‘increased pressure’ on the resources of the ICO.
A paper on overall strategic planning includes this for a ‘draft enduring strategic objective’: ‘We will not continue to simply grow the approach of dealing with more and more appeals with static or net reductions in grant-in-aid funding but will instead focus on encouraging public authorities to be more transparent and open, publishing more information routinely and so avoiding the need for the public to escalate appeals to the ICO.’
However, the notion that more routine publication will avoid ICO appeals is a futile and irrelevant hope. Proactive release of information is a good thing, but there is very limited overlap between the material organisations will willingly agree to publish routinely, and the contentious requests for material that requesters really want to access which reach the ICO.
What would cut ICO casework is getting authorities to adopt habits of responding on time and releasing the information that they should release, which needs tougher enforcement.
When Chris Graham became Information Commissioner in 2009, he made it his top priority to reduce the organisation’s then appalling FOI casework backlog. He felt it was necessary so that the ICO could speak with credibility and authority. By thoroughly reviewing processes and structures within existing resource levels, and bringing a spirit of determination that boosted morale and swept aside the fatalism of some staff, he succeeded.
These documents indicate how today’s ICO is constrained by its organisational failure to stay on top of its FOI casework.
There are other interesting themes to be gleaned from all the induction briefing papers, including:
A report on the backlog in the ICO’s own FOI requests (this is distinct to the backlog for complaints about other organisations), which says that by the end of June they expect to reach their target of answering 92% of requests within the legal time limit.
A paper from the ICO’s Parliament and Government Affairs department, its lobbying team, which states that one of PGA’s ‘important functions’ is to ‘facilitate any necessary engagement’ between government and the ICO’s enforcement and casework sections – which surprised me and suggests an uneasy crossover of internal roles for a regulator.
A list of what the management think the other ranks might ask Edwards at an initial all-staff Q&A session, from matters of top-level strategy to the vexed issues of car parking and will he base himself in Manchester or London;
A media guide containing the press office’s views, positive and negative, on certain journalists;
And a lot of unease about the new Commissioner’s personal Twitter feed, which merits a blog post of its own.
It’s often said that the freedom of information system should be ‘applicant-blind’. The identity of the person asking is irrelevant and should make no difference to how the FOI request is treated.
There are actually some exceptions to this (for example, if someone is just repeating the same request they submitted the day before), but as a legal principle it’s broadly correct.
In practice however public authorities often seem very keen to find out details about the people sending them FOIs.
I’ve just obtained from the Department for Business, Energy & Industrial Strategy (BEIS) some internal emails referring to an FOI request of mine in 2019. That was when I was still at the BBC, and it concerned UK reaction to past EU plans to reform the accountancy industry.
An official in BEIS’s Business Framework Directorate dealing with the request emailed colleagues asking them “to provide any information about this Martin Rosenbaum, previous contact with BEIS etc”.
The FOI team replied: “He is a BBC journalist with interest in politics & current affairs and is a specialist in freedom of information & data. He has been sending in requests from 2010, so he is quite a regular requestor.”
The information about me seems to have been intended for inclusion in the submission to the Deputy Director who had to sign off the FOI response. The eventual submission stated: “Martin Rosenbaum seems to be one of the main representatives of the BBC responsible for obtaining information from BEIS via FOI requests. There is no particular pattern in the subject matter of his requests.” And then it listed the topics of several of my previous requests.
Since BEIS had failed to respond to my request on time, I had sent a chasing email. An FOI unit official then emailed the Business Frameworks team to say:
“I know that we are in the process of opening up these files etc, but I just want to make you aware that the requestor has now written in to complain as he is still waiting for his response and we haven’t even completed a first draft let alone taken this up to be cleared by press office and SPAD … I would like to remind you that Martin Rosenbaum is a journalist – also a frequent FOI requestor, and is aware of the process and complaints procedure, so we really need to try and pull our socks up with this.”
The moral of this story for FOI requesters? If in practice FOI doesn’t operate in an applicant-blind manner, it’s good to have a reputation for knowing when and how to complain and being willing to do so.
The information law regulator has reached an agreement with itself to hold a regular series of meetings with itself, so that its staff can check up on how its staff are doing on answering freedom of information requests.
One of the tasks of the Information Commissioner’s Office is to try to improve the performance of public authorities with a particularly bad track record on dealing with FOI applications, such as … the Information Commissioner’s Office.
Last month the ICO revealed it would writing to the, um, ICO, to find out about the ICO’s plans to do better in future and so avoid being further criticised by, yes, the ICO.
This has now happened, and in an unusually speedy response to my FOI request, the ICO has sent me the correspondence.
It reveals that in the first nine months of the current financial year, the ICO only responded to 71% of FOI requests on time. At Christmas there were three requests that were over a year overdue.
The group manager for complaints wrote to the head of corporate planning to say: “We cannot ignore the fact that we have had an increasing number of complaints reported to us about the ICO in relation to late compliance with information requests. Adverse media coverage and blogs have also reported on these issues … We are therefore requesting a formal update on your recovery plans”.
The reply says “We recognise our performance is not where it should be”.
The recovery plan has now been published on the ICO website. It says they are recruiting and reallocating staff, with the aim of ensuring that 90% of FOI requests are responded to on time by the end of June, while requests over a year overdue should be cleared by the end of February.
This is all to do with the FOI requests which the ICO itself receives, not the complaints it assesses about the processing of FOI requests sent to other public bodies. That side of the ICO’s operations is also affected by serious delays.
The disclosed correspondence shows that the complaints team will meet monthly with the information access team to review progress. All those who comment agree that the ICO should treat itself in the same way it would treat any other public authority with an inadequate record of compliance with FOI law.
It does seem somewhat absurd, but it is nevertheless better than alternative arrangements – such as the ICO not being subject to FOI requests at all, or not falling under the oversight of an information rights regulator, or being let off the hook by its own staff rather than publicly and embarrassingly rebuked.
On the other hand, this process wouldn’t be necessary if the ICO actually responded promptly and efficiently to information requests.
While this request was responded to quickly, another of my FOI requests has not had a reply from the ICO nearly three months after it was submitted.
For a couple of years in an earlier phase of my life I gambled on politics. And I made money by doing so.
It all began with a Tory leadership election, and what turned out to be the sadly erroneous views of ITN’s then political editor. That was 1995.
I stopped after I joined the BBC in 1998, since it could have created a conflict of interest – which also prevented me taking up a consultancy role I was offered by the betting company Sporting Index to advise on political bets.
So it wasn’t a long phase, but the benefits to me were not only financial – I learnt life lessons from gambling.
Dominic Cummings and “Winning Against the Odds”
Betting involves taking specific decisions which have plainly identifiable consequences, sometimes very soon. You get a reality check on your opinions and mode of thinking. This means it should provide good opportunities for clear ‘feedback’, both positive and negative, for learning and improving.
I largely focused on spread betting, where decisions could easily have significant financial consequences, for good or ill. I wasn’t betting for ‘fun’, or to buy extra interest or excitement in events, or to hedge or reinforce my emotional reaction to what happened politically, all of which could be rational reasons for risking (or squandering) a few pounds. My aim was purely to win money.
(An explainer on spread betting, for those who want it: Suppose for example a betting company offers a ‘spread’ of 340-350 for the number of seats the Tories will win at the next election. If you think they’ll get more, you can ‘buy’ at 350 at a stipulated stake per seat, say £20. If the Tories then get eg 375 seats you’d win (375-350)x20 = £500; but if they got eg 335 seats you’d lose (350-335)x20 = £300. On the other hand, if you were predicting they’d get 335, you could ‘sell’ at 340 and if right in due course make (340-335)x20 = £100; but if they got 375 you’d lose (375-340)x20 = £700.)
Lesson 1 – It’s in the detail
I became intrigued in betting when John Major, then a beleaguered prime minister, told his many internal party critics to “put up or shut up”, and John Redwood went for the “put up” option. Which led to the 1995 Tory leadership contest.
I was watching the ITN lunchtime news on the day of the ballot, and its political editor Michael Brunson said Redwood would get about 45 votes. I thought “If it’s good enough for Michael Brunson, it’s good enough for me”, and phoned a spread betting company. Later that day it was announced Redwood got 89 votes and I lost £150.
Yet what I gained was the insight that money could be made – but it needed my own careful analysis, not a reliance on the views of others.
And this meant: no general impressions, no broad judgments – that’s more or less plucking figures out of thin air. Instead break the factors of a situation (eg the Conservative parliamentary party) down into component parts and look for whatever detailed evidence exists.
In due course in the 1997 Tory leadership election I made over 20 times what I’d lost on the 1995 one.
Lesson 2– Opinions and decisions aren’t the same
I learnt how big the difference is between holding an opinion and actually being willing to stake your money on it. When I was deciding whether to place a bet, and I forced myself to think through properly whether the evidence really supported a view I held, I then realised that maybe it didn’t. And this sometimes applied even when I’d previously been enthusiastically advocating that viewpoint in conversation with others.
There was no need to bet on an outcome just because I’d confidently told everybody that it was what I expected to happen. I learnt a healthy disrespect for my own judgment. It’s fine to enjoy a good argument if you want, but when it comes to decision-making with real consequences it might be better to be non-committal.
Lesson 3– The status quo is always changing
There’s a new status quo with each decision you take.
Sometimes it was possible to guarantee smallish wins by what is called arbitrage. When on some event two betting companies had different spreads which did not overlap, and you spotted it before one of them closed the gap, you could guarantee profit by selling with one and buying with another.
For example: If for Labour seats at the next election company A quoted 320-330 and company B quoted 333-343, you could buy with A at 330 and sell with B at 333 – and if you did that at say £10 per seat, you’d make £30 for sure, irrespective of the actual outcome.
Sometimes arbitrage opportunities were implied rather than being so explicit, and the bookies might not spot and stop these so readily. I can remember in the 1997 general election there were spreads available on the number of Labour MPs and on the number of women MPs which were way out of line with each other, given how many Labour candidates in marginals were women and therefore how tightly the two spreads should have moved together.
This was before online gambling, so you had to make phone calls to place the bets, which took a little time. I reckoned the safest course of action for arbitrage was first to place the bet which represented value in my way of thinking; then place the counterpart bet assuming it was still available.
But once I had placed the first bet I was in a new situation, and in fact one I was quite happy with – I was sitting on a bet that I felt at the time was good value. Why on earth would I want to now go and place another bet that to my mind was probably chucking money away? Well, I didn’t, so I now realised. I went through this process a couple of times and concluded this kind of arbitrage didn’t work for me (it would of course be different in another field where I was ignorant and had no idea which bet actually represented good value).
In other words I’d planned a multi-step strategy based on the situation I was in initially. But once I’d taken the first step, I was in a new situation where the rest of that strategy no longer made sense.
Lesson 4 – Going against the herd, when the herd is wrong
Success can come from spotting when the herd is wrong, when everyone else is heading in the wrong direction – in gambling and in life. You can make money and the bookies can make money, as long as other punters are losing.
I realised that one aspect of the bookies’ talents was to price bets not always in line with predicted outcomes but so that people would go for them.
Shortly before a budget someone from a spread company told me they were thinking of opening a market on how often the chancellor would be interrupted during his speech and asked what I thought. I informed him it would almost certainly be none at all, since that was the established norm for a budget. I was then surprised to see they put up a spread of 2-4 interruptions.
When I spoke to him a few days after the budget, I asked in puzzlement why they’d done this, when people who knew parliamentary convention could take money off them by selling at 2. He explained that it had worked really well, they’d had few sellers at 2 but lots of people buying at 4 who therefore lost money – including a minor TV celebrity of the time who had bought at £1,000 per interruption, losing £4,000.
Lesson 5 – Life looks leptokurtic
Life is uncertain, and genuinely unpredictable events happen. You could lose money on decisions which on the basis of the information you had at the time were probably correct. And sometimes I thought I did. Equally well, you could win on the basis of decisions which actually were probably wrong. Doubtless I did that too.
Perhaps this is particularly applicable to politics. According to a recent data journalism newsletter from the Economist magazine (it’s called ‘Off the Charts’, it’s always interesting and I’d recommend signing up for it), forecasting accuracy from a range of sources is worse about politics than for other human endeavours such as sport or culture.
Maybe – I’m not sure if that’s true or not. But I did increasingly form the view that the probability distribution of human events was fat-tailed. Weird things happened, more often than they should. In statistical terms, there’s a case for saying that life is leptokurtic.
As they say, the race is not always to the swift, and while logic may suggest that’s where to put your money, it’s funny how often in politics it turns out that the tortoise beats the hare.
