Scotland’s alphabet effect

Last week’s local election results appear to confirm how a candidate’s chance of getting elected to Scotland’s councils is dramatically influenced by a factor which is nothing to do with their abilities – alphabetical order of surnames.

This arises from the voting system used for Scottish council elections, the Single Transferable Vote (STV), where voters number candidates in their order of preference.

Parties will stand more than one candidate in a multi-member ward if they think they have a chance of getting more than one elected.

But of course lots of voters, who may have strong preferences between the parties, don’t particularly care about preferring one candidate from within a party to another.

It’s well established that under STV many voters have a tendency to number candidates from the same party just in the order they find them on the ballot paper, which is a major advantage for those listed first. In Scotland that is alphabetical order by surname.

To illustrate the striking extent of this I have looked at what happened last week in two Scottish councils, Aberdeen and West Lothian (the first and last councils alphabetically, in a limited attempt to avoid alphabetical bias in my selection).

I examined all the cases in these two councils where a party stood two or more candidates in one ward.

In West Lothian, there were 14 examples. In 13 of these, the candidate who came first alphabetically from that party got more first preference votes than the candidate listed second alphabetically, sometimes by huge margins.

The candidates listed first alphabetically for a party averaged 1,669 first preference votes; the candidates from the same party listed second alphabetically only averaged 745 first preferences – less than half as much.

The result was that the candidates listed first alphabetically for a party had a 100% success rate at getting elected; the candidates from the same party listed second alphabetically only had a 64% success rate of election.

In Aberdeen, there were 16 examples. In 14 of these, the candidate who came first alphabetically from that party got more first preference votes than the candidate listed second alphabetically, again sometimes by huge margins.

The candidates listed first alphabetically for a party averaged 1,223 first preference votes; the candidates from the same party listed second alphabetically only averaged 554 first preferences – again, less than half as much.

The result here was that the candidates listed first alphabetically for a party had an 88% success rate at getting elected; the candidates from the same party listed second alphabetically only had a 56% success rate of election.

Obviously it would be ideal to do this analysis for all the 32 local authorities in Scotland. But given the different locations and formats in which all the results are published, that would be a very laborious exercise which is too time-consuming for me to do right now. If there was one single national database of all Scottish local election results in a convenient format for exporting data then it would be a lot more feasible! (I also haven’t examined the impact in the very different political circumstances of Northern Ireland).

It seems clear that the current position in Scotland represents a form of institutionalised systemic discrimination. A council seat is often a step towards building a powerful political career on a bigger stage.

In the past the Scottish government has considered various means of ameliorating this situation but has not implemented any change. Potential options would include randomising the ballot paper order or listing candidates in reverse alphabetical order on half the ballot papers.

Parties could counteract the effect if they had loyal, disciplined voters who would order candidates as instructed, with different instructions issued to different subsets of voters. Roughly equalising the number of first preferences would help to get more than one of their candidates elected.

There has been some evidence of alphabetical voting affecting results in English and Welsh elections, but this is to a much lesser extent because of the different voting systems. Alphabetical voting is also an international phenomenon.

And alphabetical bias also exists in other contexts – here’s an interesting paper on its impact in an academic discipline where co-authors of papers were listed alphabetically.

By the way, when drafting this piece I noticed I had automatically defaulted to providing the Aberdeen data before that for West Lothian, so I went back and reversed that. But I did leave Aberdeen first in the chart.

The acceptance of alphabetical order as an apparently natural and unproblematic method may have a deeper and more insidious grip on our minds, and more important consequences, than we may consciously realise.

Scotland’s alphabet effect Read More »

What I learnt from betting on politics

Starting with Tory leadership elections …

For a couple of years in an earlier phase of my life I gambled on politics. And I made money by doing so.

It all began with a Tory leadership election, and what turned out to be the sadly erroneous views of ITN’s then political editor. That was 1995.

I stopped after I joined the BBC in 1998, since it could have created a conflict of interest – which also prevented me taking up a consultancy role I was offered by the betting company Sporting Index to advise on political bets.

So it wasn’t a long phase, but the benefits to me were not only financial – I learnt life lessons from gambling.

Dominic Cummings and “Winning Against the Odds”

Betting involves taking specific decisions which have plainly identifiable consequences, sometimes very soon. You get a reality check on your opinions and mode of thinking. This means it should provide good opportunities for clear ‘feedback’, both positive and negative, for learning and improving.

I largely focused on spread betting, where decisions could easily have significant financial consequences, for good or ill. I wasn’t betting for ‘fun’, or to buy extra interest or excitement in events, or to hedge or reinforce my emotional reaction to what happened politically, all of which could be rational reasons for risking (or squandering) a few pounds. My aim was purely to win money.

(An explainer on spread betting, for those who want it: Suppose for example a betting company offers a ‘spread’ of 340-350 for the number of seats the Tories will win at the next election. If you think they’ll get more, you can ‘buy’ at 350 at a stipulated stake per seat, say £20. If the Tories then get eg 375 seats you’d win (375-350)x20 = £500; but if they got eg 335 seats you’d lose (350-335)x20 = £300. On the other hand, if you were predicting they’d get 335, you could ‘sell’ at 340 and if right in due course make (340-335)x20 = £100; but if they got 375 you’d lose (375-340)x20 = £700.)

Lesson 1 – It’s in the detail

I became intrigued in betting when John Major, then a beleaguered prime minister, told his many internal party critics to “put up or shut up”, and John Redwood went for the “put up” option. Which led to the 1995 Tory leadership contest.

I was watching the ITN lunchtime news on the day of the ballot, and its political editor Michael Brunson said Redwood would get about 45 votes. I thought “If it’s good enough for Michael Brunson, it’s good enough for me”, and phoned a spread betting company. Later that day it was announced Redwood got 89 votes and I lost £150.

Yet what I gained was the insight that money could be made – but it needed my own careful analysis, not a reliance on the views of others.

And this meant: no general impressions, no broad judgments – that’s more or less plucking figures out of thin air. Instead break the factors of a situation (eg the Conservative parliamentary party) down into component parts and look for whatever detailed evidence exists.

In due course in the 1997 Tory leadership election I made over 20 times what I’d lost on the 1995 one.

Lesson 2 – Opinions and decisions aren’t the same

I learnt how big the difference is between holding an opinion and actually being willing to stake your money on it. When I was deciding whether to place a bet, and I forced myself to think through properly whether the evidence really supported a view I held, I then realised that maybe it didn’t. And this sometimes applied even when I’d previously been enthusiastically advocating that viewpoint in conversation with others.

There was no need to bet on an outcome just because I’d confidently told everybody that it was what I expected to happen. I learnt a healthy disrespect for my own judgment. It’s fine to enjoy a good argument if you want, but when it comes to decision-making with real consequences it might be better to be non-committal.

Lesson 3 – The status quo is always changing

There’s a new status quo with each decision you take.

Sometimes it was possible to guarantee smallish wins by what is called arbitrage. When on some event two betting companies had different spreads which did not overlap, and you spotted it before one of them closed the gap, you could guarantee profit by selling with one and buying with another.

For example: If for Labour seats at the next election company A quoted 320-330 and company B quoted 333-343, you could buy with A at 330 and sell with B at 333 – and if you did that at say £10 per seat, you’d make £30 for sure, irrespective of the actual outcome.

Sometimes arbitrage opportunities were implied rather than being so explicit, and the bookies might not spot and stop these so readily. I can remember in the 1997 general election there were spreads available on the number of Labour MPs and on the number of women MPs which were way out of line with each other, given how many Labour candidates in marginals were women and therefore how tightly the two spreads should have moved together.

This was before online gambling, so you had to make phone calls to place the bets, which took a little time. I reckoned the safest course of action for arbitrage was first to place the bet which represented value in my way of thinking; then place the counterpart bet assuming it was still available.

But once I had placed the first bet I was in a new situation, and in fact one I was quite happy with – I was sitting on a bet that I felt at the time was good value. Why on earth would I want to now go and place another bet that to my mind was probably chucking money away? Well, I didn’t, so I now realised. I went through this process a couple of times and concluded this kind of arbitrage didn’t work for me (it would of course be different in another field where I was ignorant and had no idea which bet actually represented good value).

In other words I’d planned a multi-step strategy based on the situation I was in initially. But once I’d taken the first step, I was in a new situation where the rest of that strategy no longer made sense.

Lesson 4 – Going against the herd, when the herd is wrong

Success can come from spotting when the herd is wrong, when everyone else is heading in the wrong direction – in gambling and in life. You can make money and the bookies can make money, as long as other punters are losing.

I realised that one aspect of the bookies’ talents was to price bets not always in line with predicted outcomes but so that people would go for them.

Shortly before a budget someone from a spread company told me they were thinking of opening a market on how often the chancellor would be interrupted during his speech and asked what I thought. I informed him it would almost certainly be none at all, since that was the established norm for a budget. I was then surprised to see they put up a spread of 2-4 interruptions.

When I spoke to him a few days after the budget, I asked in puzzlement why they’d done this, when people who knew parliamentary convention could take money off them by selling at 2. He explained that it had worked really well, they’d had few sellers at 2 but lots of people buying at 4 who therefore lost money – including a minor TV celebrity of the time who had bought at £1,000 per interruption, losing £4,000.

Lesson 5 – Life looks leptokurtic

Life is uncertain, and genuinely unpredictable events happen. You could lose money on decisions which on the basis of the information you had at the time were probably correct. And sometimes I thought I did. Equally well, you could win on the basis of decisions which actually were probably wrong. Doubtless I did that too.

Perhaps this is particularly applicable to politics. According to a recent data journalism newsletter from the Economist magazine (it’s called ‘Off the Charts’, it’s always interesting and I’d recommend signing up for it), forecasting accuracy from a range of sources is worse about politics than for other human endeavours such as sport or culture.

Maybe – I’m not sure if that’s true or not. But I did increasingly form the view that the probability distribution of human events was fat-tailed. Weird things happened, more often than they should. In statistical terms, there’s a case for saying that life is leptokurtic.

As they say, the race is not always to the swift, and while logic may suggest that’s where to put your money, it’s funny how often in politics it turns out that the tortoise beats the hare.

It’s a case of “winning against the odds”.

What I learnt from betting on politics Read More »