The big idea: should we be thinking about luck differently?

At around midday on 19 August 1949, wreathed in thick mist, a British European Airways DC-3 going from Belfast to Manchester flew into a hillside on Saddleworth Moor in the Peak District, near Oldham. All the crew and 21 of the 29 passengers died on impact or soon afterwards. Eight passengers survived, including a young boy and his parents, although, devastatingly, their younger child was one of the fatalities. That surviving boy became my friend and statistical colleague, Prof Stephen Evans.

I think we would agree that Stephen was lucky. But what do we mean by “luck”? We might say that someone has been lucky, or unlucky, if they have benefited or been harmed by something that was unpredictable and beyond their control. Luck has been called “the operation of chance, taken personally”.

Luck comes in three main flavours. Philosophers have identified “circumstantial luck”, meaning being in the right place at the right time, or the wrong place at the wrong time – such as Stephen’s family taking that particular flight. Then there’s “resultant or outcome luck”, where in a particular situation some people have good and some have bad outcomes due to factors beyond their control. Stephen had the good resultant luck of surviving.

But perhaps the most important is “constitutive luck”, which covers all the fortunate or unfortunate circumstances of your very existence; the period of history in which you were born, your parents, background, genes and character traits. So where was Stephen’s constitutive luck? He told me that his father’s experiences in the RAF led him to insist that the family sat at the back of the plane – and the only survivors were seated at the back. He had the right parents.

In another remarkable example, on Christmas Eve 1971, 17-year-old Juliane Koepcke had the bad circumstantial luck of being on Lansa flight 508 when it was struck by lightning over the Amazon jungle. She was thrown out, still strapped into her seat, and fell 3,000 metres (9,800ft). She had, however, amazing resultant luck when the thick jungle canopy broke her fall and she survived, although 90 other people, including her mother, died. And, just like Stephen, Juliane had constitutive luck. Her parents, who were ornithological researchers, had brought her up in the Amazon, and she had the necessary skills not only to look after her wounds, but also to journey for 11 days in the jungle until she found an encampment.

In the political arena, leaders feel the need to express complete confidence in the effects of their actions, but if they recognised the major role of chance in what happens, they might try to nurture more resilience to the unexpected. Such humility can also be valuable at a personal level. When attributing reasons for any success they have, people tend to overestimate the role of their efforts and acquired skills, whereas they should mainly be grateful for their constitutive luck.

I consider I have had great constitutive luck regarding the time, place and family into which I was born. But that presupposes the existential luck of being here at all. All of us have arisen through an extraordinary sequence of chance events that could easily have turned out very differently.

My grandfather had the bad constitutive luck to be just the right age to join up at the start of the first world war, and he ended up as a gas officer for the British 104th Brigade, inspecting gas defences and weaponry north of the village of Passchendaelein the Ypres region of Belgium on the western front. He travelled roads and trenches that had been targeted by German artillery, and in the six weeks he was in post, his diary variously recorded: “Narrow escape on return journey”, “Lucky to get through in time”, “Artillery strafe”. On 29 January 1918, his circumstantial luck inevitably ran out. Injured, he was taken to No 64 casualty clearing station.

But his resultant luck did him a favour. He was categorised as unfit for frontline duties and spent the rest of the war well behind the lines. Meanwhile, his old battalion, the 18th Lancashire Fusiliers, was moved to the Somme in northern France, which was supposed to be a quiet area, a respite from the battles of 1916 that led to around 1 million casualties. They arrived just in time to face the massive spring offensive of 1918, fighting a desperate rearguard action and then going over the top in vain attempts to recapture territory.

Of course, if one of those shells had fallen a bit closer, or if my grandfather had had to lead his men into attack, I would not be here to tell the story. And this is only one of the long chain of fortuitous events that led to my existence: my mother being captured by pirates in China and later escaping Shanghai in 1937 under shellfire; my parents meeting during the war; my father closely avoiding plane crashes in the RAF and then nearly dying of tuberculosis. Then, when a cold snap hit the UK in November 1952, they were living in a barely heated stone cottage, with no television and nothing to do but go to bed early to keep warm … and here I am.

We can go even further in considering our existential luck. Humanity would probably not exist if an asteroid had not happened to hit the Earth 66m years ago, altering the climate and wiping out the non-bird dinosaurs, allowing mammals to develop. Life on Earth would not have had the chance to emerge if our planet had not been fortuitously stable over billions of years. Our entire universe only exists because of the “cosmic luck” of physical constants being in a “Goldilocks” zone, and the ratio of matter to antimatter allowing things to coalesce after the big bang.

Of course, we wouldn’t be here to muse on our good fortune if this chain of events had not occurred, and this “survivorship bias” means that it is philosophically challenging and, I believe, rather pointless, to talk about the probability of our existence. But perhaps the fragility of this chain should induce some humility about our self-importance.

In short, once we properly acknowledge our existential and constitutive luck, we are left with a daunting but hopeful task: to try to make the best of the hand we’ve been dealt.

• Sir David Spiegelhalter is emeritus professor of statistics at the University of Cambridge and author of The Art of Uncertainty: How to Navigate Chance, Ignorance, Risk and Luck (Pelican). To support the Guardian and Observer, order your copy at guardianbookshop.com. Delivery charges may apply.

Further reading

Fluke: The Maths and Myths of Coincidences by Joseph Mazur (John Murray, £20)

On the Edge: The Art of Risking Everything by Nate Silver (Allen Lane, £30)

The Mathematics of Love by Hannah Fry (Simon & Schuster, £7.99)