An ODE to Judea Pearl, Causality and Bayesian Networks

Introduction

On this anniversary of 9/11, I would like to offer a blog in the form of a tribute to an extremely courageous and brilliant scientist, but more importantly a great human being known, as Judea Pearl, whose work has pioneered the science of Causality and AI.

Judea Pearl epitomizes great courage and humaneness, because he lost his son – Daniel Pearl, a journalist working for the Wall Street, who was kidnapped and killed in Pakistan by Al Qaeda and the Islamic Front for being a Jew and an American. And yet, Judea has been working for reconciliation between Jews and Muslims ever since – when asked why, he has replied – “Hate killed my son. Therefor I am determined to fight Hate”.

Over the last couple of weeks, I have been privileged to read a book titled ‘The Book of Why’ written by Judea Pearl and Dana Mackenzie, as a part of my journey into researching and writing at ISB.

The book is simply magnificent – through a series of intense narratives, historical anecdotes, and of course some complex math, Judea Pearl illuminates many a path towards the evolution of AI and causal sciences. Pearl speaks of his personal journey on ‘rescuing’ subjective and priori knowledge from the positivists, and more importantly the notion of causation from the collective disdain if not taboos held by statisticians world over in the debate between causality and correlation.

Pearl is one of the pioneers of Bayesian networks and perhaps the first to mathematize causal modelling and is a giant in the field of AI. He teaches and researches at UCLA, and has worked with causality even in the domain of psychology and social sciences.

Many of us who graduated in 1980s and 1990s would have missed out on Causation as the new science and definitely the link between Bayes Theorem, Bayesian networks and Artificial Intelligence. I would recommend buying and reading this book – it is a tough read but the hard work in imbibing insights also makes it relishing and gratifying. I do not intend to summarize the book – it is a near impossible task but write to awaken your curiosity.

 

Part 1

Causality & Social Sciences

As a kid, I had always been intrigued by Causality in the human society. My favourite character was Isaac Asimov’s Hari Seldon – a professor of mathematics who develops ‘psychohistory’ – an algorithmic science, to predict the future of a galactic empire, in terms of probability. I devoured the five Foundation books by Asimov at the impressionable age of 12, and today as I near the threshold of 50, I find that this fiction character has been labelled as a ‘paradigmatic figure’ in Big Data and Epidemiology. If nothing else, this has affirmed my taste in science fiction… By the way, the GoT fans ought to know that Hari Seldon gets the nickname of ‘Raven’ in the books for being able to predict the future.

My need to understand causality and human society has guided me to work with process work, group relations and group dynamics, and unconscious as a construct that impacts one’s choice making and meaning making.

Keeping these nostalgic trivia aside, let me come back to Judea Pearl.

He traces the origins of Causal Science as far as the 18^th century, and then narrates how it got denied its legitimacy by the rise of statistics for the next 100 plus years. Pearl maintains that Statistics, as it is frequently practiced, actually ends up discourages scientific thinking and encourages routine calculations on data. R A Fisher, the God of statistics if not the high priest of statistics, had written in 1925 that “Statistics may be regarded as … the study of methods of ‘reduction’ of data”.

Pearl questions this propensity to be a reductionist, and chained to methods of statisticians, and underlines the necessity of going back to ‘causation’ and probabilistic sciences. He underlines that Causation requires the thinker to take risks, and offer subjective if not intuitive insights on why things happen … it is immensely risky for these Prior Belief (priori hypotheses of causality) can be laughed at or critiqued … but this is the only way forward. Popper’s falsification of theory thus becomes a philosophical leg to stand on.

Pearl builds a ‘Ladder of Causation’, and maintains that present day learning machines /computing is still stuck at rung 1.

Rung 1 is termed as Seeing, and is all about working with statistical tools of correlation and regression. For example, “if a customer buys custard powder, how likely is she or he wanting to buy jelly crystals?”. In Causal sciences, this is stated as a conditional probability = P (Jelly crystals| Custard powder), where the vertical line means – “given that you see”. A similar question would be constructed for customer choices such as the Probability of buying (Apple earpods | Apple Iphone). The solutions to these questions can be answered through Observation (Seeing). Lots of data over years and over for example, Apple showrooms, we can emerge with correlations and well it takes high school math to figure out the probability. This is the stage where Big Data is stuck…

However Pearl constructs a third rung of Causation – terming it as ‘Imagining’ – and defining it as human capability to ask “what would happen if things had been different?”.

So going by the abovementioned example – What is the probability of the Customer buying earpods if we had doubled the price of earpods?. This is termed as a Counterfactual question for it cannot be resolved through Seeing, but by establishing a causality. Asking counterfactual questions make demands on our ability to ‘imagine’ and emerge with a subjective causal map. And this is where Pearl endorses the Human capability to ‘imagine’ and ask the right counterfactual questions … and where he says that machines include AI has not really reached.

I am appending the Ladder of Causation – a drawing by Maayan Harel, that maps the three rungs of causality.

 

Part 2

Bayesian Networks

 

The other key strand that Pearl traces is the Bayesian connection between Objectivity and Subjectivity. Unlike correlation and other tools of mainstream statistics, causal analysis demands a ‘subjective commitment’. He states that “One grain of wise subjectivity tells us more about the real world than any amount of objectivity”.

When I read this statement, I felt immensely unburdened for I was really drowning in the cesspool of objectivity for objectivity sake.

To link subjectivity to objectivity, Pearl resurrects the Bayes Theorem which simply put states that Prior Belief PLUS New Evidence gives rise to Revised Belief. Thomas Bayes coined this formula in 1750, focusing his energies on the probability of two events – where the hypothesis occurs before the evidence – this is known as the concept of inverse probability.

Bayesian inverse probability is extremely interesting to answer following questions:

1. What is the probability of the Customer buying Custard powder if she has already bought Jelly Crystals
2. What is the probability of the Customer buying an iPhone if she or he has bought Pods?

Now the notion of inverse probability, as opposed to forward probability can be quite challenging for calculating it and making decisions – it requires one to state and update one’s belief (subjectivity), and this is the new science!

Pearl then goes on to leverage the rule of inverse probability to design of causal networks – including Chains, Forks, and Colliders – these three designs take us from Rung 1 to Rung 2, and lay down a foundation to work with Causality.

Bayesian Networks are a mature technology – there are off the shelf network software available and a part of many smart devices today. The modern smartphone and telecom networks use it as error correction algorithms to enhance connectivity and voice clarity.

However, Pearl links Bayesian analysis and networks to work with confounders in causal relationships and this is why the book is a must read!

 

Conclusion

If you have reached this far, then the conditional probability of you buying the book has really gone up.

I don’t do book summaries in my blogs, but this one has been worth my time in as an ode to human brilliance and courage. I salute men like Judea Pearl, who demonstrate these qualities that build hope for the humankind as we evolve through the messiness of change and transformation.