Å·±¦ÓéÀÖ

Failed revolution

In an old joke, an engineer, a physicist and an economist are marooned on a desert island with canned food. They are trying to figure out the best way to open the cans, and while the engineer and the physicist propose various mechanical schemes to get the job done, the economist says, "Let's assume we have a can opener..." Judea Pearl's approach to causal inference brings that joke to mind. His causal calculus begins with the premise, "Let's assume we have a strong causal theory." He shows that once you know the causal relations (or lack thereof) between relevant variables, you can use graphical methods to work out how to estimate the values of the parameters you are interested in. But this puts the cart before the horse.

In The Book of Why and many earlier publications, Pearl promotes his extension of probability calculus and nonparametric structural equation models (directed acyclic graphs or DAGs) as the solution to the problem of inferring causes from observational data. He describes his approach with such terms as "revolutionary" and "miraculous." I have personally found traditional path or SEM models useful when thinking about some causal problem, but I see little new that is worthwhile in Pearl's approach. Graphical models can be useful tools in research, but there's nothing revolutionary about them; they are not a royal road to causality. If you have good subject knowledge about a research topic, you will have an understanding of the dependencies between relevant variables and you can use graphical methods as one of the tools to clarify the implications of the model, but that's really it.

What is missing from Pearl's book is very telling. Almost all the "practical" examples in the book concern either problems that were long ago solved with methods other than Pearl's (e.g. smoking causing lung cancer) or else are toy problems where the underlying causal model is presumed known. Pearl triumphantly shows how his approach "solves" these non-problems. He delights in how he can routinize the procedures for solving various inference problems when the causal structure is known with certainty. I must say that I find his explanations of well-known paradoxes less illuminating than more traditional treatments. For example, tabular data on the distributions of values for different groups usually makes Simpson's paradox entirely transparent, obviating the need for a more formal treatment. More importantly, Pearl's methods do nothing to help discover whether your data are actually confounded by something like Simpson's paradox.

A prominent early airing of Pearl's ideas about causality took place in the journal Biometrika in 1995. Pearl's target paper published in the journal contained all the main elements of his current framework. The paper was accompanied by a number of expert commentaries, and a common theme in many of them was, aside from polite comments about the technical elegance of the approach, a skepticism that a method like this could help achieve concrete, real-life advances in scientific understanding. The 1995 article and the 2018 book are separated by 23 years, and in the meantime Pearl has published two editions of his textbook on causal inference, but there is still a total paucity of real-world benefits stemming from Pearl's program. The implications of this state of affairs for Pearl's "causal revolution" are devastating, but he seems to be blind to the enormous gap between his pompous pronouncements and the reality.

If Pearl's claims about the revolutionary impact of his theory were correct, we would now be living in a golden age of science. In reality, however, there is currently a crisis of confidence in science, across many fields. Not only has the increasing utilization of Pearl's approach not done anything to prevent the crisis. It's also that Pearl completely failed to anticipate the actual problems that many fields of research are facing. No amount of DAGs will solve such problems as small sample sizes, selective reporting of analyses, lack of replication, weak theories and poor measurement.

Pearl assumes that researchers can make use of a reliable body of "background knowledge" when drawing their causal diagrams. If the relevant variables and their causal relations are presumed known, the estimation of causal parameters becomes straightforward. This shows an undue optimism on his part. The fact is that, as has become ever more obvious in recent years, many fields of research not only lack convincing theories to make sense of data and experiments, but there is also pervasive uncertainty about the basic reliability of large swaths of observational and experimental data reported in the published literature. All scientific knowledge ultimately relies on at least some uncertain assumptions, but that does not mean that you can freely make any old assumption when making causal inferences. If the "background knowledge" you have is just a muddle of unconnected, incompatible and unreplicated findings and hypotheses, Pearl's method is not useful.

You might protest here that even if Pearl's methods are impractical in immature fields, that's not the case when the background knowledge available is more solid. However, Pearl does not even try to argue that his methods are needed in the mature, physical sciences. Researchers in mature fields already know how to make causal inferences and have no need for Pearl's insights. If you have a strong theory and good data, it is usually the case that a simple regression or the like will give you the causal estimates you want. In such cases, nothing is gained by re-expressing a problem and its solution in terms of Pearl's formalisms. If you have a strong theory, Pearl's methods offer nothing to you, whereas if your theory is weak, as is by and large the case in social science, DAGs will only give a false causal veneer to associations whose nature is unclear.

Therefore, Pearl's "revolution" amounts to introducing canned procedures for dealing with some relatively trivial subproblems that a research program may contain and that a mature discipline knows how to deal with anyway. It does nothing to help tackle the hard problems of research, so it is unsurprising that the (sporadic) adoption of these methods has not coincided with any scientific progress.

Pearl's program is a failed attempt to replace the messy, trial-and-error process of scientific discovery with clean mathematical and logical formalisms. It is an overreach of rationalism by a mathematically inclined theoretician with little experience dealing with the jumbled reality of the "soft" sciences.

With all that said, I nevertheless think The Book of Why is a decent read. In particular, the historical chapters are interesting (even if sometimes tendentious) and a non-expert reader will get the gist of the method from the many examples (even if some of the exposition is overly technical). Just don't expect to find anything revolutionary in it.