Theoretical Assumptions and Nonobserved Facts

Wassily Leonteif

Economics today rides the crest of intellectual respectability and popular acclaim. The serious attention with which our pronouncements are received by the general public, hard-bitten politicians, and even skeptical businessmen is second only to that which was given to physicists and space experts a few years ago when the round trip to the moon seemed to be our only true national goal. The flow of learned articles, monographs, and textbooks is swelling like a tidal wave; Econometrica, the leading journal in the field of mathematical economics, has just stepped up its publication schedule from four to six issues per annum. (Leontief 1971, 1)

And yet an uneasy feeling about the present state of our discipline has been growing in some of us who have watched its unprecedented development over the last three decades. This concern seems to be shared even by those who are themselves contributing successfully to the present boom. They play the game with professional skill but have serious doubts about its rules. (Leontief 1971, 1)

Much of current academic teaching and research has been criticized for its lack of relevance, that is, of immediate practical impact. In a nearly instant response to this criticism, research projects, seminars and undergraduate courses have been set up on poverty, on city and small town slums, on pure water and fresh air. In an almost Pavlovian reflex, whenever a new complaint is raised, President Nixon appoints a commission and the university announces a new course. Far be it from me to argue that the fire should not shifted when the target moves. The trouble is caused, however, not by an inadequate selection of targets, but rather by our inability to hit squarely any one of them. The uneasiness of which I spoke before is caused not by the irrelevance of the particular problems to which present day economists address their efforts, but rather by the palpable inadequacy of the scientific means with which they try to solve them. (Leontief 1971, 1)

If this simply were a sign of an overly high aspiration level of a fast developing discipline, such a discrepancy between ends and means should cause no worry. But I submit that the consistently indifferent performance in practical applications is in fact a symptom of a fundamental imbalance in the present state of our discipline. The weak and all too slowly growing empirical foundation clearly cannot support the proliferating superstructure of pure, or should I say, speculative economic theory. (Leontief 1971, 1)

Much is being made of the widespread, nearly mandatory use of modern economic theorists of mathematics. To the extent which the economic phenomena possess observable quantitative dimensions, this is indisputably a major forward step. Unfortunately, any one capable of learning elementary, or preferably advanced calculus and algebra, and acquiring acquaintance with the specialized terminology of economics can set himself up as a theorist. Uncritical enthusiasm for mathematical formulation tends often to conceal the ephemeral substantive content of the argument behind the formidable front of algebraic signs. (Leontief 1971, 1-2, emphasis added.)

(….) By the time it comes to the interpretation of the substantive conclusions, the assumptions on which the model has been based are easily forgotten. But it is precisely the empirical validity of these assumptions on which the usefulness of the entire exercise depends. (Leontief 1971, 2)

What really is needed, in most cases, is a very difficult and seldom very neat assessment and verification of these assumptions in terms of observed facts. Here mathematics cannot help and because of this, the interest and enthusiasm of the model builder suddenly begins to flag: “If you do not like my set of assumptions, give me another and I will gladly make another model; have your pick.” (Leontief 1971, 2)

(….) To sum up with the words of a recent president of the Econometric Society, “… the achievements of economic theory in the last two decades are both impressive and in many ways beautiful. But it cannot be denied that there is something scandalous in the spectacle of so many people refining the analysis of economic states which they give no reason to suppose will ever, or have ever, come about…. It is an unsatisfactory and slightly dishonest state of affairs.” (Leontief 1971, 2)

But shouldn’t this harsh judgment be suspended in the face of the impressive volume of econometric work? The answer is decidedly no. This work can be in general characterized as an attempt to compensate for the glaring weakness of the data base available to us by the widest possible use of more and more sophisticated statistical techniques. Alongside the mounting pile of elaborate theoretical models we see a fast-growing stock of equally intricate tools. These are intended to stretch to the limit the meager supply of facts. (Leontief 1971, 2-3)

(….) However, like the economic models they are supposed implement, the validity of these statistical tools depends itself on the acceptance of certain convenient assumptions pertaining to stochastic properties of the phenomena which the particular models are intended to explain; assumptions that can be seldom verified. (Leontief 1971, 3)

In no other field of empirical inquiry has so massive and sophisticated a statistical machinery been used with such indifferent results. (Leontief 1971, 3)

(….) Continued preoccupation with imaginary, hypothetical, rather than with observable reality has gradually led to a distortion of the informal valuation scale used in our academic community to assess and to rank the scientific performance of its members. Empirical analysis, according to this scale, gets a lower rating than formal mathematical reasoning. (Leontief 1971, 3)

A natural Darwinian feedback operating through selection of academic personnel contributes greatly to the perpetuation of this state of affairs…. Thus, it is not surprising that the younger economists, particularly those engaged in teaching and in academic research, seem by now quite content with … building more and more complicated mathematical models and devising more and more sophisticated methods of statistical inference without engaging in empirical research. Complaints about the lack of indispensable primary data are heard from time to time, but they don’t sound very urgent. (Leontief 1971, 3)

(….) To deepen the foundation of our analytical system it will be necessary to reach unhesitatingly beyond the limits of the domain of economic phenomena as it has been staked out up to now…. To penetrate below the skin-thin surface of conventional [econometric mathematical models], it will be necessary to develop a systematic study of the structural characteristics and functioning of [economic targets, e.g., households, etc.], an area in which description and analysis of social, anthropological and demographic factors must obviously occupy the center of the stage. (Leontief 1971, 4, emphasis added.)

Establishment of systematic cooperative relationships across the traditional frontiers now separating economics from these adjoining fields is hampered by the sense of self-sufficiency resulting from what I have already characterized as undue reliance on indirect statistical inference as the principal method of empirical research. (Leontief 1971, 4, emphasis added.)

Formally, nothing is, of course, wrong with such an apparently circular procedure. Moreover, the model builder in erecting his hypothetical structures is free to take into account all possible kinds of factual knowledge and the econometrician in principle, at least, can introduce in the estimating procedures any amount of what is usually referred to as “exogenous” information before he feeds his programmed tape into the computer. Such options are exercised rarely and when they are, usually in a casual way. (Leontief 1971, 4-5, emphasis added.)

(….) The same well-known sets of figures are used again and again in all possible combinations to pit different theoretical models against each other in formal statistical combat…. The orderly and systematic nature of the entire procedure generates feelings of comfortable self-sufficiency. (Leontief 1971, 5)

This complacent feeling … discourages … attempts that would involve crossing the conventional lines separating ours from the adjoining fields. (Leontief 1971, 5)

An exceptional example of a healthy balance between theoretical and empirical analysis and the readiness of professional economists to cooperate with experts in the neighboring disciplines is offered by Agricultural Economics as it developed in this country over the last fifty years…. Preoccupation with the standard of living of the rural population has led agriculture economists into collaboration with home economists and sociologists, that is, with social scientists of the “softer” kind. While centering their interest on only one part of the economic system, agricultural economists demonstrated the effectiveness of systematic combination of theoretical approach with detailed factual analysis. They were also the first among economists to make use of the advanced methods of mathematical statistics. However, in their hands, statistical inference became a complement to, not substitute for, empirical research. (Leontief 1971, 5, emphasis added.)

The shift from casual empiricism that dominates much of today’s econometric work to systematic large-scale factual analysis will not be easy. (Leontief 1971, 5, emphasis added.)

[I]t is a common trope of theoretical modeling exercises to begin by citing a “stylized fact” about some aspect of the economy and to present the exercise as developing a possible explanation. Theoretical modeling exercises may vary in how much the foreground the questions about the target world that ultimately motivate the exercise, but some recourse of this kind is implied whenever these exercises are presented as economically relevant (as opposed to simply mathematical exercises). … This matters because it suggests that even highly abstract theoretical models are ultimately justified on the basis of their correspondence to some features of the target world. (Spielgler 2015, 67)

~ ~ ~

Henry Louis Mencken [1917] once wrote that “[t]here is always an easy solution to every human problem — neat, plausible and wrong.” And neoclassical economics has indeed been wrong. Its main result, so far, has been to demonstrate the futility of trying to build a satisfactory bridge between formalistic-axiomatic deductivist models and real world target systems. Assuming, for example, perfect knowledge, instant market clearing and approximating aggregate behaviour with unrealistically heroic assumptions of representative actors, just will not do. The assumptions made, surreptitiously eliminate the very phenomena we want to study: uncertainty, disequilibrium, structural instability and problems of aggregation and coordination between different individuals and groups. (Syll 2016, 56)

The punch line of this is that most of the problems that neoclassical economics is wrestling with, issues from its attempts at formalistic modeling per se of social phenomena. Reducing microeconomics to refinements of hyper-rational Bayesian deductivist models is not a viable way forward. It will only sentence to irrelevance the most interesting real world economic problems. And as someone has so wisely remarked, murder is unfortunately the only way to reduce biology to chemistry — reducing macroeconomics to Walrasian general equilibrium microeconomics basically means committing the same crime. (Syll 2016, 56)

— Lars Pålsson Syll. On the use and misuse of theories and models in economics

Introduction

On December 5, 1871, John Stewart Mill wrote to his friend and disciple John Elliot Cairnes expressing dismay at the work of William Stanley Jevons, one of the pioneers of the new abstract mathematical style in economics. Jevons had “a mania for encumbering questions with useless complications,” Mill wrote, “with a notation implying the existence of greater precision in the data than the questions admit of” (Mill 1972).

At the time of writing, Mill had not yet read Jevons’ recently published Theory of Political Economy, but if he had, he would have found no reason to change his view. Jevons, for his part, was equally critical of Mill’s work — and used remarkably similar language to make his complaint. According to Jevons, it was Mill’s economic doctrines — and those of the then-dominant British Classical School more generally — that were unnecessarily complicated, because they were based on “mazy and preposterous assumptions” about the basic concepts of political economy (Jevons 1965: xliv). (Spiegler 2015, 1)

What Mill and other classical political economists failed to see, Jevons argued, was that despite the apparent complexity of human social activity there was a fundamental simplicity and unity at its core. Standard economic notions such as utility, wealth, value, commodity, labor, land, and capital all reflected a single underlying theme: the basic human tendency to “satisfy our wants to the utmost with the least effort — to procure the greatest amount of what is desirable at the expense of the least that is undesirable — in other words, to maximize pleasure” (Jevons 1965: 37). This tendency manifested itself in human behavior in a manner that was uniform across people, quantitatively (Jevons thought cardinally) measurable, and separable from influences that were more context-dependent, such as morality or culture. Recognizing this, Jevons argued, would allow many of the issues that had troubled classical political economists to be bracketed, enabling the articulation of a precise “mechanics of utility and self-interest” on the model of physical mechanics (Jevons 1965: 21, emphasis original). (Spiegler 2015, 1-2)

According to Jevons, the analogy with physical mechanics ran deep. The “laws and relations” governing utility mechanics had to be “mathematical in nature,” because they “dealt with quantities,” i.e., “things … capable of being greater or less” (Jevons 1965: 3, emphasis original). These laws could also be isolated from potentially disturbing factors, not only conceptually but also empirically. Although the economist could not conduct controlled experiments to affect this isolation directly, Jevons believed that the effects of disturbing factors could be dealt with systematically, even when economists were largely in the dark about their nature and operation. (Spiegler 2015, 2)

Consequently, it seemed to Jevons that scepticism about the possibilities of a precise science of political economy, like that expressed by Mill in his letter to Cairnes, was merely conservatism standing in the way of progress. This sentiment was expressed clearly in the concluding comments to the Theory of Political Economy, in a section titled “The Noxious Influence of Authority” Jevons wrote:

I think there is some fear of the too great influence of authoritative writers in Political Economy. I protest against deference for any man, whether John Stewart Mill, or Adam Smith, or Aristotle, being allowed to check inquiry. Our science has become far too much a stagnant one, in which opinions rather than experience and reason are appealed to … Under these circumstances it is a positive service to break the monotonous repetition of current questionable doctrines, even at the risk of new error. (Jevons 1965: 276-7)

Looking back on the disagreement between Mill and Jevons from the perspective of 2015, it would seem that Jevons has been vindicated. Contemporary academic economics is a thoroughly mathematical enterprise, reflecting many features of Jevons’ approach. And one finds few doubts within the professional mainstream as to the aptness of the mathematical analysis of economic behavior. To most contemporary economists, Mill’s views on methodolgy of political economy are at best an interesting piece of intellectual history. They are irrelevant to the actual practice of economics. (Speigler 2015, 2)

Yet Mill’s skepticism toward Jevon’s approach to political economy may be more than a mere historical curiosity. Mill’s position, especially when understood in the context of his broader philosophy of science, poses a fundamental and formidable challenge to those who, like Jevons, would wish to use the power and precision of mathematics to investigate social phenomena. In fact, the issues Mill discerned continue to vex mathematical economics to this day. To see that, however, we need to understand the basis of his misgivings. (Spiegler 2015, 2-3)

As a committed empiricist, Mill held fast to the value of experience. The general principles of science were, in Mill’s eyes, contrivances in its service and subject to its discipline. Although abstractions were necessary to formulate general principles, Mill insisted that one must not make the mistake of taking abstractions to be the object of scientific inquiry, rather than the phenomena they were supposed to represent. If a scientist lost focus on the actual phenomena of interest in that matter, the concepts advanced in their service might well become detached from them. It would then become unclear what, if any, epistemic value the principles formulated using those concepts would have. As Mill explained,

If anyone, having possessed himself of the laws of phenomena as recorded in words, whether delivered to him originally by others, or even found out by himself, is content from thenceforth to live among these formulae, to think exclusively of them, and of applying them to cases as they arise, without keeping up his acquaintance with the realities from which these laws were collected — not only will he continually fail in his practical efforts, because he will apply his formulae without duly considering whether, in this case or in that, other laws of nature do not modify or supersede them; but the formulae themselves will progressively lose their meaning to him, and he will cease at last even to be capable of recognizing with certainty whether a case falls within the contemplation of his formulae or not. (Mill 1974: Bk. IV, ch. vi, sec. 6, 711)

The prime example of mechanical subject matter, according to Mill, was the physical universe. In his view, it was appropriate to express (for example) Newton’s principle of universal gravitation in mathematical language because human beings are capable of discerning specific quantities corresponding to “mass,” “force,” and “radius” (or, more generally “distance”) with sufficient precision that there could be no relevant qualitative differences among observations within each category. From the standpoint of Newtonian mechanics, it would not matter if one set of forces, masses, and distances occurred in France and another in England (or on the Moon or anywhere else in the universe), or if one set of observations were associated with a morally reprehensible purpose and another not. The only relevant difference between observations of the same type was in their magnitude. (Spiegler 2015, 4)

When confident that one was dealing with mechanical subject matter, it was not only appropriate but ideal to articulate general principles in mathematical language. Doing so enabled scientists to take full advantage of its purely quantitative nature. In particular, they could use their observations to derive and test precise empirical laws from those general principles. This, for example, is what Henry Cavendish did when estimating the value of the gravitational constant G, in Newton’s principle of universal gravitation, F = Gm₁ m₂/r² (which expresses the force exerted by a body of mass m1 on a body of mass m2, and visa versa, at a distance of r) (Cavendish 1798). That calculation would have been impossible — or rather, its result would have been meaningless — if Cavendish had not been warranted in taking each successive observation of mass (or the distance between the two objects, or degree of displacement of the objects due to gravity) as qualitatively identical to his preceding observations. (Spiegler 2015, 4-5)

Mathematical language is thus extemely useful in investigating mechanical subject matter. But, Mill argued, it would be perilous to use it to investigate subject matter that was not mechanical. There were two possible causes of concern. First, in such cases mathematical principles might simply project an underlying mechanical structure onto the subject matter whether or not the latter was mechanical in nature. That is, mathematical language might generate a purely quantitative conceptual map of the subject matter it purported to outline, with no way of telling whether the outlines on the map corresponded to the subject’s own contours. As a result, scientists would not be able to feel confident that data gathered according to the conceptual map accurately reflected the underlying subject matter. And because of that, it would be inappropriate to interpret any apparently precise empirical laws derived from that data as empirical laws applying to the actual subject matter. (Spiegler 2015, 5)

Second, and still more worryingly, Mill argued that the commitment to mathematical language could actually prevent scientists from detecting when their conceptual map had become untethered from the subject matter under study. As will be recalled, Mill’s prescribed defense against this kind of detachment was ongoing close contact between the scientist and the object of study. But if exploration of the subject matter itself developed only through the lens of mathematical language — which necessarily obscured any qualitative distinctions among the observations being made within each category — then the scientist would become blind to signs of that mismatch arising. As a result, the mismatch might persist indefinitely. Because of this danger, Mill warned that when the scientist was not certain of the mechanical character of the subject matter, the language of any general principles “should be so constructed that there will be the greatest possible obstacles to a merely mechanical use of it” (Mill 1974: Bk. IV, ch. vi, sec. 6, 707). (Spiegler2015, 5)

The risk that mathematical principles might ascribe mechanical features to non-mechanical subject matter, and thus become untethered from the subject matter they were meant to represent, was precisely what concerned Mill about Jevons’ approach to political economy, and indeed about mathematical social science generally. Human social activity was, for Mill, a paradigmatic example of a non-mechanical subject. It was a realm of almost unfathomable complexity, in two important ways. First, social phenomena were subject to innumerably more causes than physical phenomena. And second, crucially, the operations of those causes were inextricably intertwined. (Spiegler 2015, 6)

Whatever affects, in an appreciable degree, any one element of the social state, affects through it all the other elements. The mode of production of all social phenomena is one great case of Intermixture of Laws. We can never either understand in theory or command in practice the condition of a society in any one respect, without taking into consideration its condition in all other respects. (Mill 1974: Bk. VI, ch. ix, sec. 2, 899)

Thus, although Mill believed it was possible to form reliable general principles (perhaps even mathematical ones) about certain aspects of human nature in isolation, the fact that human beings always and only observe behavior in the welter of society meant it was impossible to discern whether and to what extent those general principles operated empirically. If indeed one knew, as Jevons presumed one would, that the influence of economic factors on human behavior was cleanly separable from the influence of all other factors, and one possessed a reliable method for screening off those influences, then a precise empirical science of political economy might be possible. But for Mill, whether the social world was parsable in this way was an empirical question — and, moreover, a question that could only be addressed through continual immersion in the social world itself — not a simple statement of fact or a self-evidently valid postulate, as Jevons assumed. To take Jevons’ route was to invite a split between model and target that would be undetectable using mathematical methods alone. One could go blithely on with mathematical explorations — gathering data, estimating the precise functional forms and parameters of the principles, and testing them against new data — unaware that in point of fact one had ceased to be exploring the phenomena of interest in any meaningful way. (Spiegler 2015, 6)

Mill’s challenge to Jevons may seem distant from the modern discipline of economics. Yet it finds strong echoes in the debate over the implications for economic methodology of the recent financial crisis. A central question in that debate has been whether the highly abstract mathematical modeling methods that dominated macroeconomics in the years leading up to the crisis — in particular, Dynamic Stochastic General Equilibrium (DSGE) modeling — actively prevented economists from seeing the gathering storm. Critics of DSGE have charged that these models became untethered from the phenomena they were meant to represent in precisely the manner Mill feared. In a 2010 review of DSGE modeling in the Journal of Economic Perspectives, for example, Ricardo Caballero wrote that the practice of DSGE modeling “has become so mesmerized with its own internal logic that it has begun to confuse the precision it has achieved about its own world with the precision that it has about the real one” (Caballero 2010: 85). The primary culprits in that confusion, critics charged, were the extreme simplifying assumptions necessary to ensure the tractability of DSGE models — in particular, (i) the representation of aggregate economic activity as being generated by a small number of representative agents; (ii) the expression of the macroeconomy as a linear (generally log-linear) system; and (iii) the assumption of efficient financial markets. These assumptions rendered the model incapable of taking into account many kinds of complexity that turned out to be crucial factors in the crisis — for example, the perverse incentive structures at play in the financial sector in the late 1990s and 2000s. In effect, the models became mere mathematical exercises — toy models that were not models of the late 1990s‒2000s economy in any meaningful sense. (Spiegler 2015, 6)

Critics have also been concerned with the manner in which the mismatch between DSGE models and the actual economy gave rise to certain analytical blind spots. In a 2009 New York Times Magazine piece cataloguing the failures of economic methodology in the lead-up to the crisis, Paul Krugman argued that DSGE models caused a kind of tunnel vision in which the central causes of the crisis lay outside the realm of consideration. Conceiving of the economy through the lens of the model essentially required the economist to 

[turn] a blind eye to the limitations of human rationality that often lead to bubbles and busts; to the problems of institutions that run amok; to the imperfections of markets — especially financial markets — that can cause the economy’s operating system to undergo sudden, unpredictable crashes; and to the dangers created when regulators don’t believe in regulation. (Krugman 2009) (Spiegler 2015, 7)