Tuesday, June 14, 2016

The pool player analogy is silly

In a lot of debates about economic methodology, someone will bring up Milton Friedman's "pool player" analogy. The pool player analogy was part of Milton Friedman's rationale for modeling the behavior of economic agents (consumers, firms, etc.) as the optimization of some objective function. Unfortunately, the analogy is A) not that good in the first place, and B) frequently misapplied to make excuses for models that don't match data.

Here's the original analogy:
Consider the problem of predicting the shots made by an expert billiard player. It seems not at all unreasonable that excellent predictions would be yielded by the hypothesis that the billiard player made his shots as if he knew the complicated mathematical formulas that would give the optimum directions of travel, could estimate accurately by eye the angles, etc., describing the location of the balls, could make lightning calculations from the formulas, and could then make the balls travel in the direction indicated by the formulas. Our confidence in this hypothesis is not based on the belief that billiard players, even expert ones, can or do go through the process described; it derives rather from the belief that, unless in some way or other they were capable of reaching essentially the same result, they would not in fact be expert billiard players.  
It is only a short step from these examples to the economic hypothesis that under a wide range of circumstances individual firm behave as if they were seeking rationally to maximize their expected returns (generally if misleadingly called “profits”) 16 and had full knowledge of the data needed to succeed in this attempt; as if, that is, they knew the relevant cost and demand functions, calculated marginal cost and marginal revenue from all actions open to them, and pushed each line of action to the point at which the relevant marginal cost and marginal revenue were equal. Now, of course, businessmen do not actually and literally solve the system of simultaneous equations in terms of which the mathematical economist finds it convenient to express this hypothesis, any more than leaves or billiard players explicitly go through complicated mathematical calculations or falling bodies decide to create a vacuum. The billiard player, if asked how he decides where to hit the ball, may say that he “just figures it out” but then also rubs a rabbit’s foot just to make sure; and the businessman may well say that he prices at average cost, with of course some minor deviations when the market makes it necessary. The one statement is about as helpful as the other, and neither is a relevant test of the associated hypothesis.
Actually, I've always thought that this is kind of a bad analogy, even if it's used the way Friedman intended. Using physics equations to explain pool is either too much work, or not enough.

Suppose the pool player is so perfect that he makes all his shots. In that case, using physics equations to predict what he does is a pointless waste of time and effort. All you need is a map of the pockets. Now you know where the balls go. No equations required! Actually, even that's too much...since in most pool games it doesn't matter which balls go in which pockets, you don't even need a map, you just need to know one fact: he gets them all in. It's a trivial optimization problem.

But if really good pool players made 100% of their shots, there wouldn't be pool tournaments. It would be no fun, because whoever went first would always win. But in fact, there are pool tournaments. So expert pool players do, in fact, miss. They don't quite optimize. So if you want to predict which pool player wins a tournament, or why they miss a shot, you need more than just a simple balls-in-pockets optimization model. And you probably need more than physics - you could use psychology to predict strategic mistakes, biology to predict how arms and hands slightly wobble, and complex physics to predict how small random non-homogeneities in the table and air will cause random deviations from an intended path. 

The point is, if you use an optimization model to represent the behavior of someone who doesn't actually optimize, you're going to get incorrect results.

Of course, the pool player analogy wasn't Friedman's whole argument - the next paragraph is critical:
Confidence in the maximization-of-returns hypothesis is justified by evidence of a very different character. This evidence is in part similar to that adduced on behalf of the billiard-player hypothesis - unless the behavior of businessmen in some way or other approximated behavior consistent with the maximization of returns, it seems unlikely that they would remain in business for long. Let the apparent immediate determinant of business behavior be anything at all - habitual reaction, random chance, or whatnot. Whenever this determinant happens to lead to behavior consistent with rational and informed maximization of returns, the business will prosper and acquire resources with which to expand; whenever it does not, the business will tend to lose resources and can be kept in existence only by the addition of resources from outside. The process of “natural selection” thus helps to validate the hypothesis - or, rather, given natural selection, acceptance of the hypothesis can be based largely on the judgment that it summarizes appropriately the conditions for survival. 
That turns out to just be wrong. There are plenty of theoretical ways that non-profit-maximizing agents can stay around forever. Also, there are always new people and new companies being born and entering the system - there's a sucker born every minute, so as long as they drop out at some finite rate, there's some homeostatic equilibrium with a nonzero amount of suckers present. And finally, this argument obviously doesn't work for consumers, who don't die if they make bad decisions.

So Friedman's analogy was not a great one even on its own terms. Sometimes consumers, firms, and other agents don't perfectly optimize. Sometimes that's important. So you might want to model the ways in which they don't perfectly optimize.

But actually, everything in this post up to now has been a relatively minor point. There's a much bigger reason why the pool player analogy is bad, especially when it comes to macro - it gets chronically misused.

In pool, we know the game, so we know what's being optimized - it's "balls in pockets". But in the economy, we don't know the objective function - even if people optimize, we don't automatically know what they optimize. Studying the economy is more like studying a pool player when you have no idea how pool works.

In economic modeling, people often just assume an objective function for one agent or another, throw that into a larger model, and then look only at some subset of the model's overall implications. But that's throwing away data. For example, many models have consumer preferences that lead to a consumption Euler equation, but the model-makers don't bother to test if the Euler equation correctly describes the real relationship between interest rates and consumption. They don't even care.

If you point this out, they'll often bring up the pool player analogy. "Who cares if the Euler equation matches the data?", they'll say. "All we care about is whether the overall model matches those features of the data that we designed it to match."

This is obviously throwing away a ton of data. And in doing so, it dramatically lowers the empirical bar that a model has to clear. You're essentially tossing a ton of broken, wrong structural assumptions into a model and then calibrating (or estimating) the parameters to match a fairly small set of things, then declaring victory. But because you've got the structure wrong, the model will fail and fail and fail as soon as you take it out of sample, or as soon as you apply it to any data other than the few things it was calibrated to match.

Use broken pieces, and you get a broken machine.

This kind of model-making isn't really like assuming an expert player makes all his shots. It's more like watching an amateur pool player until you he makes three shots in a row, and then concluding he's an expert.

Dani Rodrik, when he talks about these issues, says that unrealistic assumptions are only bad if they're "critical" assumptions - that is, if changing them would change the model substantially. It's OK to have non-critical assumptions that are unrealistic, just like a car will still run fine even if the cup-holder is cracked. That sounds good. In principle I agree. But in practice, how the heck do you know in advance which assumptions are critical? You'd have to go check them all, by introducing alternatives for each and every one (actually every combination of assumptions, since model features tend to interact). No one is actually going to do that. It's a non-starter. 

The real solution, as I see it, is not to put any confidence in models with broken pieces. The dream of having a structural model of the macroeconomy - one that we can trust to be invariant to policy regime changes, one that we can confidently apply to new situations - is a good dream, it's just a long way off. We don't understand most of the structure yet. If you ask me, I think macroeconomists should probably focus their efforts on getting solid, reliable models of each piece of that structure - figure out how consumer behavior really works, figure out how investment costs really work, etc. That's what "macro-focused micro" is really about, I think.

So let's put Friedman's pool player analogy to rest.


Chris House (who was the first person to ever introduce me to the pool player analogy) has a response to this post. But as far as I can tell, he merely restates Friedman's (flawed) logic without addressing the main points of my post. 

Saturday, June 11, 2016

Econ theory as signaling?

I don't expend much effort dissing macroeconomics these days, but every once in a while it's good to give people a reminder. I wrote a Bloomberg post about how academic macro (or more accurately, mainstream macro theory) has not really helped out the finance industry, the Fed, or coffee house discussions. The reason, as Justin Wolfers recently pointed out, is basically that DSGE models don't work. Brad DeLong then wrote a post riffing on mine, which is excellent and which you should read. A super-fun Twitter discussion then followed, part of which Brad storified for posterity.

But that leaves the question: Assuming Wolfers and DeLong and I aren't just blowing smoke out of our rear ends, and DSGE models really don't work, why do so many macroeconomists spend so much time on them? One obvious hypothesis is that a huge percent of their human capital is already invested in knowing how to do this technique, so they just keep doing what they know how to do, and teaching it to their grad students.

Another hypothesis could be that it's just an equilibrium of a repeated coordination game. Universities pay macroeconomists to do research, but they have absolutely no idea what good macroeconomic research is, so in practice they pay macroeconomists to do whatever other macroeconomists decide is good. Maybe since macro data is very uninformative, no one actually knows what good research looks like, so they all settle on some random thing - DSGE models. This is a kind of Kuhnian explanation.

Another hypothesis is politics - a small conservative old guard thinks that since DSGE is at some level based on RBC, forcing everyone to do DSGE will nudge macro toward anti-interventionist stances on fiscal and monetary policy. And they use their positions of influence at departments, journals, and professional organizations to enforce conformity among the younger, less politicized economists. I don't really buy this hypothesis, but someone usually brings it up.

Yet another hypothesis is that it's just fun for some people to do, or at least to watch other people do, this kind of theory. Paul Romer recently complained that "in the new equilibrium...empirical work is science; theory is entertainment." I'm sure there are people out there for whom this really is the case - I once saw V.V. Chari get very excited that he couldn't use a fixed-point theorem to prove the existence of a solution in one of his models, and had to resort to more exotic methods. Heh. 

But here's another hypothesis: What if it's signaling?

I've been very skeptical of the fad in which everyone invokes signaling to explain social phenomena. I'm also pretty critical of the signaling model of college - yeah, it's probably part of what's going on, but the signal is just too expensive (4+ years of the prime working years of millions of our most talented young people, wasted on signaling?). So I bet it's a smallish piece of the college puzzle.

BUT, when it comes to DSGE, I kind of suspect that signaling could be a bigger piece of what's going on.

That suspicion was probably planted in 2005, before I even went to grad school, by a Japanese economist I knew who had done his PhD at Stanford. He gave me his advice on how to have an econ career: "First, do some hard math thing, like functional analysis. Then everyone will know you're smart, and you can do easy stuff." That's paraphrased only a little (I can't recall his exact wording).

I then watched a number of my grad school classmates go into macroeconomics. Their job market papers all were mainly theory papers, though - in keeping with typical macro practice - they had an empirical section that was usually closely related to the theory. The models all struck me as hopelessly unrealistic and silly, of course, and in private my classmates - the ones I talked to -  agreed that this was the case, and said lots of mean things about DSGE modeling in general, basically saying "This is the game we have to play." Then all of those classmates went on to do much less silly-seeming stuff, usually more focused on empirics, usually for government agencies. Essentially, they followed the advice of that Japanese economist.

Finally, I noticed an interesting data fact. Theory papers are getting much less common in top econ journals, but are still prominent among job market papers. The pattern again looks the same - prove yourself with theory, then do more empirical stuff later on. Of course, this data is for all econ, not just macro, and some percentage is going to just be people in the micro theory field itself. Plus, the thing for job market papers is just one year. So it's far from a slam-dunk case, but it's another piece of evidence that seems to fit the pattern.

But OK, suppose signaling is going on. What's being signaled, why is it valuable, and why is it hard to observe directly? The obvious possibility is that it's signaling intelligence - that the ability to make DSGE models is just an upper-tail IQ test. That's valuable because A) in the long run, people with very high intelligence are going to do good research, and B) intelligence gets much harder to observe in the upper tail. If DSGE is an IQ test, though, the invention of tools like Dynare that make it easier to make DSGE models might push the profession toward a pooling equilibrium, lowering the prestige and/or the salary of macroeconomists.

But it might also be what Bryan Caplan calls "conformity signaling". If macroeconomics research is a coordination game (see above), and if the prevailing research paradigm is not really better than alternatives, then you probably want macroeconomists who are willing to "play the game", as it were. So DSGE might be an expensive way of proving that you're willing to spend a lot of time and effort doing silly stuff that the profession tells you to do.

So there it is: The Signaling Model of Macro Theory Research.


Of course, all this is predicated on the notion that DSGE models haven't really increased our understanding of the economy. Chris Sims, one of the smartest folks in the business, and a very empirically minded macroeconomist, is a defender of DSGE. And here's another DSGE defense. So again, my premise here could always just be wrong.

Also, there are a lot of DSGE papers I personally like, but they tend to be ones that ingeniously poke holes in other DSGE models. See this discussion in the comments for some of those. Also, a few other examples are here, here, and here.

If you want to know what I think is the actual problem with DSGE models, see my next post

Tuesday, June 07, 2016

Republic of Science or Empire of Ideology?

The Washington Post has a long story by Jim Tankersley about Charles' Koch's attempt to influence the economics profession with massive donations of money to large numbers of universities. Here are some excerpts:
Koch’s donations have fueled the expansion of a branch of economic research that aligns closely with his personal beliefs of how markets work best: with strong personal freedom and limited government intervention. 
They have seeded research centers, professors and graduate students devoted to the study of free enterprise, who often provide the intellectual foundation for legislation seeking to reduce regulations and taxes... 
From 2012 through 2014 alone, his charitable arm, the Charles Koch Foundation, donated $64 million to university programs. A tax filing from 2013 lists more than 250 schools, departments or programs that received grants from the foundation, in amounts that ranged from a few thousand dollars to more than $10 million at George Mason University in Fairfax, Va. Recipients include obscure liberal arts colleges, flagship state universities and members of the Ivy League.

Some donations flow to research hubs within an institution, such as Mercatus at George Mason and the Ed Snider Center for Enterprise and Markets at the University of Maryland, which ground their research in the belief that economic freedom — and less government intervention — is the key to increased prosperity. Some support faculty positions at schools such as Clemson University and Florida State University, which have long specialized in that same sort of research...
Koch no longer personally reviews those applications — his foundation staff does...Koch, though, has articulated a set of principles to determine who gets his money. He has prized researchers whose values, as he calls them, are rooted in an economic philosophy that aligns with his— the belief that economic and personal freedoms produce the fastest advancements in human well-being.
The Post's article is titled "Inside Charles Koch’s $200 million quest for a 'Republic of Science'". This is a reference to a 1962 article by Michael Polanyi called "The Republic of Science: Its Political and Economic Theory". In that article - which Koch cites as a big influence on his efforts - Polanyi says that research dollars should flow to the scientists whose work is supported by the scientific consensus. Tankersley drily notes:
[Koch's donation effort] raises the question of whether Koch has become, for university researchers, the sort of distorting force that Polanyi warns against.
Why yes. 

Koch is making a sustained, multi-hundred-million dollar effort to push the academic economics profession toward a libertarian ideology. This is a "Republic of Science" to the same degree that North Korea is a "Democratic People's Republic of Korea".

One way to see this is as a defensive reaction against the interventionist turn in economic thinking. On many issues, academic economists are now less pro-free-market than the general public. And the most famous public-facing economists now tend to be left-leaning rather than right-leaning - Hayek and Friedman have given way to Piketty and Krugman. So the Koch donation campaign might be an attempt by libertarians to stem the tide.

Another way is to see it as a defensive reaction against the overall leftward turn of academia. Many social science disciplines - anthropology, urban studies, social psychology, and probably sociology - seem to have been captured by leftist ideology to a greater degree than econ was ever captured by libertarianism, even in the 70s and 80s. Koch might be using his hundreds of millions to try to preserve econ as a bulwark against this leftist capture of social science.

A final interpretation is that Koch is just doing what Koch always does - steadily pushing libertarian thought on the world by whatever means seem most expeditious.

Whatever it is, though, I don't like it. Unlike Koch, and unlike many of the lefty social science types I've been having debates with recently, I don't believe that social science is an inherently ideological enterprise. And I think it sets back our understanding of the world when people try to flood any portion of academia with researchers whom they think will promote a certain set of conclusions.

I don't have much more to say than that, so here's one of my favorite Feynman quotes:
Our responsibility is to do what we can, learn what we can, improve the solutions, and pass them on. It is our responsibility to leave the people of the future a free hand. In the impetuous youth of humanity, we can make grave errors that can stunt our growth for a long time. This we will do if we say we have the answers now, so young and ignorant as we are. If we suppress all discussion, all criticism, proclaiming “This is the answer, my friends; man is saved!” we will doom humanity for a long time to the chains of authority, confined to the limits of our present imagination. It has been done so many times before.
A real "Republic of Science" would focus on an open-minded search for truth, not the enshrinement of one pre-decided dogma.


I also thought this passage from Tankersley's article was interesting:
None of the largest recipients of Koch dollars appear on a list of the most influential academic economic departments in the United States, as calculated by the research arm of the Federal ­Reserve Bank of St. Louis. Only one professor who works at one of Koch’s most-supported centers cracks a similar list that calculates the top 5 percent of influential economists in the research community 
Koch-funded researchers make a larger impact in the public arena. They frequently testify before Republican-led committees in Congress. Their work often guides lawmakers, particularly conservatives, at the state level in drafting legislation, and they have provided the foundations for judicial opinions that affect the economy on issues such as whether the government should intervene to stop large companies from merging.
It's possible that the Koch doesn't want to influence economic science itself, as much as he wants to sculpt its public-facing component. The end result could be two econ professions - a dispassionate, truth-seeking one occupying the upper levels of the ivory tower, at MIT and Princeton and Stanford, doing hard math things and careful honest data work that slowly trickles out through traditional media channels...and a second econ profession in the lower-ranked schools, doing a slightly fancier version of the kind of political advocacy now done by conservative think tanks. The former would have the best brains and the best understanding of the real world, but the latter would have much more policy influence and impact on the wider intellectual world. This is different from the wholesale yoking of science to ideology that I was envisioning, but it also doesn't seem like a pleasant vision of the future.

Many Koch money recipients have pushed back on Twitter, saying that unions and left-leaning think tanks also fund university research too. Of course, that does worry me too - maybe it's time for a general code of ethics for econ funding. But it worries me a lot more if A) the funding becomes the main source of funding for whole departments, B) it's hundreds of millions of dollars from one single source, C) it's explicitly ideological, and D) it seeks to make hiring decisions along ideological lines instead of simply funding research by existing profs.

There's lots of dirty stuff out there in econ, but the Koch effort just seems so huge and so unapologetically ideological that it's worth singling out. Quantity, as one of Koch's favorite authors once said, has a quality all its own.

A commenter talks about the situation at Western Carolina University. I've mainly been thinking about the science and policy-advocacy aspects of this issue, but education seems important as well.

Saturday, June 04, 2016

Do feathers fall as fast as iron balls?

Josh Hendrickson's Twitter account is @RebelEconProf, and his blog is The Everyday Economist. So if both these names are accurate, I can only assume that Josh adheres to Mao's theory of Perpetual Revolution!

That has nothing to do with this post, I just always wanted to say that.

What this post is really about is that Josh wrote a post about my Bloomberg post about Econ 101! So I decided to write a counter-rebuttal post. Hmm.

OK, let's back up. I have two basic criticisms related to Econ 101:

1. I think 101 classes don't include enough empirics.

2. I think 101 models often get misapplied in public discussions, a phenomenon I call "101ism".

Josh is arguing about (1). I think. Mostly. But I think he doesn't always quite get what I'm saying. Therefore, I will do - you guessed it! - a point-by-point response. You know you love em.

Noah Smith’s dislike of Econ 101 seems to come from the discussion of the minimum wage. 
Not really, no. That's just one example. It's probably one of the more egregious examples when it comes to the quality of the public discussion, but in terms of 101 models not fitting the data, there are better examples. For example, immigration is a positive labor supply shock, and positive labor supply shocks push down wages, right? Well, no, not in reality. That debate is probably a lot more settled than the minimum wage debate.

[Noah's] basic argument is that Econ 101 says that the minimum wage increases unemployment. However, he argues that: 
That’s theory. Reality, it turns out, is very different. In the last two decades, empirical economists have looked at a large number of minimum wage hikes, and concluded that in most cases, the immediate effect on employment is very small. 
This is a bizarre argument in a number of respects. First, Noah seems to move the goal posts. The theory is wrong because the magnitude of these effects are small? The prediction is about direction, not magnitude.
If a theory represents only one tiny piece of reality, should we teach that theory front-and-center, in intro classes, as the main lens through which we are to understand the world?

I say no.

Here's an analogy. Suppose I drop a feather and an iron ball off of the Leaning Tower of Pisa, at the same height. Which one hits the ground first? Correct answer: The iron ball. It's denser than the feather, so it is less slowed down by air resistance. In fact, it's not even close.

Now, Newton's laws (including the classical Law of Gravitation) are a lot more general than air resistance. You need to learn Newton's Laws, just like you need to learn supply-and-demand.

But if you teach Physics 101 kids that Newton's Laws imply that feathers and iron balls fall at the same rate here on Earth, you're going to be embarrassed when some smartypants kid points out that no, they don't actually. And then the kids are going to start thinking physics is quackery.

This is why Physics 101 teachers are careful to emphasize that Newton's Laws only describe motion well when you can neglect air resistance. And then they send the physics kids to a lab, where they can see how and when air resistance matters, by looking at the evidence.

Econ 101 teachers are not always so humble in their presentation of theories, nor do they always defer to the evidence as the ultimate arbiter. And because of this, Econ 101 students are going to grow up, and they're going to read a Nick Hanauer column saying econ is total bullshit, because we keep raising the minimum wage and it hasn't and they're going to think "Everything I learned in Econ 101 was wrong!" Then they're going to turn to alternate sources - ideological movements, wordy literary tomes, etc. - to help them understand the economy.

In fact, this has already happened to a substantial degree.


Second, David Neumark and William Wascher’s survey of the literature suggests that there are indeed disemployment effects associated with the minimum wage and that these results are strongest when researchers have examined low-skilled workers.
OK, yeah, the debate surely isn't settled. Not as much as, say, the immigration debate. But meta-analyses show that the estimates of the studies with the largest sample sizes cluster at exactly zero effect. It seems to me that the people saying the short-term effect is quantitatively small haven't yet won, but they're winning.

Forgetting the evidence, let’s suppose that Noah’s assertion that the discussion of the minimum wage in Econ 101 is empirically invalid is correct. Even in this case, the idea that Econ 101 is fundamentally flawed is without basis. When I teach students about price controls, I am careful to note the difference between positive and normative statements. For example, many students tend to see price controls as a “bad” thing. When I teach students about price controls, however, I am quick to point out that saying something is “bad” is a normative statement. In other words, “bad” implies that things should be different. What “should be” is normative. The only positive (“what is”) statement that we can make about price controls is that they reduce efficiency. Whether or not this is a good or a bad thing depends on factors that are beyond an Econ 101 course — and I provide some examples of these factors.
Josh seems a bit confused here about what I'm saying. I'm not arguing for the inclusion of normative economics in Econ 101. I'm saying that if you don't teach Econ 101 kids some evidence, you're getting the positive economics wrong.

The value of Econ 101 is the very process of thinking through [the] possible effects [of a policy change like the minimum wage]. What effect we actually observe is an empirical question, but it is of secondary importance to teaching students how to logically think through these sorts of examples.
Here's a real difference in philosophy between me and Josh. Josh thinks that teaching kids how to think deeply about the implications of models is Job #1, and everything else is of secondary importance. I think that if people use the wrong model to think about real-life situations, then this kind of deep logical thinking becomes worse than useless. Thinking deeply about bad models just leads to yet more mistaken conclusions about reality. I think Job #1 is to figure out how to use evidence to tell good models from bad.

You can learn how to think deeply through model implications in your second-year classes, after you have a realistic understanding of how to know whether you should do so in real life. Theories are powerful tools, and I think the first lesson for any powerful tool should be how to use it responsibly.

If you are a student who only learned the perfectly competitive model in Econ 101, then you should politely ask for a refund. Econ 101 routinely includes the discussion of externalities, public goods, monopoly, oligopoly, etc. All of these topics address issues that the competitive market model is ill-equipped to explain. 
On this point, Josh and I are in total and complete agreement. This is what I mean when I bash "101ism".

Anyway, thanks to Josh for responding, and I look forward to purging him and the other capitalist roaders from our glorious Cultural Revolution talking about this further!

And now, back to my regularly scheduled caffeine-overdosing.

Wednesday, June 01, 2016

Can social science yield objective knowledge?

I've been having a fairly epic email argument with a lefty* social scientist friend, about whether social science can give us objective knowledge about the world. Apparently, it has become accepted in lefty* social science and humanities circles that the study of human beings is an inherently subjective enterprise, and will never yield the kind of knowledge delivered by physics, chemistry, biology, etc.

There are essentially three arguments for this. Paraphrasing heavily:

1) Social science has policy implications, and so ideological bias will always leak in, affecting both researchers' methodological choices and their interpretations of conclusions.

2) Social phenomena are highly complex, and hence can never be understood in the way natural phenomena can be.

3) Social science = humans studying other humans, and "reflexivity" prevents us from understanding ourselves in the same way we could understand the behavior of ants or atoms.

I think all that each of these arguments highlights an important difficulty of doing social science, but gets the implications wrong.

In fact, in recent decades, a few very successful predictive social science theories have emerged that don't suffer much from any of these problems. My four favorite examples are auction theory, gravity models, matching theory, and random-utility discrete choice models. Each of these is not just very predictive, but very useful to humanity. They power Google auctions, allow people to forecast trade flows, improve organ transplants, let cities predict how many people will use the train, and allow humanity to do many other things.

Keeping these examples in mind, let's go through the (heavily paraphrased) arguments one by one.**

1) First, the presence of ideological bias. Yes, these days few people care about the policy implications of the orbit of Venus, while most people care about the policy implications of minimum wage studies. So these days, people are more likely to be objective about the former than the latter. But it was not always thus! There was a time when scientists were being put under house arrest (or worse) for saying politically incorrect things about the orbits of the planets.

Eventually, the facts won out. Natural scientists who ignored the prevailing ideology were able to predict the motion of the planets better than their rivals, and that basically settled it. There seems no reason, in principle, why a similar process wouldn't happen with social science.

Now, it might happen a lot slower, because really super-predictive social science theories are harder to get than super-predictive physics theories. But predictive success seems to drive out ideology, meaning that social science has a chance of being objective.

Some people think it's a good idea for social science researchers to lay their cards on the table - to admit their ideologies when they report their research results. That sounds like a nice idea, but I suspect it is not even slightly feasible in practice. Imagine an economist saying "From this natural experiment, I find the elasticity of labor supply with respect to minimum wage increases to be -0.1. Also, you should know I'm a lefty type who wants to use policy to help the working class."

Well, how much did the economist let said ideology affect said estimation? Did he underestimate the elasticity because he thinks that reporting a small number will make people more likely to enact minimum wages, which he thinks will help the working class? Did he overestimate the elasticity in a conscious or unconscious attempt to correct for his bias? Did he try to get an unbiased estimate, because he doesn't know whether minimum wage would help or hurt the working class, and he wants to find out?

Who knows??? Not him. Not the reader. So this sounds like a nice idea, but I don't think it would work in practice at all. In practice it would just lead to a lot of confusion, suspicion, and noise.

2) Second, complexity. Well, again, this is a big challenge in natural sciences too! Quantum mechanics and relativity have passed every empirical test, to arbitrary levels of precision. But will these things tell us how a tree grows? Maybe. But if so, that's certainly far in the future. Right now, there are plenty of phenomena too complex for particle physics to understand. That doesn't mean particle physics is incapable of yielding objective knowledge.

To me, the argument that social science phenomena are too complex seems quite a bit like the "irreducible complexity" that creationists use to argue for "intelligent design". Yes, you can always find some phenomenon so complex that existing theories can't (yet) explain it. And as theories get better and better, you can keep on jumping up to even more complex theories, saying  "Oh yeah? Explain that, scientists!". But that just means you keep losing and losing, as scientists get better and better at explaining the world.

I guess you can jump directly to the most complex, hard-to-study phenomena of all - macroeconomics, politics, and history - and camp out there for a good long while, constantly saying "See? Told you so! You haven't explained this stuff yet, and you never will!" And you're probably safe - you'll be in your grave before science explains these hellishly complex, probably-non-ergodic macro-phenomena.

Well, good for you! But in the meantime, the sphere of things that can be explained by science will expand...

3) "Reflexivity". The idea that humans can't study their own behavior. If you manage to make a theory that predicts human behavior, people's behavior will change so that the theory no longer works.

Well that's obviously wrong. Here's a very useful, robust law of human behavior that many humans have rediscovered over the years: if you walk up and threaten random humans with a deadly weapon, they'll probably hand over their money.

Obviously, reflexivity often matters. In economics there are plenty of examples. The Lucas Critique. The disappearance of asset market anomalies. I'm sure there are also tons of examples in other social sciences.

But there are obviously lots of situations in which it probably doesn't come into play very much. Epidemiologists figured out that when everyone washes their hands, disease has more difficulty spreading. So they made rules and public awareness campaigns trying to get people to wash their hands. The rules and campaigns worked, and now disease has a harder time spreading in rich countries. Reflexivity be damned! There are also plenty of examples in econ - the response to taxation, for instance, or the labor market effects of immigration - that have no obvious reflexivity problem.

So while ideology, complexity, and reflexivity are real challenges in social science, they don't seem insurmountable. They don't seem to represent a fundamental difference between social and natural science.

These arguments against objective social science really feel a lot like the "God of the gaps" reasoning that religious thinkers use to argue against the universality of natural science. Every gap in science's current ability to explain the world is presented as a reason to embrace religion - usually the specific religion of the person making the argument.

When it comes to social science, the "natural alternative" for the people making the above arguments isn't God, it's lefty* ideology. Into every gap in our current understanding of social phenomena should flow the conviction that the have-nots are oppressed by the power of the haves. Arguing with lefty* friends in the soft-sociology/anthropology/humanities complex feels a bit like arguing with a Catholic priest about science back in 1700. "You can predict balls rolling down ramps, but can you tell me when the next thunderstorm is going to happen? No you can't! See? Nature is inherently mysterious! Only the Bible will show you the truth!"

Social science is damn hard (and not just for the reasons described above). It'll be many years before predictive social theories get good enough to understand things like recessions, elections, or the rise and fall of great powers. Maybe it'll never happen. But that's not a reason to give in to ideological, desire-based worldviews. We should keep crawling forward toward a better and better objective understanding of the world.

*"Lefty" is not meant as a pejorative here. I just don't have a word for the Marxist-influenced, left-leaning idea/ideology package that has become dominant in the humanities and soft social sciences, along with the methodology of critical theory. It is a very broad, complex, multi-faceted idea/ideology package with no commonly accepted over-arching name, so "lefty" will have to do for now. If you know a better term, let me know.

**Obviously this is a one-sided exercise, since I'm responding to my own summaries/paraphrasing of the opposite side. But that doesn't mean it's a straw man. The concepts and ideas I put forth will be summarized and paraphrased in your own mind, and next time you encounter someone who says something kinda-sorta like the arguments I describe, you can use your internally summarized and paraphrased arguments to think about the filtered, summarized, paraphrased versions of that other person's arguments that appears in your own mind. The purpose here is to present ideas, not to definitively win an argument or settle a point.

Monday, May 30, 2016

The incredible miracle in poor country development

The amazing improvement in the quality of life of the world's poor people should be common knowledge by now. For example, you have the now-famous "elephant graph", by Branko Milanovic, showing recent income growth at various levels of the global income distribution:

This graph shows that over the last three decades or so, the global poor and middle class reaped huge gains, the rich-country middle-class stagnated, and the rich-country rich also did quite well for themselves.

You also have the poverty data from Max Roser, showing how absolute poverty has absolutely collapsed in the last couple of decades, both in percentage terms and in raw numbers of humans suffering under its lash:

This is incredible - nothing short of a miracle. Nothing like this has ever happened before in recorded history. With the plunge in global poverty has come a precipitous drop in global child mortality and hunger. The gains have not been even - China has been a stellar outperformer in poverty reduction - but they have been happening worldwide:

The fall in poverty has been so spectacular and swift that you'd think it would be a stylized fact - the kind of thing that everyone knows is happening, and everyone tries to explain. But on Twitter, David Rosnick strongly challenged the very existence of a rapid recent drop in poverty outside of China. At first he declared that the poor-country boom was purely a China phenomenon. That is, of course, false, as the graphs above clearly show. 

But Rosnick insisted that poor-country development has slowed in recent years, rather than accelerated, and insisted that I read a paper he co-authored for the think tank CEPR, purporting to show this. Unfortunately this paper is from 2006, and hence is now a decade out of date. Fortunately, Rosnick also pointed me to a second CEPR paper from 2011, by Mark Weisbrot and Rebecca Ray, that acknowledges how good the 21st century has been for poor countries:
The paper finds that after a sharp slowdown in economic growth and in progress on social indicators during the second (1980-2000) period, there has been a recovery on both economic growth and, for many countries, a rebound in progress on social indicators (including life expectancy, adult, infant, and child mortality, and education) during the past decade. 
Weisbrot and Ray, averaging growth across country quintiles, find the following:

By their measure, the 2000-2010 decade exceeds or ties the supposed golden age of the 60s and 70s, for all but the top income quintile.

I'm tempted to just stop there. First of all, because Weisbrot and Ray are averaging across countries, rather than across people (as Milanovic does), China is merely one single data point among hundreds in their graph above. So the graph clearly shows that Rosnick is wrong, and the recent unprecedented progress of global poor countries is not just a China story. Case closed.

But I'm not going to stop there, because I think even Weisbrot and Ray are giving the miracle short shrift, especially when it comes to the 1980s and 1990s. 

See, as I mentioned, Weisbrot and Ray weight across countries, not across people:
Finally, the unit of analysis for this method is the country—there is no weighting by population or GDP. A small country such as Iceland, with 300,000 people, counts the same in the averages calculated as does China, with 1.3 billion people and the world’s second largest economy. The reason for this method is that the individual country government is the level of decision-making for economic policy. 
Making Iceland equal to China might allow a better analysis of policy differences (I have my doubts, since countries are all so different). But it certainly gives a very distorted picture of the progress of humankind. Together, India and China contain well over a third of humanity, and almost half of the entire developing world. 

And when you look at the 1980s and 1990s, you see that the supergiant countries of India and China did extremely well during those supposedly disastrous decades. Here's Indian real per capita GDP:

As you can see, during the 1960s, India's GDP increased by a little less than a third - a solid if unspectacular performance. From 1970 to 1980 it increased by perhaps a 10th - near-total stagnation. In the 1980s, it increased by a third - back to the same performance as the 60s. In the 1990s, it did even better, increasing by around 40 percent. And of course, in the 2000s, it has zoomed ahead at an even faster rate.

So India had solid gains in the 80s and 90s - only the new century has seen more progress in material living standards. Along with Indian growth, as you might expect, has come a huge drop in poverty (and here's yet more data on that).

Now China:

The 60s were a disaster for China, with GDP essentially not increasing at all. It's hard to see from the graph, but the 70s were actually great, with China's income nearly doubling. The 80s, however, were even better, with GDP more than doubling. The 90s were similarly spectacular. And of course, rapid progress has continues in the new century.

So for both of the supergiant countries, the 80s and 90s were good years - better than the 60s and 70s. Collapsing these billions of people into two data points, as Weisbrot and Ray do, turns these miracles into a seeming disaster, but the truth is that as go India and China, so goes the developing world. 

Now let's talk about policy.

My prior is that the 80s and 90s look bad for poor countries in Weisbrot and Ray's dataset - and the 70s look great - because of natural resource prices. Metals prices rose steadily in the 60s, surged in the 70s, then collapsed in the 80s and 90s:

Oil prices didn't rise in the 60s, but boy did they soar in the 70s and collapse in the 80s and 90s:

The same story is roughly true of other commodities.

My prior is that the developing world contains a large number of small countries whose main industry is the export of natural resources, and whose economic fortunes rise and fall with commodity prices. Just look at this map of commodity importers and exporters, via The Economist:

Yep. Most of the countries in the developing world are commodity exporters...with the huge, notable exceptions of China and India.

So I strongly suspect that Weisbrot and Ray's growth numbers are mostly just reflections of rising and falling commodity prices. Averaging across countries, rather than people, essentially guarantees that this will be the case.

Do Weisbrot and Ray recognize this serious weakness in their method? The authors mention commodity prices as an explanation for the fast developing-country growth of 2000-2010, but completely fail to bring it up as an explanation for the growth during 1960-1980. In fact, here is what they say:
[T]he period 1960-1980 is a reasonable benchmark. While the 1960s were a period of very good economic growth, the 1970s suffered from two major oil shocks that led to world recessions—first in 1974-1975, and then at the end of the decade. So using this period as a benchmark is not setting the bar too high. 
But the oil shocks, and the general sharp rise in commodity prices, should have helped most developing countries hugely, not hurt them! Weisbrot and Ray totally ignore this key fact about their "benchmark" historical period.

So I think the Weisbrot and Ray paper is seriously flawed. It claims to be able to make big, sweeping inferences about policy by averaging across countries and comparing across decades, but the confounding factor of global commodity prices basically makes a hash of this approach. (And I'm sure that's not the only big confound, either. Rich-country recessions and booms, spillovers from China and India themselves, etc. etc.)

As I see it, here is what has happened with poor-country development over the last 55 years:

1. Start-and-stop growth in China and India in the 60s and 70s, followed by steady, rapid, even accelerating growth following 1980.

2. Seesawing growth in commodity exporters as commodity prices rose and fell over the decades.

Of course, this means that some of the miraculous growth we've seen in the developing world since 2000 is also on shaky ground! Commodity prices have fallen dramatically in the last year or two, and if they stay low, this spells trouble for countries in Africa, Latin America, and the Middle East. Their recent gains were real, but they may not be repeated in the years to come.

But the staggering development of China and India - 37 percent of the human race - seems more like a repeat of the industrialization accomplished by Europe, Japan, Korea, etc. And although China is now slowing somewhat, India's growth has remained steady or possibly even accelerated.

So the miracle is real, and - for now, at least - it is continuing. 

Thursday, May 26, 2016

101ism, overtime pay edition

John Cochrane wrote a blog post criticizing the Obama administration's new rule extending overtime pay to low-paid salaried employees. Cochrane thinks about overtime in the context of an Econ 101 type model of labor supply and demand. I'm not going to defend the overtime rule, but I think Cochrane's analysis is an example of what I've been calling "101ism".

One red flag indicating that 101 models are being abused here is that Cochrane applies the same model in two different ways. First, he models overtime pay as a wage floor:

Then he alternatively models it as a negative labor demand shock:

Well, which is it? A wage floor, or a negative labor demand shock? The former makes wages go up, while the latter makes wages go down, so the answer is clearly important. If using the 101 model gives you two different, contradictory answers, it's a clue that you shouldn't be using the 101 model.

In fact, overtime rules are not quite like either wage floors or negative labor demand shocks. Overtime rules stipulate not a wage level, but a ratio between base wages and wages paid on hours worked per worker above a certain amount.

In the Econ 101 model of labor supply and demand, there's no distinction between the extensive and the intensive margin - hiring the same number of employees for fewer hours each is exactly the same as hiring fewer employees for the same number of hours each. But with overtime rules, those two are obviously not the same. For a given base wage, under overtime rules, hiring 100 workers for 40 hours each is cheaper than hiring 40 workers for 100 hours each, even though the total number of labor hours is the same. That breaks the 101 model.

With overtime rules, weird things can happen. First of all, base wages can fall while keeping employment the same, even if labor demand is elastic. Why? Because if companies fix the hours that their employees work, they can just set the base wage lower so that overall compensation stays the same, leading to the exact same equilibrium as before.

Overtime rules can also raise the level of employment. Suppose a firm is initially indifferent between A) hiring a very productive worker for 60 hours a week at $50 an hour, and B) hiring a very productive worker for 40 hours a week at $50 an hour, and hiring 2 less productive workers at 40 hours a week each for $25 an hour. Overtime rules immediately change that calculation, making option (B) cheaper. In general equilibrium, in a model with nonzero unemployment (because of reservation wages, or demand shortages, etc.), overtime rules should cut hours for productive workers and draw some less-productive workers into employment. In fact, this is exactly what Goldman Sachs expects to happen.

Now, to understand the true impact of overtime rules, we probably have to include more complicated stuff, like unobservable effort (what if people work longer but less hard?), laws regarding number of work hours, unobservable hours (since the new rule is for salaried employees), sticky wages, etc. But even if we want to think about the very most simple case, we can't use the basic 101 model, since the essence of overtime rules is to force firms to optimize over 2 different margins, and S-D graphs represent optimization over only 1 margin.

Using 101 models where they clearly don't apply is 101ism!

Monday, May 23, 2016

Theory vs. Evidence: Unemployment Insurance edition

The argument over "theory vs. evidence" is usually oversimplified and silly, since you need both to understand the world. But there is a sense in which I think evidence really does "beat" theory most of the time, at least in econ. Basically, I think empirical work without much theory is usually more credible than the reverse.

To show what I mean, let's take an example. Suppose I was going to try to persuade you that extended unemployment insurance has big negative effects on employment. But suppose I could only show you one academic paper to make my case. Which of these two papers, on its own, would be more convincing?

Paper 1: "Optimal unemployment insurance in an equilibrium business-cycle model", by Kurt Mitman and Stanislav Rabinovitch

The optimal cyclical behavior of unemployment insurance is characterized in an equilibrium search model with risk-averse workers. Contrary to the current US policy, the path of optimal unemployment benefits is pro-cyclical – positively correlated with productivity and employment. Furthermore, optimal unemployment benefits react nonmonotonically to a productivity shock: in response to a fall in productivity, they rise on impact but then fall significantly below their pre-recession level during the recovery. As compared to the current US unemployment insurance policy, the optimal state-contingent unemployment benefits smooth cyclical fluctuations in unemployment and deliver substantial welfare gains.

Some excerpts:
The model is a Diamond–Mortensen–Pissarides model with aggregate productivity shocks. Time is discrete and the time horizon is infinite. The economy is populated by a unit measure of workers and a larger continuum of firms...Firms are risk-neutral and maximize profits. Workers and firms have the same discount factor β...Existing matches [i.e., jobs] are exogenously destroyed with a constant job separation probability δ...All worker–firm matches are identical: the only shocks to labor productivity are aggregate shocks...[A]ggregate labor productivity...follows an AR(1) process...The government can insure against aggregate shocks by buying and selling claims contingent on the aggregate state...The government levies a constant lump sum tax τ on firm profits and uses its tax revenues to finance unemployment benefits...The government is allowed to choose both the level of benefits and the rate at which they expire. Benefit expiration is stochastic...

Paper 2: "The Impact of Unemployment Benefit Extensions on Employment: The 2014 Employment Miracle?", by Marcus Hagedorn, Iourii Manovskii, and Kurt Mitman

We measure the aggregate effect of unemployment benefit duration on employment and the labor force. We exploit the variation induced by Congress' failure in December 2013 to reauthorize the unprecedented benefit extensions introduced during the Great Recession. Federal benefit extensions that ranged from 0 to 47 weeks across U.S. states were abruptly cut to zero. To achieve identification we use the fact that this policy change was exogenous to cross-sectional differences across U.S. states and we exploit a policy discontinuity at state borders. Our baseline estimates reveal that a 1% drop in benefit duration leads to a statistically significant increase of employment by 0.019 log points. In levels, 2.1 million individuals secured employment in 2014 due to the benefit cut. More than 1.1 million of these workers would not have participated in the labor market had benefit extensions been reauthorized.

Some excerpts:
[W]e exploit the fact that, at the end of 2013, federal unemployment benefit extensions available to workers ranged from 0 to 47 weeks across U.S. states. As the decision to abruptly eliminate all federal extensions applied to all states, it was exogenous to economic conditions of individual states. In particular, states did not choose to cut benefits based on, e.g. their employment in 2013 or expected employment growth in 2014. This allows us to exploit the vast heterogeneity of the decline in benefit duration across states to identify the labor market implication of unemployment benefit extensions. Note, however, that the benefit durations prior to the cut, and, consequently, the magnitudes of the cut, likely depended on economic conditions in individual states. Thus, the key challenge to measuring the effect of the cut in benefit durations on employment and the labor force is the inference on labor market trends that various locations would have experienced without a cut in benefits. Much of the analysis in the paper is devoted to the modeling and measurement of these trends. 
The primary focus of the formal analysis in the paper is on measuring the counterfactual trends in labor force and employment that border counties would have experienced without a cut in benefits...The first one...allows for permanent (over the estimation window) differences in employment across border counties which could be induced by the differences in other policies (e.g., taxes or regulations) between the states these counties belong to. Moreover, employment in each county is allowed to follow a distinct deterministic time trend. The model also includes aggregate time effects and controls for the effects of unemployment benefit durations in the pre-reform period...The second and third models...reflect the systematic response of underlying economic conditions across counties with different benefit durations to various aggregate shocks and the heterogeneity is induced by differential exposure of counties to these aggregate disturbances. 

These two papers have results that agree with each other. Both conclude that extended unemployment insurance causes unemployment to go up by a lot. But suppose I only showed you one of these papers. Which one, on its own, would be more effective in convincing you that extended UI raises U a lot?

I submit that the second paper would be a lot more convincing. 

Why? Because the first paper is mostly "theory" and the second paper is mostly "evidence". That's not totally the case, of course. The first paper does have some evidence, since it calibrates its parameters using real data. The second paper does have some theory, since it relies on a bunch of assumptions about how state-level employment trends work, as well as having a regression model. But the first paper has a huge number of very restrictive structural assumptions, while the second one has relatively few. That's really the key.

The first paper doesn't test the theory rigorously against the evidence. If it did, it would easily fail all but the most gentle first-pass tests. The assumptions are just too restrictive. Do we really think the government levies a lump-sum tax on business profits? Do we really think unemployment insurance benefits expire randomly? No, these are all obviously counterfactual assumptions. Do those false assumptions severely impact the model's ability to match the relevant features of reality? They probably do, but no one is going to bother to check, because theory papers like this are used to "organize our thinking" instead of to predict reality.

The second paper, on the other hand, doesn't need much of a structural theory in order to be believable. Unemployment insurance discourages people from working, eh? Duh, you're paying people not to work! You don't need a million goofy structural assumptions and a Diamond-Mortensen-Pissarides search model to come up with a convincing individual-behavior-level explanation for the empirical findings in the second paper.

Of course, even the second paper isn't 100% convincing - it doesn't settle the matter. Other mostly-empirical papers find different results. And it'll take a long debate before people agree which methodology is better. 

But I think this pair of papers shows why, very loosely speaking, evidence is often more powerful than theory in economics. Humans are wired to be scientists - we punish model complexity and reward goodness-of-fit. We have little information criteria in our heads.

Update: Looks like I'm not the only one that had this thought... :-)

Also, Kurt has a new discussion paper with Hagedorn and Manovskii, criticizing the methodology of some empirical papers that find only a small effect of extended UI. In my opinion, Kurt's team is winning this one - the method of identifying causal effects of UI on unemployment using data revisions seems seriously flawed.

Friday, May 20, 2016

What's the difference between macro and micro economics?

Are Jews for Jesus actually Jews? If you ask them, they'll surely say yes. But go ask some other Jews, and you're likely to hear the opposite answer. A similar dynamic tends to prevail with microeconomists and macroeconomists. Here is labor economist Dan Hamermesh on the subject:
The economics profession is not in disrepute. Macroeconomics is in disrepute. The micro stuff that people like myself and most of us do has contributed tremendously and continues to contribute. Our thoughts have had enormous influence. It just happens that macroeconomics, firstly, has been done terribly and, secondly, in terms of academic macroeconomics, these guys are absolutely useless, most of them.
Ouch. But not too different from lots of other opinions I've head. "I went to a macro conference recently," a distinguished game theorist confided a couple of years back, sounding guilty about the fact. "I couldn't believe what these guys were doing." A decision theorist at Michigan once asked me "What's the oldest model macro guys still use?" I offered the Solow model, but what he was really claiming is that macro, unlike other fields, is driven by fads and fashions rather than, presumably, hard data. Macro folks, meanwhile, often insist rather acerbically that there's actually no difference between their field and the rest of econ. Ed Prescott famously refuses to even use the word "macro", stubbornly insisting on calling his field "aggregate economics".

So who's right? What's the actual distinction between macro and "micro"? The obvious difference is the subject matter - macro is about business cycles and growth. But are the methods used actually any different? The boundary is obviously going to be fuzzy, and any exact hyperplane of demarcation will necessarily be arbitrary, but here are some of what I see as the relevant differences.

1. General Equilibrium vs. Game Theory and Partial Equilibrium

In labor, public, IO, and micro theory, you see a lot of Nash equilibria. In papers about business cycles, you rarely do - it's almost all competitive equilibrium. Karthik Athreya explains this in his book, Big Ideas in Macroeconomics:
Nearly any specification of interactions between individually negligible market participants leads almost inevitably to Walrasian outcomes...The reader will likely find the non-technical review provided in Mas-Colell (1984) very useful. The author refers to the need for large numbers as the negligibility hypothesis[.]
Macro people generally assume that there are too many companies, many consumers, etc. in the economy for strategic interactions to matter. Makes sense, right? Macro = big. Of course there are some exceptions, like in search-and-matching models of labor markets, where the surplus of a match is usually divided up by Nash bargaining. But overall, Athreya is right.

You also rarely see partial equilibrium in macro papers, at least these days. Robert Solow complained about this back in 2009. You do, however, see it somewhat in other fields, like tax and finance (and probably others).

2. Time-Series vs. Cross-Section and Panel

You see time-series methods in a lot of fields, but only in two areas - macro and finance - is it really the core empirical method. Look in a business cycle paper, and you'll see a lot of time-series moments - the covariance of investment and GDP, etc. Chris Sims, one of the leading empirical macroeconomists, won a Nobel mainly for pioneering the use of SVARs in macro. The original RBC model was compared to data (loosely) by comparing its simulated time-series moments side by side with the empirical moments - that technique still pops up in many macro papers, but not elsewhere. 

Why are time-series methods so central to macro? It's just the nature of the beast. Macro deals with intertemporal responses at the aggregate level, so for a lot of things, you just can't look at cross-sectional variation - everyone is responding to the same big things, all at once. You can't get independent observations in cross section. You can look at cross-country comparisons, but countries' business cycles are often correlated (and good luck with omitted variables, too). 

As an illustration, think about empirical papers looking at the effect of the 2009 ARRA stimulus. Nakamura and Steinsson - the best in the business - looked at this question by comparing different states, and seeing how the amount of money a state got from the stimulus affected its economy. They find a large effect - states that got more stimulus money did better, and the causation probably runs in the right direction. Nakamura and Steinsson conclude that the fiscal multiplier is relatively large - about 1.5. But as John Cochrane pointed out, this result might have happened because stimulus represents a redistribution of real resources between states - states that get more money today will not have to pay more taxes tomorrow, to cover the resulting debt (assuming the govt pays back the debt). So Nakamura and Steinsson's conclusion of a large fiscal multiplier is still dependent on a general equilibrium model of intertemporal optimization, which itself can only be validated with...time-series data.

In many "micro" fields, in contrast, you can probably control for aggregate effects, as when people studying the impact of a surge of immigrants on local labor markets use methods like synthetic controls to control for business cycle confounds. Micro stuff gets affected by macro stuff, but a lot of times you can plausibly control for it.

3. Few Natural Experiments, No RCTs

In many "micro" fields, you now see a lot of natural experiments (also called quasi-experiments). This is where you exploit a plausibly exogenous event, like Fidel Castro suddenly deciding to send a ton of refugees to Miami, to identify causality. There are few events that A) have big enough effects to affect business cycles or growth, and B) are plausibly unrelated to any of the other big events going on in the world at the time. That doesn't mean there are none - a big oil discovery, or an earthquake, probably does qualify. But they're very rare. 

Chris Sims basically made this point in a comment on the "Credibility Revolution" being trumpeted by Angrist and Pischke. The archetypical example of a "natural experiment" used to identify the impact of monetary policy shocks - cited by Angrist and Pischke - is Romer & Romer (1989), which looks at changes in macro variables after Fed announcements. But Sims argues, persuasively, that these "Romer dates" might not be exogenous to other stuff going on in the economy at the time. Hence, using them to identify monetary policy shocks requires a lot of additional assumptions, and thus they are not true natural experiments (though that doesn't mean they're useless!). 

Also, in many fields of econ, you now see randomized controlled trials. These are especially popular in development econ and in education policy econ. In macro, doing an RCT is not just prohibitively difficult, but ethically dubious as well.

So there we have three big - but not hard-and-fast - differences between macro and micro methods. Note that they all have to do with macro being "big" in some way - either lots of actors (#1), shocks that affect lots of people (#2), or lots of confounds (#3). As I see it, these differences explain why definitive answers are less common in macro than elsewhere - and why macro is therefore more naturally vulnerable to fads, groupthink, politicization, and the disproportionate influence of people with forceful, aggressive personalities.

Of course, the boundary is blurry, and it might be getting blurrier. I've been hearing about more and more people working on "macro-focused micro," i.e. trying to understand the sources of shocks and frictions instead of simply modeling the response of the economy to those shocks and frictions. The first time I heard that exact phrase was in connection with this paper by Decker et al. on business dynamism. Another example might be the people who try to look at price changes to tell how much sticky prices matter. Another might be studies of differences in labor market outcomes between different types of workers during recessions. I'd say the study of bubbles in finance also qualifies. This kind of thing isn't new, and it will never totally replace the need for "big" macro methods, but hopefully more people will work on this sort of thing now (and hopefully they'll continue to take market share from "yet another DSGE business cycle model" type papers at macro conferences). As "macro-focused micro" becomes more common, things like game theory, partial equilibrium, cross-sectional analysis, natural experiments, and even RCTs may become more common tools in the quest to understand business cycles and growth.