Bad Economics: NYC comptroller can't tell causation from correlation

Part of a series debunking incorrect economics on the internet
TL;DR: NYC Comptroller makes an undergraduate mistake. Result gets publicized in Bloomberg.
Intended readership: I assume you took some undergrad statistics.

Bloomberg reports that airbnb cost NYC renters some ridiculous amount of money, which they got from this study by the NYC comptroller. This is an exceptional case of schizophreny on Bloomberg’s part, because not two weeks before, their best economics columnist laid out the case for the exact opposite.

The question is: how can we arrive at situation where the NBER finds no significant effect and the comptroller finds that Airbnb is responsible for 9.2% of NYC rent increase [footnote] Not a joke. Read their paper [/footnote]

Adam Millsap already laid the case why the Comptroller’s study is a newbie error. To reiterate: the comptroller estimates a panel regression (per year and neighborhood) with rent level on the left hand side and airbnb listings as well as a few controls on the right hand side.

Of course, both NYC rents and airbnb listings have year-over-year growth trends, so they both predict each other very well (even at the neighborhood level, by being a proxy for how trendy/touristy the neighborhood is). Note that the fact that they control for neighborhood-level attributes doesn’t matter here, because those trends are common to many neighborhoods and as such will still get picked up in the regression coefficient common to the neighborhoods – that is, the airbnb listings.

Interestingly, Airbnb listings seem to have a large correlation with housing price growth in this naive regression model regardless of who runs it. The NBER paper also gets this result before correcting for it with an instrument variable approach. This paper finds the same result when using distance from airbnb listing instead of neighborhood groupings. Only the NBER paper tries to correct for this by using google search popularity as an instrument to correct for these trends.

Lastly, rent levels don’t adapt instantly to shocks like an increase in Airbnb units. It would be interesting to see if some of the results on “previous year” lagged explanatory variables (including the IV result in the NBER study). [footnote] I’m aware it takes a bit more effort to get lagged variables to work in a panel data model, but it’s well within the realm of possible [/footnote]