Courtesy of Mish.
Salil Mehta at Statistical Ideas commented on my recent post that compared the Atlanta Fed GDPNow model to the FRBNY Nowcast model.
Mehta notes that both models claim accuracy within one percentage point. However, that’s no longer mathematically possible given the difference between the two models is 2.3 percentage points.
Rogue one: Faithful GDP nowcasts by Salil Mehta
There is a 2/3 chance that both competing Federal Reserve 2017 Q1 GDP nowcasts are wrong! That’s an audacious prediction for the storied NY and Atlanta institutions (one of them led by my former big boss Timothy Geithner), and yet there is no way around the current confusion they are in. This is also critically important as one is showing a robust 3.2% growth reading, while the other is at 0.9% (the 2nd lowest reading in nearly 3-years) and essentially indicates that we are descending towards recession.
Are we descending towards recession? While we don’t forecast that, we certainly think there is only a single digit probability of a >3% GDP. How could the NY Fed plausibly give such a madly high estimate (which if true would be the second highest in 2-years)? Yet, there you have it, two extreme readings, and a 2.3% (3.2%-0.9%) chasm between them.
We show here that the Federal Reserve’s conclusions are somewhat ridiculous, though shouldn’t be since they impact the open market committee monetary decisions that the world looks to. And there are humbling lessons from these nascent Big Data, overfit models.
The chart here shows some basic information regarding the current GDP nowcasts. As we via the two blue bars, we have the Atlanta nowcast on the left (the bar was recently as high as 3.4% earlier this year). And the NY nowcast on the right (the bar was recently as low as 1.5%). That’s right, both nowcasts passed each other, while aggressively moving further in the opposite direction! The large swings in each are also doubtful, given each nowcast’s eventually advertised, margin of error. For a good chronology of these nowcast reports, refer to MishTalk.
Each nowcast boasts a margin of error of just ~1%, and this clearly poses a problem since the average of these two nowcasts (shown in orange at 2.1%) is clearly outside of both the Atlanta and the NY stated margin of error! As supportive reference, we also show (in green) that the current 2016 Q4 GDP is nearby at 1.9%. Now we should ask some important questions about how we keep getting into more strange nowcasts in the past year that they both have operated. The first thing to appreciate is that the nowcasts are supposed to predict very tight errors that are uncorrelated to the variance in the actual GDP itself. And good nowcasts should have errors independent of one another, except since the NY and Atlanta Fed operate independent of one another there is a good chance that there may be some modeling similarities. We modestly assume this and derive through the variance formula (VarianceAtlanta+VarianceNY+2σAtlantaσNYρAtlanta,NY) that the margin of error of the difference between the models is just less than 1% (silver vertical interval arrows in chart above). This is a highly plausible tight expected variance. Sample size is also trivial here as we don’t have the true expectation to model a limit from. And with this, the probability of seeing an inadvertent 2.3% difference between the two correct Federal Reserve models is <5%. Or that their publicized margin of error is awkwardly too low (to the point we’ll show that randomly guessing the GDP would be safer).
