Why is a prediction model with an accuracy rate of over 90% unreliable?

Before diving into the main topic, let's start with a small quiz.

There is a study published in the internationally renowned journal "Journal of Finance": Researchers investigated market timing strategies in the U.S. stock market. Every January, they would construct a model to predict the market's trend for the year, as measured by the S&P 500 index. If the model predicted an increase, they would go long on the market; if it predicted a decrease, they would go short.

The researchers tested this model for 22 years, and the results showed that the model correctly predicted the market direction 20 out of 22 times, with an accuracy rate of 91%. Moreover, when the model signaled an increase, the average annual return was 15.2%, and when it signaled a decrease, the average annual return was -10.9%. The results were statistically significant.

Of course, there is also an undeniable factor: the study was published in the "Journal of Finance" (one of the top three journals in the field of finance).

The question is, would you refer to this model to construct your own investment strategy?

If your answer is Yes, please continue reading. (Those who answered No can think about their reasons and then continue reading.)

Now, I will tell you that the predictions made by the entire model are based on the win-loss situation of the Super Bowl teams, such as an NFL (later NFC) team winning, the stock market will rise, and an AFL (later AFC) team winning, the market will fall.

Now, are you still willing to refer to this model for investment? Why?

If the answer is still Yes, how would you explain the relationship between the Super Bowl game results and the subsequent stock market trends?

Below is the answer section.The above statements are factual. The paper titled "An Examination of the Super Bowl Stock Market Predictor" was first published in June 1990 in the Journal of Finance. The paper was authored by Associate Professor Thomas Krueger from the University of Wisconsin-La Crosse and Associate Professor William Kennedy from the University of North Carolina at Charlotte. They offered two possible explanations for the relationship between the Super Bowl and stock market trends: one is that it is purely coincidental; the other is that if enough people believe in it, the phenomenon might actually exist. They observed the market reaction the day after the Super Bowl and found some support for the second hypothesis.

The study concluded that although the Super Bowl Indicator (SBI) was very accurate (up to 1988), "it cannot ensure that the predictive accuracy will continue, not to mention that the theoretical relationship between the Super Bowl Indicator and stock market trends is not apparent."

So, how has this predictive indicator performed after the publication of the paper?

More than twenty years later, Professor Bill Schmidt and Ronnie Clayton from Jacksonville State University continued the research on this Super Bowl Indicator and published a paper in 2017 in The Journal of Business Inquiry, updating the prediction results up to 2016. Their paper title was more fitting and straightforward: "Super Bowl Indicator and Equity Markets: Correlation Not Causation."

After completing the results from 2016 to 2023, it can be seen that in the subsequent 35 years of predictions, only 18 years were correct, with an accuracy rate of 51%, just slightly better than flipping a coin. In the years when the market was predicted to rise, it increased by 9.1%, and when the prediction was for a decline, the market rose even more, reaching 11.5%.

For most of the 57 years, the accuracy rate of the Super Bowl Indicator has been around 72%, and as the market indices increasingly deviated from this indicator, the correlation has also gradually weakened. However, an accuracy rate of over 70% is already enough for the Super Bowl Indicator to attract a following.

For a long time, this indicator has been a topic of endless debate among sports fans, economists, and investors, with some superstitious about its accuracy (although this is becoming less and less tenable), treating it as a financial oracle (even including Wall Street analysts), while others consider it mere coincidence and dismiss it with disdain.

One thing that can be certain is that the Super Bowl Indicator is not a valid stock market prediction indicator. However, it can serve as a classic case of "correlation does not imply causation."

In 1978, Leonard Koppett, a sportswriter for The New York Times, first proposed the hypothesis that the winning situation of the Super Bowl team is related to the direction of the stock market. For the 11 years prior to this and the 12 years following, the Super Bowl Indicator accurately predicted the market trend, with an accuracy rate that was legendary in its early days.Interestingly, the index Koppett initially used to measure market trends was not the S&P 500, but the New York Stock Exchange Composite Index (NYSE). After maintaining a completely accurate market tracking record for 17 years, a twist occurred.

In 1984, the contest results predicted the opposite of the market trend compared to the reference index, prompting Koppett to modify the measurement criteria.

He later wrote: "I did what any good statistician would do: I expanded my statistical base until I got the numbers I wanted." Thus, despite some minor inaccuracies that year, investors could still trust the Super Bowl indicator, as at least one of the NYSE, Dow Jones, or S&P indices would correspond to the result of the Super Bowl game.

Consequently, the adjusted Super Bowl indicator has an astonishing 96.8% accuracy rate in predicting the market over a "reliable sample size" of 31 years.

Koppett later added that he intended to emphasize the difference between causality and correlation, as he expressed in an interview: "My original intent was to satirize the flaws in human statistical reasoning; it's so foolish that it's unbelievable."

However, after the Super Bowl indicator's early high accuracy rate was continuously reported by the media, it evolved into a nationwide popular financial topic.

To investors in the 80s and 90s, Koppett inadvertently discovered one of the most accurate market prediction indicators in investment history. Over the years, the Super Bowl indicator has been supported or criticized by various studies and analysts, and even though its reliability is no better than flipping a coin, it remains a popular and interesting topic. Beyond the indicator's prediction of overall market trends, S&P Global Market Intelligence also provides more extensive and segmented market performance statistics based on the outcomes of the Super Bowl, which are equally fascinating.

For instance, the higher the score in the game, the better the stock market performs. S&P Global states: "When the combined score of the Super Bowl teams is at least 46 points, the average market return is 16.3%. If the score is below 46 points, the average market return is only 7.2%."

Additionally, the venue of the Super Bowl game is also related to the stock market's performance. When the Super Bowl is held west of the Mississippi River, the stock market performs poorly (8.6% on the east side, 14.3% on the west side), and when the game is held in a domed stadium (with a retractable roof), the stock market also tends to perform poorly.

Setting aside the far-fetched causal relationships and conclusions that seem like jokes, we should actually focus more on what makes the Super Bowl indicator unreliable.Sample Size: The Super Bowl Indicator is based on a relatively small sample size, making it susceptible to random fluctuations and outliers. For instance, if the indicator correctly predicts the market direction for several consecutive years, it is easy for people to assume it possesses some predictive power. Conversely, a few incorrect predictions can easily lead to the opposite conclusion.

Economic Conditions: The Super Bowl Indicator assumes that the stock market is a reflection of the overall economy. While this is often the case, there can be divergences between the two. For example, if investors believe a recovery is imminent, the stock market may perform well during a recession. Similarly, if investors anticipate a recession, the market may underperform during prosperous times.

Media Attention: The extensive media coverage of the Super Bowl Indicator could influence its accuracy. For example, if more people become aware of the indicator and its purported predictive abilities, they may be more likely to invest based on its suggestions, potentially skewing the market in a certain direction. This is one explanation for its predictive accuracy as discussed in previous papers.

Cherry Picking Fallacy: Proponents of the Super Bowl Indicator often focus only on data that supports their theory while ignoring any data that contradicts it. This is a common issue in many fields of research and can lead to inaccurate conclusions.

Super Bowl Game Outcomes: The Super Bowl Indicator uses the results of specific team matchups to predict market trends, and each game's outcome is influenced by countless variables. For example, an injury to a star player before the game could significantly affect the game's result and subsequent predictions about market trends.

In summary, the Super Bowl Indicator may be an interesting concept and hypothesis, but its role in predicting the stock market can at best be considered for entertainment purposes, serving at most as a way to make investing more interesting. More importantly, it serves as a reminder that one should be cautious about assuming correlation implies causation before drawing conclusions about any phenomena that appear to be correlated. The Super Bowl Indicator is one of many alternative (non-scientific) market prediction indicators and is among the most famous. Other similar sports indicators include the Triple Crown horse races in the United States, Major League Baseball, the Tour de France, and so on.

Just as ancient people studied the entrails of sheep for divination, some more unconventional market predictors look for signs of market trends in lipstick shades, cardboard box production, the price of Big Mac hamburgers, the length of women's skirts, and the covers of the Sports Illustrated Swimsuit Issue. There are so many of these indicators that they could fill an entire article on their own.

Leave a reply

Your email address will not be published. Required fields are marked *