Using big data in finance: Example of sentiment-extraction from news articles
Nitish Sinha
FEDS Notes, March 26, 2014
There is much discussion and research in finance on using “big data” to understand market “sentiment.” In this note, I will draw on some of my own research in behavioral finance–Sinha (2010) and Heston and Sinha (2013)–to share my perspective the current state of affairs in this area, particularly on the meaning of “sentiment” in the context of big data research.1
The Meaning of “Sentiment” among Finance and Computer Science Researchers
Let me start with some possible confusion that might be caused by how a simple word is used quite differently in two disciplines that meet in this particular area–Finance and Computer Science. In finance, the word “Sentiment” is generally understood to be irrational belief about future cash flows.2 The key test of sentiment is outlined in Tetlock(2007) where he points out that “The sentiment theory predicts short-horizon returns will be reversed in the long run, whereas the information theory predicts they will persist indefinitely.” In other words, “sentiment” is a good short-term contrarian indicator. In finance, sentiment has been measured using sunshine days in New York City, and wins in soccer games among others.3 The common feature these measures share is that short-term positive returns associated with these sentiment measures tend to be reversed over next few days.
Computer scientists and computational linguists use the word “sentiment” differently. For example, Pang and Lee (2008) explain that over time “sentiment” has morphed into opinion or subjective information. It is not clear, whether market participants’ opinion should be non-informative. There are many places where investors express their opinions. Of course, traded price is where the rubber meets the road and people vote with their bank accounts. But it is possible that somebody with informed opinion, and perhaps thinner wallet might lose to somebody else with uninformed opinion and a thicker wallet.4
The other venues where investors express opinions are newspaper articles, news wires, op-eds, and now twitter feeds. In “Underreaction to News in the U.S. Stock Market,” I explore “opinion-extraction” and find that market prices tend to under-react to textual information appearing in news articles. This finding differs from other related findings in the finance literature such as Tetlock(2007) and Laughran and McDonanld(2010). In “News versus Sentiment”, a co-authored paper with Steve Heston at University of Maryland, we set out to find why it is that some evidence points to the market under-reacting to information contained in news articles, while other evidence points to the market properly reacting to the same type of information, and still other evidence suggests a tendency for the market to overreact. I will talk a little bit more about this paper since it also provides a window into an approach to working with big data.
One reason for apparently disparate results on the reaction of market prices to textual information could be that researches have used different texts to start with. Press releases are written by firms themselves and might not be impartial. Journalists are trained to write in impartial fashion and could blunt the opinions in a deliberate manner. Tweets are small in length, although can be quite opinionated dense in information. Tweets also present a challenge in parsing since tweeters often use a different vocabulary. We use common corpus of news articles for all text-processing techniques that allows us to specifically identify the effect of text-processing technology. In my research with Heston, we chose all news articles written by Thomson Reuters journalists because news from a wire-service news has some distinct advantages. First, wire-services cater primarily to the investment needs of their subscribers; stories do not appear because they are catchy but because they are economically relevant.5 Also at a wire-service, news does not need to out-compete all the other news articles to get published, a potential source of trunctation of information in newspapers. Comedian Seinfeld is attributed to have said, “It’s amazing that the amount of news that happens in the world every day always just exactly fits the newspaper.” It is also somewhat surprising the news that does not find its way into the newspaper since the newspaper was already full.6 Given all these conditions, we expect wire-services to have fewer of these biases. Thomson Reuters itself provides a measure of the tone of the news articles based on a neural network application on top of some linguistic analysis. (Please consult Sinha(2010) for the methodological details.)
A Machine-Learning Algorithm for Classifying News
Perhaps a digression into the area of “machine learning” might be useful here, since it is potentially relevant to extracting “sentiment” from big data. I will somewhat oversimplify some of the issues, at the risk of maintaining brevity. Machine learning used in the context of big data is really a method for classifying data into different categories. Machine learning can be further subdivided into supervised and un-supervised learning. In an unsupervised learning method, the researcher lets the data dictate the categories, the data will be classified into. A trivial example would to let as many categories as there are observations. The methodology of supervised machine learning can be thought of as comprising three steps–“Tag”, “Train,” and “Classify.” In the case of classifying the textual information in news articles, the first stage, “Tag,” requires the researcher to carefully select some articles considered likely to representative of the broader group that will need to be classified for the project. Then she tags the articles into desired categories, “good news” and “bad news” for example. There is some temptation to tag the article as positive, if the return following the article’s publication is positive, and negative if the return following the article’s publication is negative. Similarly, one may want to identify some article as pertinent, if there was any market reaction following the news article’s publication and not relevant if there was no reaction to the news article. Those tags would capture how market reacted to the news article, in light of all of the other information the market had at the time of news article release. For example, if a news article was written on a Friday or a day in which the market was otherwise engaged with thoughts of sunny days or Super Bowl, the research would likely run the risk of classifying the article as not even pertinent.7 In our case tagging was done by humans, one of the many reasonable classifiers. The second step or Training follows the tagging; the scientist feeds these tagged articles to the classifier while holding out some tagged articles. The classifier “learns” or establishes a relationship between article attributes and its category. The researcher tests the classifier on the held-out tagged articles till she is satisfied with the learning process. Once the machine has learned the relationship between document attributes and document category, it is ready to classify all the articles in the corpus in the final, “Classification” stage.
Heston-Sinha’s Appliction of the Machine Learning Algorithm
In our case, the machine provides us with the probability of an article being “positive,” “negative,” and “neutral.” The three probabilities sum to 1. We also score all articles, about 1 million in total, using two other popular methods in finance. The first one involves counting the proportion of positive and negative words using Harvard IV dictionary.8 Loughran and McDonald (2011) note that many words that are negative in common English language do not have such associations in Finance. Two simple examples are the words “Liability” and “Cancer.” They propose a dictionary tailored for finance. We also count the proportion of positive and negative words using the Loughran and McDonald dictionary.
We find that when the degree of positive “net” sentiment is measured as the difference in proportion of positive and negative words using the Harvard IV dictionary, it does predict price increases on the day of those news releases, but the initial price reaction reverts over the next couple of days to annul the effect of the news. This pattern suggests the market initially over-reacts to sentiment. We also find that when sentiment is measured as the difference in proportion of positive and negative words using the finance-specific dictionary, it also affects prices on the day of news release, but in this case prices do not tend to revert in subsequent days. This pattern suggests that news articles contain “durable” information and the market very quickly incorporates that information. Finally, we find that the neural-network based sentiment, measured as the difference of the probability of the news article being positive and negative, tends to affect prices on the day of news release, then continues to do so over next 13 weeks. This pattern suggests that news articles contain information that only gradually finds its way into prices. In other words, the market tends to underreact to the textual information.
Overall, our research confirms that news articles frequently include market-relevant information. Such information is somewhat complex and it is possible to gleam that by using a sophisticated analytic tool, simple information (such as those that can be measured by a domain-specific dictionary) is efficiently incorporated into price. We also provide an interesting technique to parse the information content of a continuous trading signal through a multivariate regression method. Using the method, we demonstrate that there is a significant “no-news” effect, stocks with news have different return pattern than those without news. Hence, when examining the “sentiment effect”, it is important to parse out the news effect; the details are in our working paper.
That brings us back to the potential pitfalls of using big data. There is a lot of information out there, but so is a lot of noise. As a scientist, one has to be careful with technique and introspect on what the data is telling you. With that let me propose three broad categories to think about using big data in finance.
- What opportunities do you see for the application of big data and/or machine learning in investment management?
- How do you deal with big data as a researcher?
- What do you think are some challenges in using big data in investment management?
References:
Baker, Malcolm and Jeffrey Wurgler, 2007. “Investor Sentiment in the Stock Market”, Journal of Economic Perspectives, vol. 21(2), pages 129-152.
DellaVigna, Stefano and Joshua Pollet, 2009. “Investor Inattention and Friday Earnings Announcements”, Journal of Finance, April 2009, vol. 64(2), pages 709-749.
Edmans, Alex, Diego García and Øyvind Norli, 2007. “Sports Sentiment and Stock Returns”, Journal of Finance, vol 62(4), pages 1967-1998.
Eisensee, T., & Strömberg, D.,2007. “News droughts, news floods, and US disaster relief”, Quarterly Journal of Economics, vol. 122(2), pages 693-728.
Heston, Steven L. and Sinha, Nitish Ranjan, 2013. “News versus Sentiment: Comparing Textual Processing Approaches for Predicting Stock Returns”, Robert H. Smith School Research Paper. Available at SSRN: http://ssrn.com/abstract=2311310 or http://dx.doi.org/10.2139/ssrn.2311310
Hirshleifer, David. 2001. “Investor psychology and asset pricing”, Journal of Finance Vol 56(1), pages 1533–98.
Hirshleifer, David and Tyler Shumway, 2003. “Good Day Sunshine: Stock Returns and the Weather”, Journal of Finance, vol. 58(3), pages 1009-1032
Kamstra, Mark J., Lisa A. Kramer, and Maurice D. Levi. 2003. “Winter Blues: A SAD Stock Market Cycle”, American Economic Review, vol. 93(1), pages 324-343.
Loughran, Tim and McDonald, Bill, 2011. “When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10-Ks”, Journal of Finance, vol. 66(1), pages 35-65.
Mullainathan, S., and A. Shleifer, 2005, “The market for news”, American Economic Review, vol. 95, pages 1031-1053.
Pang, Bo and Lee, Lilian, 2008, “Opinion Mining and Sentiment Analysis”, Foundations and Trends in Sentiment Retrieval. vol 2(1), pages 1-135.
Shleifer, Andrei and Vishny, Robert W, 1997. “The Limits of Arbitrage”, Journal of Finance, vol. 52 (1), pages 35-55.
Sinha, Nitish Ranjan, 2012, “Underreaction to News in the US Stock Market”, Available at SSRN: http://ssrn.com/abstract=1572614 or http://dx.doi.org/10.2139/ssrn.1572614
Tetlock, Paul C., 2007. “Giving Content to Investor Sentiment: The Role of Media in the Stock Market”, Journal of Finance, Vol. 62(3), pages 1139-1168.
1. Due to paucity of space, I am not exhaustive but will urge you to see the references in Sinha (2010) and Heston and Sinha (2013). Return to text
2. See Hirshleifer (2001), Baker and Wurgler (2007), and Tetlock (2007) for representative literature. Return to text
3. See Kamstra, Kramer, and Levi (2003) and Hirshleifer, and Shumway (2003) for evidence on effect of sunshine on equity prices; Edmans, Garcia and Norli (2007) for sports sentiment. Return to text
4. In finance, this idea is expressed as limits of arbitrage. See for example Shlifer and Vishny (1997). Return to text
5. See Mullainathan and Shleifer (2005), for a formal model of catering by newspapers. Return to text
6. Eisensee and Strömberg (2007) provide an empirical example of media selectivity based on what media perceived to be its consumers’ preference. Return to text
7. See Dellavigna and Pollet (2009) provide an illustration of distracted investor behavior. Return to text
8. The dictionary takes commonly used words in the English language and identifies them in 70 plus categories. It is described in detail in Tetlock (2007). Return to text
Disclaimer: FEDS Notes are articles in which Board economists offer their own views and present analysis on a range of topics in economics and finance. These articles are shorter and less technically oriented than FEDS Working Papers.
