Aggregate News Sentiment and Stock Market Returns in India

Abstract

This paper contributes to the advancement of noise trader theory by examining the connection between aggregate news sentiment and stock market returns during days of significant stock market movement. In contrast to previous studies that solely focused on company-specific news sentiment, this research explores the impact of aggregate news sentiment. To draw conclusions, GARCH modeling, regression analysis, and dictionary-based sentiment analysis are employed. The findings, based on data from India, reveal that aggregate news sentiment has a short-lived influence, with notable effects stemming from the business and politics categories.

1. Introduction

Stock market research has evolved from solely relying on quantitative, data-driven methodologies to methodologies that focus on analyzing unstructured text available in news media and social media. This evolution in stock market research has occurred due to global stock exchanges operating electronically and investors utilizing digital information from news and social media platforms. As a result, researchers have been motivated to determine whether the content read by investors on these platforms impacts the stock market. Moreover, some recent studies have also revealed a substantial change in the behavior of market participants across global financial markets.

For instance, Aramonte and Avalos (2021) reported the growing role of individual investors leading to a rise in trading volumes by speculating on companies through options trading and relying on social media platforms to update their portfolio strategies. Ülkü et al. (2023) studied the participant structure of the Asian stock markets during the COVID-19 pandemic. They found that individual investors’ buying was driven by their contrarian behavioral traits and self-serviced investing in stocks due to their work-from-home lifestyle along with the monetary and fiscal policy support from Governments and Central banks around the world during the pandemic. This increased participation of individual investors in global financial markets and dependence on news and social media signals need to be explored further to examine misinformation’s potential influence on market dynamics and accordingly design risk management strategies (see Youssef et al. 2021; Kashyap and Stein 2023).

Numerous studies have revealed the influence of news and social media sentiment related to companies on stock variables, such as returns and volatility (Chowdhury et al. 2014; Ferguson et al. 2011; Kräussl and Mirgorodskaya 2014; Ranco et al. 2015; Zhang and Skiena 2010). While there has been extensive research on news sentiment concerning company-specific events, there has been less focus on political, macroeconomic, international, and other similar types of events (Brans and Scholtens 2020; Ebong and George 2021). Furthermore, investors do not read a single news article in isolation; rather, they read a collection of news articles from different categories, each with its own sentiment (Duan et al. 2021). There is a scarcity of literature on this view, which necessitates further investigation.

This paper addresses the following research questions: (a) “Does aggregate news sentiment influence the stock market?” and (b) “Do news sentiments related to business, economy, politics, and international categories influence extreme stock market returns?” With the noise trader approach proposed by De Long et al. (1990) as a backdrop, it posits that noise traders are influenced by aggregate news sentiment, which has the maximum impact when arbitrageurs face difficulties in trading against them, ultimately driving prices to extreme levels. Since this aggregate news sentiment is induced by news published across different categories of events, it is likely to affect a wide variety of stocks. Therefore, instead of examining the influence on individual stocks, it is more appropriate to analyze the influence of aggregate news sentiment on a portfolio of stocks. Accordingly, this study examines the influence of aggregate news sentiment on the returns of the NIFTY index, a diversified portfolio widely used as a benchmark index representing 62% of the Indian market (Source: https://www.nseindia.com, accessed on 6 August 2023).

To obtain aggregate news sentiment, we collect news articles from the archives of the news portal economictimes.indiatimes.com (accessed on 8 July 2023), specifically those related to the categories of business, economy, politics, and international news. We form an observation window of five trading days centered around the day of extreme stock market movement. The influence of aggregate news sentiment on the stock market is examined for each day within the observation window to understand the pattern of influence. Additionally, the news sentiment of each news category is analyzed to determine its impact on stock market returns.

The findings reveal that aggregate news sentiment significantly influences NIFTY returns on the day of extreme market movement and the day preceding it. The strength of this influence is most pronounced on the day of extreme market movement. Furthermore, the impact of news sentiment from the business and politics categories is found to be significant. Importantly, the research demonstrates that the influence of aggregate news sentiment is short-lived, which suggests that long-term investors need not panic. However, intraday traders or options traders can benefit from this knowledge by gauging the sentiment polarity of aggregate news sentiment from business and political news, allowing them to trade cautiously and potentially profit from anticipating market movements.

The main contribution of this research has been in providing evidence of the influence of aggregate news sentiment on the Indian stock market during its extreme movement. This further enriches the literature on noise trader theory by uncovering how aggregate news sentiment interacts with noise trader beliefs, influencing their sentiment and ultimately driving prices to extreme levels.

2. Theoretical Background

2.1. Efficient Market Hypothesis

The Efficient Market Hypothesis (EMH) proposed by Fama (1965) was one of the most popular classical finance theories that discussed the relationship between information and stock prices. According to the EMH, all information is reflected in stock prices if the market is efficient, and hence no opportunity exists for market participants to earn excess profits in the long run. This includes past prices, public and private (monopolistic) information. Despite being highly appreciated by researchers at the time, the EMH had its share of drawbacks, one of which was its inability to account for extreme deviations in stock prices. The EMH never considered the role of investor sentiment in stock pricing, which was contested by Baker and Wurgler (2007), Barber and Odean (2008), De Bondt and Thaler (1985), and many others.

2.2. Noise Trader Theory and the Behavioral Biases in Investors

The Noise Trader Theory, proposed by Black (1986), De Long et al. (1990), and Shleifer and Summers (1990), stands out as one of the prominent behavioral finance theories explaining extreme deviations in stock prices. This theory highlights two crucial factors, i.e., sentiment and limits to arbitrage as significant contributors to such deviations from the intrinsic value. It categorizes investors into two distinct groups: informed investors, known as arbitrageurs, and uninformed investors, referred to as noise traders. While noise traders are subject to noise in the form of sentiment or trends in prices, arbitrageurs are not (Alfano et al. 2020). Apart from the fundamental risk, arbitrageurs also bear noise trader risk (De Long et al. 1990), which deters them from aggressively betting against noise traders. This limits the size of positions arbitrageurs take in the market because of which asset prices significantly deviate from fundamental values even in the absence of fundamental risk.

Although the noise trader theory provides an understanding of the process behind significant price deviations, the underlying reasons why noise traders tend to make decisions influenced by their sentiments can be attributed to the works of Barber and Odean (2008), Barberis et al. (1998), Bikhchandani and Sharma (2000), Fenzl and Pelzmann (2012), Shleifer and Summers (1990), and Tversky and Kahneman (1974). These studies identified various psychological and sociological biases, such as conservatism and representativeness bias, as significant factors contributing to market phenomena like underreaction, overreaction, momentum, and more.

2.3. News Sentiment and Related Literature

While investor sentiment studies discussed the role of the inherent beliefs of investors in stock price movement, Tetlock (2007) focused on an external factor—pessimism—as a measure of investor sentiment and revealed that elevated levels of media pessimism serve as a predictor of downward pressure on market prices, followed by a subsequent reversion to fundamental values. The idea of media pessimism and optimism was further extended in the form of media sentiment by Ferguson et al. (2011), Ranco et al. (2015), and Zhang and Skiena (2010) using news, blogs, and tweets, respectively. Chari et al. (2017) defined media sentiment as an implicit mood contained in the content of media which has the potential to change the investor’s psychological bias towards a stock or an index at a given point in time. Sentiment analysis on the content of news media is referred to as news media sentiment or simply news sentiment (Alfano et al. 2020; Mo et al. 2016; Uhl et al. 2015). Corbet et al. (2020) analyzed the relationship between the largest cryptocurrency returns and the economic shock resulting from negative sentiment related to COVID-19 pandemic and found that cryptocurrency returns are significantly influenced by negative sentiment of COVID-19 pandemic. Biswas et al. (2020) examined the influence of the COVID-19 pandemic on stocks in India using views and opinions posted on Reddit (a US-based aggregator of social news and discussions) of shares and found that there is a firm correlation between news on the pandemic and movement of share prices in stock markets. Ali et al. (2022) derived Twitter-based sentiments during the COVID-19 pandemic and found that negativity in the market sentiment has a positive and significant impact on gold-backed Islamic cryptocurrency OneGram Coin (OGC), indicating increased investor attention toward this cryptocurrency during negative sentiment in the market.

News media plays an important role in carrying information to investors about events having relevance to the stock market.

Approaches for capturing news sentiment: Medhat et al. (2014), in their review of different approaches in sentiment analysis, found two broad approaches being used in sentiment analysis, i.e., machine learning approaches and lexicon-based approaches. Guo et al. (2016) found naïve Bayes, support vector machine, and neural network among the most popular machine learning techniques used for sentiment analysis by contemporary researchers. In a study conducted by Nemes and Kiss (2021), they utilized BERT as the baseline technique for sentiment analysis. The researchers compared the impact of sentiment analysis results from three other methods, namely VADER, TextBlob, and recurrent neural network (RNN), on stock changes during the same period and found that the RNN model outperformed the other techniques and yielded results that were comparable to those obtained using BERT. While machine learning algorithms require labeled data, lexicon-based methods do not require labeled data but make use of a predefined dictionary to obtain the count of words with positive and negative valence from review text. Some popular dictionaries often used for research on sentiment analysis using lexicons include the Harvard General Inquirer (Stone and Hunt 1963), the MPQA (Multi-Perspective Question Answering) Subjectivity lexicon (Wiebe et al. 2005), Henry’s financial sentiment dictionary (HE) (Henry 2008), Loughran-McDonald financial sentiment dictionary (LM) (Loughran and McDonald 2011), etc. Ferguson et al. (2011), Kräussl and Mirgorodskaya (2014), and Mao et al. (2011) utilized the LM dictionary, while Alfano et al. (2020), Jandl et al. (2014), Sadique et al. (2008), and Taylor and Keselj (2020) opted for the HE dictionary.

Prior research on the influence of news sentiment on the stock market: Ferguson et al. (2011) found that measures of positive and negative media sentiment have significant relationships with the stock returns of UK companies on the day news articles are published. Additionally, they found that return predictability is inherent in negative news sentiment the day following the publication of news articles which is weaker compared to the one on the day of the news release. Chowdhury et al. (2014) using relevant real-time news headlines and press releases predicted sentiment and showed that there is a 67% correlation between positive sentiment curve and stock price trend. Zhang and Skiena (2010) designed a market-neutral strategy based on sentiment to predict the company’s stock trading volumes and financial returns using company-related blogs and news. They observed several differences between sentiments obtained from news and blogs. One of the key differences is that the sentiment conveyed by blogs lasts longer than the news sentiment.

Table 1. News sentiment research related to the stock market.

Author	Sentiment Analysis Approach	Methodology	Findings
Gidofalvi and Elkan (2001)	Naïve Bayesian classifier	Linear regression	Predictable stock price movement 20 min before and after news release.
Cahan et al. (2013)	Thomson Reuter News Analytics	Linear regression	Overreaction to earnings surprises with positive media sentiment, stronger in hard-to-value firms.
Mo et al. (2016)	SentiWordnet dictionary (lexicon-based)	Regression models, VAR model, Granger causality test	Lag-5 effect of news sentiment on market returns; lag-1 effect of market returns on news sentiment.
Heston and Sinha (2017)	Thomson Reuters NewsScope Data (neural network-based)	Cross-sectional regression	Daily news predicts stock returns for 1–2 days; Weekly news for one quarter. Negative news reaction is delayed.
Huynh and Smith (2017)	Thomson Reuter News Analytics	Cross-sectional regression	Market underreacts to good news, driving weekly momentum returns; similar findings in international markets.
Shi and Ho (2021)	RavenPack Dow Jones News Analytics	MRS-FIGARCH model, discrete choice models	MRS-FIGARCH outperforms other models; news sentiment affects likelihood of intraday stock return volatility states.
Kabbani and Usta (2022)	Not specified, uses overall sentiment score	Machine learning models (logistic regression, random forest, gradient-boosting machine)	Random forest model outperforms others with 63.58% accuracy.
Fazlija and Harder (2022)	NLP using BERT models	Sentiment scores in random forest classifier	Sentiment scores based on news content predict stock price direction effectively.

Source: Compiled by the researchers.

reports the findings of other authors who examined the influence of news sentiment on stock variables.

The overarching theme of research articles we reviewed covered the influence of sentiment on stock returns and financial market behavior. This sentiment is derived from various sources, including institutional and individual investors, news media, and social media. The impact of institutional investor sentiment on aggregate stock returns was investigated by Gao et al. (2021). Their study demonstrated that excessive optimism or pessimism among institutional investors significantly influences aggregate stock returns. This finding aligns with the study by Smales (2016), who examined the time-varying relationship between news sentiment, investor sentiment, and speculator positions in U.S. energy futures markets, illustrating the dynamic interplay between these factors. The role of media sentiment in predicting stock returns and volatility was a recurrent theme as well. For instance, Hsu et al. (2021) investigated the effect of news sentiment on stock market volatility, while Dong et al. (2022) explored the impact of media sentiment on stock prices. Both studies provide empirical evidence supporting the predictive power of media sentiment on market behaviors. The impact of news sentiment on stock returns was further explored in the context of specific events or sectors. Case and Clements (2021) investigated the impact of news sentiment on day-ahead spot electricity prices, while Rahadian and Nurfitriani (2022) studied the impact of news related to COVID-19 on stock market returns. These studies highlight that the impact of news sentiment can be sector-specific and can be influenced by particular events. The sentiment derived from social media was another critical factor discussed in the dataset. Fekrazad et al. (2022) investigated the link between social media sentiment and the stock market, revealing a significant influence of social media sentiment on stock market movements. The advent of advanced computational techniques has allowed researchers to explore the application of deep learning models in predicting stock market movements based on sentiment analysis. For instance, Halder (2022) proposed a deep-learning-based model to predict stock prices using sentiment analysis, while Rekha and Sabu (2022) presented a cooperative deep learning model for stock market prediction using news sentiment. These research papers collectively illustrate the pervasive influence of sentiment derived from various sources on stock returns and financial market behaviors. The methodologies employed range from econometric modeling to advanced machine learning techniques, underlining the interdisciplinary nature of this research area.

The literature review presented here also indicates that both classical finance and behavioral finance consider information as an important factor in explaining the movement of the stock market. While EMH explained the stock market movement without the role of investor sentiment, it had to be augmented by using behavioral finance theories for practical purposes. Recent past literature in behavioral finance provides evidence for the influence of stock-specific news sentiment on their related stock variables like price, return, volatility, etc. These studies mainly differ in terms of duration of influence on the stock variable, type of method used for deriving sentiment, and the stock variables affected such as returns, volatility, trading volume, etc.

The literature review reveals some gaps in research related to news sentiment and its influence on the stock market, which are discussed below:

(a)

News sentiment during extreme movement in the stock market: Noise trader theory and other derived research from it point towards limits on arbitrage and investor sentiment as factors that give rise to large deviations in prices. However, past studies on news sentiment during extreme movement in stocks are not well documented. For instance, Chowdhury et al. (2014), Ferguson et al. (2011), Zhang and Skiena (2010), etc., explain the response of stock variables like price, returns, volatility, etc., to stock-specific news sentiment without any reference to extreme movement. A study on the influence of news sentiment during extreme movement in the stock market would be a useful contribution towards extending the scope of noise trader theory by associating news sentiment with noise traders.

(b)

Aggregate news sentiment and its influence on the stock market: Ferguson et al. (2011), Heston and Sinha (2017), Mo et al. (2016), and other similar studies reveal that sentiment obtained from a company’s news influences its returns. However, they do not take into the account influence of other news released on the same day. This means that these studies are conducted by considering only company-specific news, thereby undermining the influence of other news released during the same period. Every day, investors come across various types of news which may influence them to take positions in those stocks having some linkage with the news published. When various types of news are taken together, the expectation is that they will generate an aggregate news sentiment due to sentiment induced by the news having the same or differing polarity. On a given day, aggregate news sentiment may be such that it may have a higher probability of influencing investors.

(c)

Influence of news sentiment on a portfolio of stocks: While most of the studies like Cahan et al. (2013), Ferguson et al. (2011), Huynh and Smith (2017), Mo et al. (2016), Zhang and Skiena (2010), etc., have focused on studying the influence of company-specific news sentiment on individual stocks, there is not enough evidence available from past research in which the influence of news sentiment has been examined on a portfolio of stocks. It would be worth examining the influence of news sentiment on different portfolios of stocks and finding out if a certain type of portfolio is more strongly influenced by news sentiment than others.

(d)

Influence of news sentiment from other news categories: While studies mentioned above have predominantly focused on company-specific news sentiment and its influence on respective stock variables, there is not much evidence available in past research about the sentiment of other categories of news and their influence on the stock market. Event studies using quantitative data by Geetha et al. (2011), Liew and Rowland (2016), MacKinlay (1997), etc., have revealed that events related to business, politics, economy, and international categories influence stock market returns. Hence, the expectation is that news sentiment derived from news articles related to these categories may also influence the stock market.

In summary, gaps in the literature necessitated the need to conduct a study on the influence of aggregate news sentiment and category-wise news sentiment during extreme stock market movement on a portfolio of stocks.

2.4. Research Objectives and Hypotheses

In light of the gaps discussed above, this paper focuses on the following objectives:

• To study the relationship between aggregate news sentiment and stock market returns during extreme movement in the stock market.
• To study the relationship between news sentiment of select news categories and stock market returns during extreme movement in the stock market.

Based on the research objectives framed here, the following hypotheses are proposed:

Hypothesis 1.

There is a significant relationship between aggregate news sentiment and returns of the stock market on day (t₀).

Hypothesis 2a.

The relationship between aggregate news sentiment and stock market returns is stronger on day (t₀) as compared to day (t₋₂).

Hypothesis 2b.

The relationship between aggregate news sentiment and stock market returns is stronger on day (t₀) as compared to day (t₋₁).

Hypothesis 2c.

The relationship between aggregate news sentiment and stock market returns is stronger on day (t₀) as compared to day (t₊₁).

Hypothesis 2d.

The relationship between aggregate news sentiment and stock market returns is stronger on day (t₀) as compared to day (t₊₂).

Hypothesis 3.

There is a significant relationship between news sentiment of select news categories (business, politics, economy, and international) and stock market returns on day (t₀).

Hypothesis 3a.

News sentiment from the business category has a significant influence on stock market returns on day (t₀).

Hypothesis 3b.

News sentiment from the economy category has a significant influence on stock market returns on day (t₀).

Hypothesis 3c.

News sentiment from the international category has a significant influence on stock market returns on day (t₀).

Hypothesis 3d.

News sentiment from the politics category has a significant influence on stock market returns on day (t₀).

Hypothesis 4a.

The relationship between news sentiment of select news categories and stock market returns is stronger on day (t₀) as compared to day (t₋₂).

Hypothesis 4b.

The relationship between news sentiment of select news categories and stock market returns is stronger on day (t₀) as compared to day (t₋₁).

Hypothesis 4c.

The relationship between news sentiment of select news categories and stock market returns is stronger on day (t₀) as compared to day (t₊₁).

Hypothesis 4d.

The relationship between news sentiment of select news categories and stock market returns is stronger on day (t₀) as compared to day (t₊₂).

3. Data and Methodology

3.1. Data

Data used in this study include news articles related to business, economy, politics, and international categories which are further used to derive aggregate news sentiment. Since the news categories considered here are diverse, their impact can be better examined using a portfolio of stocks rather than a single stock. In this situation, an ideal portfolio to represent the behavior of the stock market is the NIFTY index. It is considered a benchmark for the Indian stock market and hence widely followed by investors. It is a diversified 50-stock index comprising stocks from 13 sectors of the Indian economy. It tracks the behavior of a portfolio of bluechip companies, the largest and most liquid Indian securities capturing approximately 65% of its float-adjusted market capitalization (NSE 2015). Stocks of these companies generally tend to receive more attention from market participants and hence remain more active in news and social media conversations compared to midcap and smallcap stocks. Being attention-grabbers, these stocks are more likely to be influenced first, particularly when there is a significant change in the aggregate news sentiment that has a bearing on the entire stock market. Their behavior will then tend to spill over to midcap and smallcap stocks with more vigor.

In this research, daily adjusted closing prices of the NIFTY index for 12 years starting from 1 January 2008 to 31 December 2019 are used. Since the NIFTY index price series is not stationary, the difference of its logarithmic price lags is obtained to give logarithmic returns.

$$R_t=log\left(P_t\right)-log\left(P_{t+1}\right)$$

(1)

where R_t is the daily logarithmic return at time t, P_t and P_t+1 are NIFTY daily adjusted closing prices on two successive days—(t) and (t + 1), respectively.

The logarithmic return series when tested with the Augmented Dickey–Fuller (ADF) test is found to be stationary.

Another important requirement for this study is the selection of an appropriate news source. It is important to gather news published by a source that is popular, authentic, and has good coverage of relevant news. As the data points identified are from the past, the source should have a good repository of archived news. In India, some of the most popular web portals which provide freely available news archives include www.moneycontrol.com, financial-express.com, economictimes.indiatimes.com, accessed on 8 July 2023, etc. This paper utilizes the news archive from economictimes.indiatimes.com (accessed on 8 July 2023), which meets the aforementioned requirements. It is an online news portal affiliated with the Times group, renowned for publishing the largest English newspaper in the world, “The Times of India”. Additionally, it is noteworthy that their sister publication, “The Economic Times”, holds the distinction of being the second-largest English business daily globally (Source: http://www.timesmediastudies.com/aboutus.html, accessed on 8 July 2023).

3.2. Methodology

According to the noise trader theory, the presence of limits to arbitrage and sentiments plays a significant role in prompting noisy traders to adopt positions that prove challenging for arbitrageurs to counteract. As a result, substantial price deviations occur. Recognizing that extreme levels tend to attract noise trader activity, this research primarily focuses on examining the influence of aggregate news sentiment on the stock market within these extreme levels. To provide clarity in the context of this study, the following operational definitions are presented:

Extreme returns: Extreme returns are defined as logarithmic returns beyond a cutoff level of 2.58 daily standard deviation obtained using the GARCH(1,1) model.

Observation window: Period in trading days around the day with extreme stock market returns within which a comparison of the influence of news sentiment on the stock market is done daily to understand the pattern of the influence of news sentiment.

To test the hypotheses, we implement the steps of the methodology as given below:

(a) 

Identification of trading dates having extreme returns: One of the ways by which extreme movement in the stock market can be gauged is by examining volatility in returns. Since logarithmic returns are approximately normally distributed, with a confidence interval of 99%, approximately 1% of cases with extreme returns can be obtained. This confidence interval is equivalent to a cutoff level of 2.58σ (sigma), beyond which returns can be considered to be extreme. However, volatility is not constant in the return series, making it heteroscedastic. Because of this, using a point estimate of standard deviation for creating confidence intervals is not appropriate, and hence, we need a method to model such volatility. One of the ways to model volatility in this manner is by using the GARCH model. According to Poon and Granger (2003), empirical findings suggest that the GARCH(1,1) model is the most popular structure for modeling financial time series exhibiting volatility clustering. When employing the GARCH(1,1), model, the following assumptions must be satisfied: (a) Logarithmic stock market return series must be stationary. (b) Conditional variance and the model parameters must be non-negative. (c) Past volatility must influence the current conditional variance. (d) The ARCH parameter (α) depicting the lagged squared residuals and the GARCH parameter (β) depicting the lagged conditional variance must be positive. (e) The sum of the ARCH and GARCH parameters (α + β) must lie between 0 and 1 for model stability. (f) The residuals representing the difference between the observed data and the predicted values based on the model’s conditional mean equation must be independent and identically distributed (i.i.d.) random variables following a normal distribution with zero mean.

Using the GARCH(1,1) model, the standard deviation (σ) of the return series for each data point (trading date) is estimated. Trading dates with extreme returns are then filtered by setting a cutoff level of 2.58σ. This gives a total of 45 data points. Since the cause of the extreme movement is unknown, news published around the day of extreme returns is examined by keeping an observation window. Referring to the short-term event studies like Reuer and Miller (1997), (Lee 1997), etc., as cited in Oler et al. (2008), an observation window size of five trading days which includes the days t₋₂, t₋₁, t₀, t₊₁, and t₊₂—t₀ being the day of extreme returns—is set. If there is any overlap between observation windows, the data point that is chronologically older is discarded. Applying this rule, we obtain 43 valid data points whose returns are obtained and used for further analysis. The stock market is not open on all 7 days of the week. In case there are non-trading days in the observation window, news published on these days is obtained and added to the collection of news published on the next trading day. Because of this, there are 322 dates for which news articles are obtained although there are only 215 trading days corresponding to 43 data points.

(b) 

Web scraping and news categorization: After identifying trading dates with extreme returns, news articles are web-scraped from archives of the “economictimes.indiatimes.com” (accessed on 8 July 2023) portal related to four news categories: economy, international, business, and politics. Since news articles are stored in archives using a hierarchical URL (uniform resource locator) structure, identifying categories of news articles is easier. The web-scraping process starts by obtaining URLs of hyperlinks from news archives for all dates obtained above. Broader categories present in the URL include news, opinions, recommendations, interviews, etc. Only URLs having the word “news” in the URL are filtered from the rest and considered for further analysis. Others that are related to opinions, recommendations, interviews, and articles by column writers on a specific topic are filtered out and not considered relevant. URLs related to news further show that they are again categorized according to various subjects like economy, industry, politics, company, environment, sports, science, defense, international, etc. Categories like sports, science, defense, environment, etc., generally relate to articles that focus on general issues but are not necessarily related to the stock market. Therefore, all of them are brought under the general category “others” and excluded from this study. News articles from the categories “company”, “industry”, and “stocks” are brought into a single category, “business”, as news from all these categories refers to news articles related to business. A total of 42,485 news articles related to four categories economy, politics, business, and international are web-scraped and used for further processing of which 10,742 news articles belong to the business, 7976 to the economy, 3736 to the international and 20,031 to the politics category.

(c) 

Data cleaning: Web-scraped news articles obtained in the previous subsection are available in HTML format. They cannot be used in this form in the sentiment extraction process. Therefore, they are converted into plaintext by removing HTML (hypertext markup language) tags. The content of each news article is then saved in the variable “content”. Also, the date of publishing is saved as a “date” variable.

(d) 

Building text corpus and obtaining sentiment of each news article: News articles in plaintext are transferred to a text repository called a text corpus. Stop words like “is”, “an”, “shall”, “the”, etc., which are repeated and do not convey any sentiment, making them irrelevant, are hence removed. Also, extra white spaces and punctuation marks are removed. Words like “started”, “starting”, etc. are brought to their root word ‘start’ using a process called stemming. Also, the entire corpus text is converted to lowercase. The “Sentometrics” package from R (Ardia et al. 2021) is then used to compute the sentiment of each news article in the corpus. This requires the selection of an appropriate dictionary. Henry and Leone (2016) suggested using a domain-specific wordlist in the sentiment analysis of qualitative content of financial disclosure as it can significantly increase the power of the tests compared to other dictionaries. Compared to the dictionaries mentioned in the literature review, Henry’s finance dictionary, and the Loughran–McDonald financial sentiment dictionary are finance-domain-specific dictionaries. While Henry’s financial sentiment dictionary has a limited set of words, the Loughran–McDonald financial dictionary suffers from bias towards negative words as it has a higher ratio of negative words to positive words (2355/354 = 6.652) compared to Henry’s financial sentiment dictionary, which has a ratio of negative words to positive words close to 1 (85/105 = 0.809). Moreover, Meier et al. (2018) reported that for sentiment analysis, it does not seem to matter whether the extensive dictionary by Loughran and McDonald or the much shorter dictionary by Henry is used; the quality measures of both dictionaries seem to differ only marginally. Therefore, even though Henry’s dictionary has a lower number of positive and negative words, comparatively, it is more suitable for studies that examine text for both negative and positive sentiment. Pröllochs et al. (2015a, 2015b) also found a higher consensus classification share with human annotators and reliability in Henry’s financial sentiment dictionary compared to the Loughran–McDonald financial sentiment dictionary.

In order to improve the performance of the sentiment analysis process, the valence-shifting bigrams approach as described in Ardia et al. (2021) is applied wherein a bigram like “not good” receives a sentiment score of −1 due to the presence of the negator “not” instead of 1, corresponding to “good” in the case of the default unigram approach. Amplifying valence shifters, such as “very”, strengthen a polarized word by 80%, while deamplifying valence shifters, such as “hardly”, downtone a polarized word by 80%. Sentiment score is obtained by taking the net sentiment value of positive and negative words and then normalizing it by the total number of words in the news article (see, Ardia et al. 2021, for more detail). This gives a sentiment score in the range of −1 to +1. Business, economy, international, and politics are used as features to obtain news sentiment related to four news categories along with another feature, “daily” to obtain aggregate news sentiment for each day.

(e) 

Examining the relationship between news sentiment and returns: Aggregate news sentiment of each trading day in the observation window is obtained by taking an average of the sentiment polarity score of all news articles published on the same trading day. In case there is a/are non-trading day(s) before a trading day in the observation window, the sentiment polarity score of news articles published on the non-trading day(s) is aggregated with the sentiment polarity score of articles published on that trading day. Linear regression is then used to examine the strength of the relationship between returns and aggregate news sentiment on day t₀ and on day t₋₂, t₋₁, t₊₁, and t₊₂. The strength of the relationship on day t₀ is then compared with that on t₋₂, t₋₁, t₊₁, and t₊₂ days. Similarly, news sentiment of each of the business, economy, international, and politics categories is obtained for five days in the observation window. Multiple regression is then used to find whether there is any influence created by these news categories on stock market returns on day t₀. The strength of this relation is then compared with that on days t₋₂, t₋₁, t₊₁, and t₊₂.

4. Analysis and Findings

Aggregate news sentiment is denoted as “Sentiment”, and NIFTY 50 index returns are denoted as “Returns”. Variables are obtained as described in the methodology. Also, news sentiment variables related to news categories are obtained and denoted as Business, Economy, Politics, and International. The findings of this analysis are reported below:

(a)

Influence of aggregate news sentiment on stock market returns during extreme movement in the stock market:

Regression analysis using Sentiment and Returns variables is conducted to test Hypothesis 1 that there exists a significant relationship between aggregate news sentiment and stock market returns. Results of this analysis shown in

Table 2. Regression analysis of aggregate news sentiment (Sentiment) and returns of the stock market (Returns) in a five-day observation window.

	Dependent Variable
	Returns
	(t−2)	(t−1)	(t0)	(t+1)	(t+2)
	(1)	(2)	(3)	(4)	(5)
Sentiment	0.124	0.288 *	0.526 ***	0.19	0.089
	(0.801)	(1.929)	(3.963)	(1.241)	(0.574)
Observations	43	43	43	43	43
R2	0.015	0.083	0.277	0.036	0.008
Adjusted R2	−0.009	0.061	0.259	0.013	−0.016
Residual Std. Error (df = 41)	0.013	0.012	0.043	0.018	0.018
F Statistic (df = 1; 41)	0.642	3.720 *	15.703 ***	1.541	0.329

Note: Coeff., t-statistic in parenthesis; significance: * p < 0.1; ** p < 0.05; *** p < 0.01.

indicate that there is a significant relationship between aggregate news sentiment and market returns on day t₀, (R² = 0.277; F(1; 41) = 15.703; p < 0.01). Sentiment was found to be a significant predictor of Returns, (β = 0.526; t(41) = 3.963; p < 0.01). This implies support for Hypothesis 1.

To test Hypothesis 2a, Hypothesis 2b, Hypothesis 2c, and Hypothesis 2d, which examine whether the relationship on day t₀ is stronger than that on days t₋₂, t₋₁, t₊₁, and t₊₂, respectively, the strength of the relationship in each regression model corresponding to days t₋₂, t₋₁, t₊₁, and t₊₂ days is compared to that on day t₀.

Table 2 shows that while on day t₋₁, the relationship is significant at 10% confidence level (R² = 0.083; F(1; 41)= 3.720; p < 0.1), it is not significant on days t₋₂, t₊₁, and t₊₂. Also, the standardized coefficient of Sentiment, β = 0.288 on day t₋₁ is smaller compared to β = 0.526 on day t₀. This result indicates stronger relationships on day t₀ compared to that on days t₋₂, t₋₁, t₊₁, and t₊₂, respectively, thereby supporting Hypothesis 2a, Hypothesis 2b, Hypothesis 2c, and Hypothesis 2d. Using the standardized coefficient from Table 2, the equation for the relationship between Returns and Sentiment on day (t₀) can be written as follows:

$$Returns=0.526\times \left(Sentiment\right)+0.043$$

(2)

(b)

Influence of news sentiment of select news categories on market returns during extreme movement:

Having found that aggregate news sentiment has a significant relationship with stock market returns on the day of extreme stock market movement and a day before it, further, an analysis is performed to assess whether news sentiment derived from select news categories viz. business, economy, international and politics have any significant relationship with stock market returns on days t₋₂, t₋₁, t₀, t₊₁, and t₊₂. The results of this regression analysis are presented in

Table 3. Regression analysis of business, economy, international, and politics news sentiment and Returns variables in a five-day observation window.

	Dependent Variable
	Returns
	(t−2)	(t−1)	(t0)	(t+1)	(t+2)
	0.317	0.210	0.397 ***	0.137	0.033
Business	(1.649)	(1.292)	(3.010)	(0.837)	(0.206)
	−0.041	0.118	0.036	0.170	−0.101
Economy	(−0.262)	(0.700)	(0.268)	(1.078)	(−0.580)
	0.008	0.018	0.076	−0.067	0.146
International	(0.051)	(0.107)	(0.587)	(−0.421)	(0.821)
	0.263	0.291 *	0.543 ***	0.146	0.084
Politics	(1.358)	(1.849)	(4.178)	(0.902)	(0.513)
	43	43	43	43	43
Observations	0.075	0.114	0.390	0.060	0.034
R2	−0.022	0.020	0.325	−0.039	−0.068
Adjusted R2	0.013	0.012	0.041	0.019	0.019
Residual Std. Error (df = 38)	0.773	1.217	6.065 ***	0.605	0.329
F Statistic (df = 4; 38)	0.317	0.210	0.397 ***	0.137	0.033

Note: Coeff., t-statistic in parenthesis; significance: * p < 0.1; ** p < 0.05; *** p < 0.01.

.

Results on day t₀ shows that the relationship between Returns and business, economy, politics, and international is significant (Adjusted R² = 0.325; F(df = 4; 38) = 6.065; p < 0.01). This result indicates support for Hypothesis 3 that there is a significant relationship between news sentiment of select news categories and returns of the stock market on day t₀. Table 3 further reveals that the variables business (β = 0.397; p < 0.01) and politics (β = 0.543; p < 0.01) are significant predictors of Returns in the relationship between news sentiment from select news categories and Returns on day t₀. However, the variables international (β = 0.076; p > 0.1) and economy (β = 0.036; p > 0.1) are found to be insignificant. Table 3 also shows that the relationships between news sentiment of select news categories and returns on days t₋₂, t₋₁, t₊₁, and t₊₂ are not significant (p > 0.05 for all relationships). Also, it can be seen that standardized coefficients β in the relationship on day t₀ for all the variables, i.e., business, economy, international, and politics, are more than those on days t₋₂, t₋₁, t₊₁, and t₊₂. These results thus indicate that the relationship on the day t₀ is stronger and hence provides support for Hypothesis 4a, Hypothesis 4b, Hypothesis 4c, and Hypothesis 4d. Using the β of business, economy, politics, and international variables on the day t₀, the equation for returns can be written as follows:

$$\begin{gathered} Returns=0.397\times \left(Business\right)+0.036\times \left(Economy\right)+ \\ 0.076\times \left(International\right)+0.543\times \left(Politics\right)+0.041 \end{gathered}$$

(3)

5. Discussion

5.1. Theoretical Contribution Theoretical Contribution

(a) 

The results of this study lead us to derive the following theoretical contributions: Influence of aggregate news sentiment on noise traders: The results of this study found that aggregate news sentiment emanating from the news articles belonging to the business, economy, politics, and international categories influences the returns of a portfolio of stocks on the day of extreme stock market movement and a day before it. There are many news articles published every day belonging to these categories, which gives rise to noise consisting of two components—aggregate news sentiment and white noise. While aggregate news sentiment carries signals of expectation in the direction of the polarity, white noise is random and is of no use to the noise trader. This view is consistent with Black (1986), which states that noise generated because of a large number of small events is often a stronger causal factor compared to a small number of large events. When considering noise trader theory with informed investors called arbitrageurs and uninformed investors called noise traders, aggregate news sentiment is a component of noise that influences beliefs of noise traders. However, this noise may not have a sufficient amount of aggregate news sentiment to influence investors all the time because sentiment polarities of a large number of news articles may be negatively correlated giving rise to white noise. When sentiment polarities of a large number of news articles are having the same polarity, this can give rise to a net positive or negative aggregate news sentiment having greater magnitude. This can reinforce the beliefs of noise traders, encouraging them to participate in the market, causing stock prices to start deviating from their intrinsic value. Deviated prices can further bring more noise traders into the market. Also, because of their risk-seeking attitude and obtaining confidence from aggregate news sentiment complementing their beliefs, they trade large quantities. This situation is detrimental for arbitrageurs with their risk-averse attitude who find it difficult to trade due to limits on arbitrage. This inability of arbitrageurs to bet against noisy traders leads to extreme deviation in prices.

(b) 

Short-term influence of aggregate news sentiment on stock market returns: The results indicate that aggregate news sentiment has a relatively weaker influence on the stock market on day t₋₁ compared to that on day t₀ and no significant influence on days t₋₂, t₊₁, and t₊₂. This means that aggregate news sentiment has a very short-term impact that begins on the day t₋₁ and ends on the day t₀. These results are similar to those reported by Ferguson et al. (2011), wherein a higher correlation was found between stock-specific news sentiment and returns for companies in the UK. They also reported a declining impact around the news release. This study thus contributes to the existing literature by showing that aggregate news sentiment is short-lived.

(c) 

Categories of news that influence stock market returns: Of the four news categories examined, news sentiment from politics and business categories influence the stock market in the short term. Since news from the business category is comprised of news related to stocks, industry, and the market, this finding is similar to other studies like Cahan et al. (2013), Ferguson et al. (2011), Ranco et al. (2015), and Zhang and Skiena (2010), wherein they report similar evidence for company-specific news. Moreover, news sentiment from the politics category showing a significant contribution proves that investors get impacted by sentiment emanating from political situations. This is a significant contribution to the existing literature as prior studies had shown evidence of influence from company-specific news sentiment only. However, no significant influence on the stock market from news sentiment from the economy and international news categories was found in this study.

In summary, this research has contributed to the existing literature in behavioral finance by studying aggregate news sentiment as a general case.

5.2. Limitations of the Study

Some of the limitations of this study are:

This study is limited by the limitations associated with document-level sentiment analysis. For instance, document-level sentiment analysis does not capture sarcasm in sentences of news articles and hence may not reveal the correct sentiment associated with it. Also, it does not consider the context in which a sentiment word is being used. For instance, a “low” with regards to “cost” has positive sentiment, but a “low” in the context of “sales” has a negative sentiment.

Henry’s finance-specific dictionary has only 53 positive words and 44 negative words, which poses a limitation on this study if news articles contain sentiment words that do not exist in this dictionary.

6. Conclusions

The ongoing digital transformation in the finance and investment sector extends beyond just the incorporation of new technologies within organizational structures (George and Paul 2020). It also encompasses a broad spectrum of other aspects, such as the deliberate and accidental spread of news (Coffi and George 2022). This expansion of digitalization has led to an increased interest in understanding its effects on various facets of the industry. The current research was designed to answer questions about the role that overall news sentiment, originating from various categories, such as business, economics, international affairs, and politics, might have on stock market behavior. It sought to understand the potential influence of sentiments specific to these categories on the stock market. The research was not limited to a superficial investigation; it conducted empirical tests focusing on extreme stock market levels, which are typically associated with a high level of participation from noise traders, as postulated by noise trader theory.

This study has taken significant strides in exploring the short-term influence of aggregate news sentiment on stock markets and brought to light the role of extreme deviations in the stock market. It provides a fresh perspective on the noise trader theory and underscores the importance of understanding the sentiment behind news reports. These findings pave the way for more nuanced investigations into the interplay between news sentiment and stock market performance.

The findings from this study provided evidence of a significant correlation between the overall sentiment of news and stock market returns on days of extreme market volatility as well as the day preceding such events. However, the impact on the preceding day was less pronounced compared to the day of the market volatility. It is noteworthy that the study found no significant influence either two days before or two days after the occurrence of extreme stock market movements. When we further dissected the influence of news sentiments derived from different categories—business, economy, politics, and international—over a span of five trading days, we found significant impacts on stock market returns only on the day of extreme market volatility. Interestingly, the sentiment of news in the business and politics categories was identified as a significant predictor of stock market returns. In contrast, news sentiment from the economy and international categories did not appear to have a significant impact on the stock market. This demonstrates the selective influence of news sentiment across different categories in shaping stock market trends, especially during periods of extreme market volatility.

6.1. Managerial Implications

Managerial implications that can be drawn from the findings of this research are as follows:

• This study reveals that the influence of aggregate news sentiment is short-lived, not even lasting for a day after extreme returns. Therefore, a long-term investor need not worry too much about the stocks they hold if the market is affected by aggregate news sentiment. This implies that there are noise traders active in the market who trade on news sentiment obtained from news media, particularly from business and politics news categories, and drive the stock market to extreme levels. Therefore, arbitrageurs should be cautious when there is extreme pessimism or optimism in the market due to events related to business and politics.
• Mutual fund managers can take this opportunity by smartly balancing the portfolios of their clients soon after the day of extreme market movement by picking up stocks based on their fundamentals but affected by aggregate news sentiment. Since the sentiment impact is temporary and short-lived, fundamentally stronger stocks may be available at lower market prices which in the future will be more likely to appreciate.
• Traders in the derivative segment can design strategies in the index futures and options by studying business and political events lined up shortly and trade with caution.
• An intraday trader can take advantage of this situation and earn profit by taking positions like noise traders but needs to be cautious in doing so by applying stop losses for the positions taken.

6.2. Future Research

Even though this paper follows a parsimonious approach to studying aggregate news sentiment and its influence on the stock market, it has provided some encouraging results for further research. Some suggestions for further research are given below:

• The influence of aggregate news sentiment on different types of portfolios of stocks may be performed during extreme returns in the market. For instance, one may examine the influence of aggregate news sentiment on returns of sectoral indices or other portfolios based on market capitalization, beta, etc.
• One may focus news categories further to find named entities and their influence on the stock market. For instance, one may examine news sentiment related to the ruling party or opposition party and examine their influence on the stock market.
• A comparative study may be performed on the influence of positive and negative aggregate news sentiment on the stock market returns during extreme returns in the stock market.
• Instead of taking returns as a response variable, a study may be conducted to examine the influence of aggregate news sentiment and news sentiment from various news categories on other stock market variables like trading volume, liquidity, etc.
• This study has opened vistas for further research in aggregate news sentiment and its influence on stock markets in the short term. It has also brought extreme deviations in the stock market into the limelight through the lens of aggregate news sentiment following noise trader theory.

In conclusion, this research has shed new light on the role of aggregate news sentiment in driving stock market volatility and how different stakeholders can leverage this understanding for their benefit. We found that the impact of overall news sentiment is ephemeral and does not persist beyond a day following extreme market returns. This has implications for various market players. For instance, long-term investors can rest easy knowing that the stocks they hold are not significantly impacted by short-term news sentiment. Arbitrageurs, on the other hand, should tread cautiously during periods of extreme market pessimism or optimism, especially those triggered by business and political news. Moreover, mutual fund managers can seize this opportunity to strategically rebalance their clients’ portfolios after a day of extreme market movement by choosing fundamentally strong stocks impacted by overall news sentiment. As the impact of news sentiment is transient, these stocks could potentially be bought at a discount and may appreciate in value in the future (Nemes and Kiss 2021). Derivatives traders and day traders can also benefit from these insights by designing their strategies carefully.

Looking forward, this research presents several exciting avenues for future studies. Researchers may want to examine the influence of aggregate news sentiment on different types of stock portfolios during periods of extreme market returns. An investigation into the impact of news sentiment related to specific entities, such as political parties, on the stock market could also yield interesting insights (Souma et al. 2019). A comparative analysis of the influence of positive and negative aggregate news sentiment on stock market returns during periods of extreme returns could provide a deeper understanding of market dynamics. Furthermore, future research could explore the impact of aggregate news sentiment and news sentiment from various categories on other stock market variables, such as trading volume and liquidity.