Tuesday, 18th August 2020 By Oliver Kroll and Tushar Shetty
On the Madness of Men

Adorning the courtyards of many stock exchanges around the world, the Bull and the Bear are one of the most ubiquitous yet uncanny mascots in the world of Finance today. Quite why these particular beasts came to represent market sentiment remains a mystery to me; after all, while the presence of a bear could cause understandable distress in investor and layman alike, there are precious few who can claim to be filled with unbridled joy at the sight of a raging bull. 

Perhaps the first literary evidence connecting this unusual pairing to capital markets can be attributed to Alexander Pope in 1720, who wrote:

Come fill the South Sea goblet full;

The gods shall of our stock take care:

Europa pleased accepts the Bull,

And Jove with joy puts off the Bear.

Astute students of finance would find something rather interesting in this text, for it mentions one of the most infamous events in the annals of financial history: The South Sea Bubble. The South Sea Bubble was the world’s first true financial crisis, one that presaged many of the shortcomings of financial markets today, from bull runs and bubbles to insider trading and government bailouts. 

Conceived of by the British government as a way to reduce its national debt, the South Sea company was awarded monopoly rights in 1720 to trade in the lucrative South American market; a problematic prospect at best since they were at war with both Spain and Portugal, who controlled most of South America at the time. Nonetheless, it resulted in a speculative mania that gripped the nation like nothing before. Everyone from the heights of aristocracy to the lowest rungs of the fledgling middle class became investors almost overnight, and began hurling their savings into an array of increasingly disreputable schemes, including one, amusingly, “for carrying-on an undertaking of great advantage but no-one to know what it is!!”

The crash, when it inevitably came, obliterated much of the national wealth and saw many people lose their fortunes, including most notably Sir Isaac Newton, who lamented that he could “calculate the movement of the stars, but not the madness of men”.Newton, prescient as ever, highlighted one of the key foibles that has plagued financial markets ever since their conception: the vagaries of human emotion. 

Despite over 300 years of breath-taking advances in technology, instant access to massive amounts of data and the proliferation of increasingly sophisticated financial models, the fact that we still experience bubbles, bursts, manias and panics in the 21st century is a testament to the outsize role that human emotion still plays in financial markets. We’ve devoted an entire article to exploring the impact of human psychology on efficient markets, but today it would be useful to delve into one of the key conceptual approaches towards measuring the salience of emotion in financial markets: Market Sentiment.

Market Sentiment and its Impact

Market sentiment, in essence, is a qualitative measure of the mood and attitude of investors towards a certain asset or the market in general. Unlike fundamental or technical indicators, which are grounded in hard data and statistics, market sentiment is largely driven by emotions and feelings, which makes it notoriously difficult to assess and quantify. 

Amongst the innumerable factors that have a statistically significant correlation to market sentiment include: weather patterns, the phases of the moon, the number of sunshine hours in a given day and even the performance of sports teams. Market sentiment is broadly divided into two categories: A ‘positive’ market outlook with expectation of upward price movement is termed bullish, whereas a more pessimistic outlook with expectations of downward price movement is termed bearish

Normally, bull and bear markets follow a self-correcting cycle as a result of the ebb and flow of the market but on occasion, a particularly strong and sustained bull run can result in market bubbles that, when they inevitably burst, have significant economic and social repercussions. Even under normal market conditions, market sentiment is lamentably susceptible to unpredictable swings and causes much of the volatility we see in day-to-day market operations.

Investors, thus, have much to gain by understanding the impact of market sentiment and to that end have developed a plethora of tools and techniques to analyse various aspects of market sentiment. Notable examples of these include put-call ratios, the VIX Index, the Bullish Percentage Index and, the ‘Fear and Greed’ Index. While these serve as useful indicators of risk appetite and outlook, they are merely derivatives and not pure indicators of sentiment; akin to trying to estimate a household’s level of happiness by perusing their weekly grocery receipts.

Textual Attraction

A more promising approach towards tracking market sentiment is the utilization of text-based sources. Over 80% of the information on the internet is in the form of unstructured text, from the millions of news articles, blogs, posts and tweets that are shared online every single day. Unlike other fundamental data, this information is updated on a real time basis, and can provide a more comprehensive and accurate reflection of investor sentiment than mere market figures. The challenge is to mine this data in an automated and scalable way that can derive accurate and actionable information.

This, unfortunately, is harder than it may first seem. Financial data is usually presented in a standardized, numeric form that can be analysed statistically, something which text-based information is not conducive towards. Secondly, computers, for all their vaunted capabilities, have a hard time dealing with the subtleties and irregularities present in verbal and written communication. All the allegories, innuendos, subtexts and turns-of-phrase that humans can process without a hitch are meaningless jargon to a computer processor. Moreover, computers, as of writing, have considerable difficulty establishing context. This flaw was showcased by the famous ‘Hathaway Effect’, where it was observed that shares of Warren Buffet’s Berkshire Hathaway corporation inexplicably rose every time a new movie starring Anne Hathaway was released. This was eventually deduced to be the fault of rudimentary sentiment analysis software that utilized name-mentions as an indicator of investor attention, but amusingly failed to distinguish between the celebrated Hollywood actress and the massive multinational holdings corporation.

AI in Market Sentiment Analysis

The solution to this quandary lies in the ever-growing promise of Artificial Intelligence (AI). AI is already extensively used in the financial sector from calculating default risk, to combating money laundering to identifying trading patterns in the stock market. However, there are two specific subsets of AI that, when combined, have the potential to revolutionize the science of market sentiment analysis.

The first is NLP, or Natural Language Processing, a combination of AI and linguistics that aims to understand and interpret human language. Delving into the technicalities of how it functions is beyond the scope of this article, but suffice it to say that it involves envisioning words in the form of mathematical vectors, and associating meaning and connections to those vectors to build an artificial neural network that, in theory, processes words and meanings in a similar fashion to the human brain. 

If you’ve ever translated a document using Google, asked Alexa to play a song or even let your phone’s autocorrect do its ducking job, congratulations, you’re already a beneficiary of the vast potential of NLP. NLP in sentiment analysis goes a step further by attempting to derive opinions from a given text. It does so by assessing the polarity of the given text, i.e. whether it is positive, neutral or negative. In essence, NLP is slowly but steadily learning to process textual information the same way we do, except on an unimaginably larger and more efficient scale. 

It manages to do this using the power of Machine Learning (ML) and Big Data, sifting through voluminous reams of information, identifying links, patterns and correlations therein and constructing a model based on the inferences derived from its understanding of the data. The system can automatically learn from the data it processes and use it to improve the overall performance and accuracy of its models, with little to no explicit instruction by human beings. The implications of this are astounding: ML systems can not only engage in a level of complex decision making far beyond the realm of conventional computing but can learn and grow from their experience in a manner remarkably similar to human intelligence. Machine learning systems are already being used in nearly every field conceivable, and are rapidly making inroads into the turbid world of Finance.

When you combine these two prodigious disciplines, you get a system that can access and analyse trillions of bytes of written data from every corner of the world, keeping you abreast of daily changes in market sentiment while constantly improving itself in a kaizen-like manner to deliver you increasingly better insights. This field is known as Sentiment Analysis, and in the coming years will be an indispensable tool to complement fundamental and technical analysis in the arsenals of investment managers worldwide. 

Firstly, it relies on the aggregate knowledge of multiple sources of information and opinion and can thus reflect vital information that may be missing or overlooked by individual reports or analyses. Secondly, it not only reflects past and present opinion of the markets but also the future outlook, as many articles include predictions across varying time spectrums, something difficult to ascertain solely from financial reports. 

Furthermore, given the potency of market sentiment in determining price fluctuations, it can provide valuable information to investors seeking to predict price changes and perhaps even help them beat the market. Lastly, and most significantly, the sheer scale and volume of data it processes dwarfs the capacities of even the most well-resourced players in the industry, while giving smaller players easy access to hitherto unreachable data. To illustrate this, while we may read a few articles on our way to work to gauge the current mood of the market, Sentiment Analysis tools could read thousands of articles, blogs or tweets published online in the last 24 hours and mine the data to derive the cumulative global opinion on the subject within the time it takes for your morning commute.

Sentiment Analysis and Credit Risk

Investors in the equity market have already recognized the potential of Sentiment Analysis, and many already utilize it, with varying degrees of success, in their investment strategies. Its benefits in the area of credit risk, which is much less susceptible to market fluctuations, are also substantial, if less obvious. 

Banks, insurance firms and asset managers relying solely on formal financial data are missing out on the wealth of information in the form of news articles, strategy analyses and customer reviews from unorthodox sources of public opinion that can provide a much clearer picture of the current and future prospects of their investments. Asset managers can now expand beyond their traditional investment horizons into opportunities like SMEs and emerging market firms, unconstrained by previously limiting factors such as familiarity, distance and access to financial data. 

Given that these systems update and operate on a virtually real-time basis, firms can monitor sentiment on a day-to-day basis instead of relying on quarterly reports, which can act as a crucial early warning system for financial trouble. Best of all, firms can adapt Sentiment Analysis to complement their existing risk-analysis processes, without having to overhaul their entire system. At Scorable, for instance, we combine NLP systems with an AI-driven analysis of over 350 variables, ranging from market data to fundamental values to macroeconomic indicators, and derive the likelihood of credit rating changes quicker and more accurately than conventional tools in the market.

The barrage of negative news following the Covid-19 outbreak has induced a level of fear and uncertainty in the market like never before. The future of the global economy hangs in balance, and even after the crisis abates it is unclear which companies will stay afloat and which ones will go under. In these trying times, it is more pertinent than ever for investors to use technologies like Sentiment Analysis to sift through the chaos of the market, and get a clear view on the strength and viability. Despite all the gloom and doom, there will inevitably be a tomorrow, and our actions and decisions in the present will determine where we stand when tomorrow finally comes.