Why Stock Analysis Technical Indicators Don't Work?

Since ages, we humans, the so-called intelligent biological material, have been working on solving one fundamental problem and that problem is to predict the future. And if that intelligent material is also investing in the stock market, then predicting the stock market becomes the universal goal for that person.

Heisenberg's uncertainty principle says we cannot be certain about the current position and the velocity at the same time. Technical analysis Gurus must be experiencing this frequently while analysing stocks. Using their technical indicators if they are very certain about the current position ( value ) of a stock then according to the uncertainty principle, the future predicted by the same technical indicators is totally uncertain. Conversely, if they are sure about the future, then there is uncertainty about the current valuation of the stock.

However, Albert Einstein has a totally different opinion. Einstein said “God does not play dice with humans!”. There is no inherent uncertainty in understanding the current situation and predicting the future situation; we might simply not know the universal formula yet, using which we can predict both. For those of you who still believe that the stock market can be predicted with certainty, don't lose hope. I also think the same, so we will ignore Heisenberg and only believe in Einstein. 😊

Now, coming to the main topic: “Why don't technical indicators work? “ Let's examine where the problem lies. When I asked ChatGPT, “How many technical indicators are there for stock market analysis?” ChatGPT replied that there are around 150 widely recognized and used technical indicators. The number 150 proves that none of these indicators is complete. If one of them were predicting correctly, we wouldn't need the other 149.

Some people say they use technical indicators in combinations. Practically speaking, it is very difficult to find the best possible combination from a set of 150 indicators, as there exist billions of such combinations. To determine the total number of possible combinations for selecting between 1 and 150 indicators out of a set of 150, we need to sum the number of combinations for each possible number of selections. That sum of combinations is an astronomical number: 1.427×10^30 trillions! So, practically, it is impossible to find the best possible combination of indicators that will always work.

When I say those technical indicators are incomplete, I mean that those indicators address only a minor subset of the problem and that too using a limited context. For example, overlay indicators such as Bollinger Bands only consider the standard deviation from the moving average and no other context while predicting the future. Similarly, momentum indicators do not take sentiment analysis into consideration while calculating future stock movements. Without considering the complete context for stock data analysis, it is nearly impossible to predict future stock movements.

So, what is the solution now? Can we come up with a universal formula to predict the future (at least for stock market)? I do believe that someday we will discover that perfect universal formula, but until then, there is another way to approach this. And that way is to learn from the history, is to learn from the history with context. In the case of the stock market, the historical stock market data itself is context. There is no difference between the stock data and stock context. There is a lot of information embedded in stock market patterns. The analysis and categorization of those patterns can provide us with valuable insights.

Let me explain this in details. If there ever exists a perfect formula to predict the stock market future, it will be like this: Y = f(x) + C, where Y is the future stock movement, C is the context, and f(x) is some complex mathematical expression that is not known yet. Now, suppose you want to predict the stock movement for the next n days using the same formula and considering our proposition that "in the case of the stock market, the stock market data is itself a context." The formula can be written as:

( next n days of stock prediction ) = f(x) + ( last n days of stock data )

So, we are saying that the next n days of stock movement depend on the last n days of stock performance. Now, if we search in the historical data of all similar stocks for the pattern generated by the last n days of stock data, and then for the matched stocks, we would know what happened with them in the next n days since it's historical data. Hence, we have now changed the dynamics of the problem. We have changed the problem from a mathematical regression problem to a pattern-matching problem.

Now, let's see how to solve the stock prediction problem after modifying it to pattern matching problem. There are some algorithms such as Dynamic Time Wrapping (DTW) which are used to find the similarities between two time series datasets. With some highly sophisticated normalization techniques and a curated version of the DTW algorithm, we can identify the historical stock datasets that closely match the current stock pattern. Then, by using the next n days of the matched stock data, we can predict the current stock movement.

Author - Sumit Khedkar,  Sumit Khedkar on Linkedin