Error Message

Research and publications Finance and trading

Supervised and unsupervised technical analysis 2020-2021

Used supervised and unsupervised learning to build a robust labeling system that was used to enhance an AI based trading strategy (while also increasing explainability)

Technical Analysts use charts to make decisions about the stock market. I decided to research how well chart patterns predict future prices
My background

My prior experience studying Finance and Accounting, as well as working briefly at both an Investment Bank and Hedge Fund. The I-bank provided buy-side advisory services to private equity firms, and the Hedge Fund had a long-only policy. Also, in college I worked at Merril Lynch and UBS, so I have seen first hand several different approaches to valuing financial assets. Obviously, with a long-only strategy, the goal is to find "under valued" assets and to buy them. Presumably, if the market "over values" the assets after you purchased them...you should sell them.

Qualitative analysis looks at the underlying qualities of a company.

Technical analysis looks at chart patterns and derived analytical tools (such as moving averages, volatility patterns, volume spikes, etc...)

Qualitative Analysts (like Warren Buffet) typically dismiss technical analysis. Technical Analysts swear by their techniques though successful strategies are rarely shared publicly

The Hedge Fund that I worked at did a combination of Technical and Qualitative analysis, and generally required that both approaches confirmed their investment hypothesises. The firm was successful, the assets under management grew from about $2 billion to $4 bilion in the brief time that I was there, and their investment strategies worked well.

...and now onto the research!

References: 7 technical analysis tools, 4 common indicators for trend trading, 12 technical indicators
Some preliminary research
Common aspects technical traders explore are:
Leading indicators
Lagging indicators
Momentum/trends:
  • Moving Averages
  • Moving Average Convergance & Divergence (MACD)
  • Parabolic Stop and Reverse (SAR)
  • Stochastic Oscillator
  • Relative Strength Index (RSI)
  • Commodity Channel Index (RSI)
  • Average Directional Index (ADX)
Volatility
  • Standard deviation
  • Bollinger bands
  • Average True Range
Volume
  • Volume Rate of Change
  • Chaikin Oscillator
  • On Balance Volume (OBV)
  • Accumulation/Distribution (A/D) Line
Plus chart patterns are analyzed for indications of break-up, break-down or expected sideways movement:
Example patterns
Common Strategies:
Buy/sell points based on "known" patterns
Riding waves (momentum/swing trading)
Regression to the mean (reversal strategies)
Trading support or resistance lines based on past prices or Moving Averages (for breakout or reversal strategies)
What have I tried in the past?

Many years ago I hand coded "trade signals" such as "when the 30 day Moving Average (MA) crosses the 90 day MA" (as shown in the image below)

Based on my past research, with enough backtesting, you CAN find SIMPLE strategies that SEEM to work well on various stocks, commodities, currencies or indices. It was fun to imagine that I found some brilliant strategy, but this approach very much runs the risk of "overfitting" aka "finding parameters that worked in the past, but have no reason to continue working in the future"
References: Time series Convolutional Neural Network, K-Means clustering with python
What am I planning on trying now?
1) Supervised labeling of chart patterns using synthetic data
2) Unsupervised labeling of chart patterns
3) Comparison of performance between supervised and unsupervised labeling of chart patterns
4) Addition of OTHER technical indicators
Supervised labeling:
"Synthetic data" is artificially created information that can be used to TRAIN a neural network. For example: I want to teach a neural network to recognize basic chart patterns. To do so, I could 1) manually label actual charts, or 2) generate chart data from basic templates
A basic template consists of an array (or list) of data points:
prices = [1.0, 0.99, 0.98, 0.97, 0.96, 0.95, 0.945, 0.942, 0.94, 0.94, 0.942, 0.945, 0.95, 0.96, 0.965, 0.97, 0.98, 0.99, 1.0, 1.0, 1.0, 1.005, 1.01, 1.012, 1.015, 1.018, 1.02, 1.022, 1.025, 1.03]
If those points are graphed, they look like a common pattern known as the "cup and handle":
Now thousands of variations are randomly generated, with X axis and Y axis variation, as well as randomness added to the movement between days!
A sample of 16 randomly generated synthetic chart patterns used for training a neural network
Also, thousands of variations are randomly generated by grabbing daily returns and randomizing them into charts. Currently, there is NO X axis variation
A sample of 16 randomly generated synthetic chart patterns used to identify the label "noshape"
Note: Some portion of the ABOVE charts will by COINCIDENCE resemble the TARGET LABEL: "cup_and_handle", but for now this might be an acceptable approach for generating synthetic data for 2 labels: "cup_and_handle" and "noshape"
Training supervised network
Using a BASIC Neural Network
Using a 1D Convolutional Neural Network
Using an Encoder/Decoder LSTM network with attention
RESULTS will be looked at on the training data, MORE generated synthetic data, and ACTUAL stock market data
Augmenting the data...to see if it helps
...by adding extra columns such as:
Slope
Slope offset
Slope Double offset
Note: I pass in an array of SLOPE PARAMETERS, such as [2,5,10]
For each SLOPE variable, I calculate the following 3 features: Slope, SlopeOffset, and SlopeDoubleOffset. Thus, an array of [2,5, and 10] adds 9 new features (aka columns)
if T0 is TODAY's PRICE, then Slope offset of 10 would be (PriceT-10 - PriceT-20)/PriceT-10
if T0 is TODAY's PRICE, then Slope double offset of 10 would be (PriceT-(10*2) - PriceT-(10*2-10))/PriceT-(10*2) (i.e. the slope from 20 days ago to 30 days ago)
Note: May want to explore doing slopes using at least a 2 day MA, to provide some smoothing. Currently, the extra columns
Next part
This is a 2 part series