Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals

UPDATE: 10/1/21 15 years later- Still the BEST BOOK on security trading you can own. IMAGINE— That in one hand you held a bag full of returns from YOUR technical trading system.... .... In the other hand you held a bag full of RANDOM returns from the market over the same period. NOW IMAGINE— That there was a 500 year old scientific tool which would allow you to COMPARE each bag of returns to determine if YOUR trading system actually works better (makes more $$$) than a system driven by pure LUCK. The tool actually TELLS YOU IF YOU ARE “WINNING” SIMPLY BECAUSE OF LUCK or YOUR TRADING SYSTEM IS ADDING ANY VALUE, ANY VALUE AT ALL, TO YOUR TRADES. Aronson’s book explains, in thorough detail, THAT scientific method. ###########. ORIGINAL REVIEW OF 12 YEARS AGO ############ What you take-away from a reading of this book really depends on where you're coming from. For STATITICIANS with an interest in trading markets--You'll likely walk away with the feeling: "Yeah, that's what I've been thinking for years, nice to see someone took the time to debunk the TA myth." For TECHNICIANS (traders) with an interest in statistics--You'll likely walk away thinking "You gotta be kidding. There are a hundred good books which can show you how to use TA to make money. This book sucks." Aronson suggests that the truth does NOT lie in between-- He is firmly in the camp of the Statistician. But a close reading of this powerful book does not "close the door" on profitable TA, it simply confirms what every first-year MBA learns: "No OBJECTIVE black-box trading strategy CONSISTENTLY beats the market AVERAGES over the LONG TERM." But hard-core TA fans take heart. You will find something interesting in this book also. I'm certain Aronson would agree with the following: "Sure you can add COMPLEX rules to the black-box, and PERIODICALLY find runs of profitability with TA. If EXCESS returns exist only in the SHORT TERM, hey, that's good enough for me." For those not willing to take the time to digest the painstakingly presented statistical concepts, there will be little value in this book-- This is a serious study with lots of math. Its not hard math. But math best understood after fully internalizing a college level stats class. Even for those with a Stats or Econometrics degree statistics are tough--both computing and interpreting statistical data takes a little work. To complicate the issue, market-related statistics are fraught with half-truths, mind-bending math, and wall-street lore. This book goes a long way to put bogus TA lore to rest by presenting a clear, scientifically sound procedure to test Technical rules. For those seriously considering buying this book let me suggest that you find Aronson's website [...] and download and read Dr. Timothy Masters' .pdf "Monte-Carlo Evaluation of Trading Systems." The document, both in tone, and sophistication, mirrors Aronson's book. If you like Masters' 43 page doc--you will love Aronson's 500+ page book. The Review I break the book up into four parts, each with various degrees of usefulness depending on your background--ie: Technician or Statistician. Below, I'll simply give what I thought was the "money-quote" from each part, plus a couple of observations for those considering buying the book. ***** Part 1: Chapter 1 - 31 pages Objective Rules and Their Evaluation "The isolated fact that a rule earned 10 percent rate of return in a back test is meaningless. If many other rules earned over 30 percent on the same data, 10 percent would indicate inferiority, whereas if all other rules were barely profitable, 10 percent might indicate superiority." - Aronson, page 23 Constructing Rules - Intro to bi-modal rule construction and trigger thresholds Data Transformation - Nice review of position-bias, log-differences and testing biases Benchmarking Rules - Good review of why "Relative-Benchmarking" is important Beating the Benchmark - Why a profitable back test is not conclusive proof of good rule ***** Part 2: Chapters 2-3 - 130 pages The Illusory Validity of Subjective Technical Analysis The Scientific Method and Technical Analysis "Statistician Harry Roberts said that technical analysts fall victim to illusion of patters and trends for two possible reasons. First, the "usual method of graphing stock prices gives a picture of successive (price) levels rather than of price changes and levels can give an artificial appearance of pattern or trend. Second, chance behavior itself produces patterns that invite spurious interpretations""-- Aronson, page 83 The Eye Deceives - Charting a random process and the representativeness heuristic Subjective vs. Objective -- Why its important to be able to "hard-code" a TA rule The Role of Logic - Why "Falsification" is more important than "Affirmation" in TA Astrology vs Astronomy - Pushing the TA boundaries from pseudo- to science ***** Part 3: Chapter 4-7 - 230 pages Statistical Analysis Hypothesis Tests and Confidence Intervals Data-Mining Bias: The Fool's Gold of Objective TA Theories of Non-Random Price Motion "Informal data analysis is simply not up to the task of extracting valid knowledge from financial markets. The data blossoms with illusionary patterns whereas valid patterns are veiled by noise and complexity. Rigorous statistical analysis is far better suited to this difficult task." - Aronson, page 172 Hypothesis Testing--Good review of probability and statistical inference The Traditional Solution - Actually put your college-level stats knowledge to use The Monte-Carlo Solution - Putting computer randomization and re-sampling to work The Data-Mining Problem -- Why traditional MC solutions don't work Inefficient Markets - How, where and why profitable TA rules should STILL exist ***** Part 4: Chapter 8-9 - 100 pages Case Study of Rule Data Mining for the S&P 500 Case Study Results and the Future of TA "Few rule studies in popular TA apply significance tests of any sort. Thus, they do not address the possibility that rule profits may be due to ordinary sampling error. This is a serious omission, which is easily corrected by applying ordinary hypothesis tests." - Aronson. page 449 The Operators - Reviews: channel-break-outs, moving averages, channel-normalization The Indicators -- Reviews: price, volume, breadth, spreads, yields The Rules - Reviews: trends, inverse trends, reversions, divergence The Results - Analysis of why 0 of the 6,402 tested rules produced no significant results The Bottom Line Aronson's book reminds me of that masked-magician on TV who has given away the secrets to all the best stage illusions. Novice magicians and apprentice conjurers will undoubtedly be "pissed-off." But true professionals are liberated. The best in the field can focus on new and potentially MORE exciting illusions--not the same old tricks.

This is a tough book to review. The material covered is simply not covered anywhere else I've found, and it is absolutely crucial in building a scientific approach to building trading systems. As such, you pretty much have to read this book if you want to trade and not lose your shirt. On the other hand, it's got some fairly serious flaws. The author seems to be a "seat of his pants" proprietary trader who eventually got science-religion, and became a scientific trader. As such, it is probably more or less oriented towards people like him; people who may not have been exposed to ideas like "standard deviation" or "statistical distribution" before they read this book. I'm not sure it succeeds in explaining this issues. I found the explanations to be excellent and extremely clear; but I have a Ph.D. in physics, and have been thinking in these terms since I was a teenager. Will some 40 year old knuckle-dragger who has never heard of the Student-T distribution get anything from this? I don't know. I kind of suspect he won't. Can I think of a better way to explain these concepts to an older student coming to the ideas for the first time? Nope; certainly not. I'd probably just give them this book and hope for the best. The other flaw is also kind of a strength: the author "talks" too much. This book is over 500 pages long. The crucial material in it; the explanation of White's reality check and the Monte-Carlo analogue by Tim Masters is really only a couple of pages. Most of the other text is interesting and well written as the author is a learned and experienced man, but, well, Aronson could use an editor. I believe Ambrose Bierce once reviewed a book with, "The covers of this book are too far apart." This is unfortunately sort of true here. I'll say it again: this book is the only one I know of which deals seriously with the issue of data mining bias. This is what separates the men from the boys. It's easy to build signal processing techniques which find real signal in financial time series (and yes, they work a lot better than the lame TA signals the author uses), but more difficult to find out when these techniques are lying. I'm planning on giving away a piece of software you can use to find some kinds of signal fairly painlessly: I probably won't give away the "reality check" stuff, because that's the hard part. What would I have liked in the ideal world? Maybe a little less Popper and bad history of science, drop the specific test he did and add more technical stuff on the various forms of reality check. For example, the reality checks described here deal exclusively with simple entry points: how do you deal with more complex entries and exits and money management? There are ways of doing this for certain, but this book is only the beginning in figuring them out. What do you do about signals which have the Markov property, or, for example, what do you do with signal-finding algorithms which have the bootstrap property baked into them already? What about a data mining reality check for Sharpe ratio? What do you do when you have a signal with varying probability of being true? By this, I mean, you may have a signal you have determined has a 51% chance of being correct, and in some cases, you may have a signal which you know has a 54% chance of being correct (you probably will never have a 99% correct signal; not in finance anyway); what you do with such signals is different. Sometimes you have a signal where you have no idea what the probability of success is: these need to also be handled differently. There are also issues with correlations between trading systems, bet sizing ... I supposed there are lots of issues like this which I would have liked to see addressed in a book like this, but until someone writes such a book, we have to make due with this book, Grinold and Kahn and SSRN. Speaking of Grinold and Kahn, while this is probably outside the author's field of expertise, application of these ideas to classical macro/microeconomic models used in the "alpha plus" investment funds would have been incredibly awesome. Those fields use plain old regression to build their accounting based models. G & K's book doesn't mention much beyond Student-T tests for backtesting (Elton and Gruber does mention the bootstrap without telling much on how to use one). Applying the machinery of White's reality check to this "arbitrage pricing model" sort of thing would have been a huge win: far more interesting than using it on various technical analysis methods as he does in the second to last chapter. Anyway, that's how I would have rolled.

Testo didattico scritto da uno dei tanti teorici di borsa che probabilemnte non hanno mai aperto una posizione in vita loro. La prima parte consiste in una demolizione sistematica di tutta l'impalcatura teorica dell'analisi tecnica classica e francamente non dice cose del tutto sbagliate anche se si prende la briga di scomodare filosofi e naturalisti. Nella seconda parte va, in modo piuttosto inconcludente e contradditorio, alla ricerca del sacro Graal, quello che tutti vorrebbero avere ma che forse soltanto un veggente dotato di capacità metafisiche potrebbe avere. Da aggiungere alla collezione, come molti altri del genere.

An overdue examination of technical analysis from a scientific perspective. For too long TA practitioners have used overly vague terminology and methods for predicting the market. Presumably many thousands of investors have tried to put these into effect losing themselves money and causing heartache in the process. There is a role for TA but one that is based on reasonable testable propositions, this book is a major step in this direction.The sections on statistics were some of the clearest explanations on this topic that i have ever seen and helped me with my finance masters. This is not the typical TA book with 20 surefire ways to double your profits. It is more realistic but at the same time a little depressing ....success in trading (as in life) requires a lot of hard work , study and training ...there are no shortcuts. NOTE: this is not the typical TA book, do not buy it if you are looking for that killer chart formation (that worked in the past but probably won't in the future) but do buy it if you genuinely want to learn something about the market and how to backtest stock movements. From this perspective this is the best book available and should be on the shelf of any serious investor.

Good introduction on reviewing Technical Analysis from the scientific perspective

There's a lot of useful material in this book - there's also a lot of pseudo scientific bigotry. The scientific method is held up as the Holy Grail and without doubt it has it's uses - but it's only part of the story. As Einstein said - imagination is the most important thing. Once you've hit upon some innovative idea then the scientific method is merely a process of shaping it up. Half the book can be dismissed as the author attempting to constrain the world within the scientific method - the rest of the book is very useful - particularly for avoiding the hunt for fool's gold.