WILEY Advances in Financial Machine Learning

The best guide to apply ML in finance and personal investment

The title sets high expectations, but the content doesn’t live up to them. This book doesn’t really teach machine learning for trading — at least not in a serious or modern way. It presents well-known techniques like cross-validation in backtesting as if they’re new, which they’re not. Anyone with some experience in finance or data science will find much of this material basic or outdated. The author tries to introduce a strict separation between research and development in the very first chapter. Even big institutions have moved away from such failed ideas. Overall, the book promises more than it delivers. If you’re serious about learning how to use machine learning in trading, you’re better off looking elsewhere.

This book explains about a lot of important tips about how to use machine learning technique in financial data. I tried to use machine learning for my fund managing but I didn't notice about some important tips in this book. Now I'm really excited to use these important technique for analyze the stock data.

This book opens your eyes over the world of algoritmic trading. I'm giving a course of trading and it gives another point of view. I've found very interesting the approach using machine learning in a different way, threading very carefully to prevent errors that are usual, and others that are not as easy to spot when using statistics for this type of problems. Highly recommended lecture but it's a little dense, so you will be looping over the same chapter and when you break the loop, you can find some insight in after chapters.

Written for data scientists and financial professionals, not for beginners. Very insightful.

Per chi si interessa di machine learning e algoritmi per la finanza è davvero un libro ottimale. Insegna molte cose utili dal pretrattamento dei dati all'analisi dei risultati per evitare di finire in algoritmi che non funzionano. Insegna a ragionare su misure di riferimento diverse dalle classiche candele dell'analisi tecnica in modo da aver dati molto più digeribili per algoritmi di automazione.

Physics Professor Richard Feynman’s commencement address at Caltech in 1974: “But this long history of learning how to not fool ourselves – of having utter scientific integrity – is, I’m sorry to say, something that we haven’t specifically included in any particular course that I know of. We just hope you’ve caught on by osmosis. The first principle is that you must not fool yourself – and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.” This is one of the main endeavours that Marcos Lopez de Prado’s (MLdP henceforth) aims to achieve with the dissemination of his research through “Advances in Financial Machine Learning”. With more than two decades of experience in finance, both as a practitioner and as an academic researcher, MLdP’s book is a gift. Unique on its own. A blend of the practical, the theoretical, the hypothetical but verifiable, and the abstract, but always with the feet firmly on the ground, always atuned with and subservient to reality. I started to follow MLdP’s work when studying flash crashes, intraday market patterns and other high frequency topics. Ever since, the author’s diverse practice and research has contributed to make Finance more rigorous, more evidence-based, less susceptible of ‘false discoveries’. MLdP’s practice and research has spanned a diverse range of areas (e.g. liquidity, intraday price discovery and market microstructure, portfolio/asset allocation, HF strategies, correlation/cointegration, risk management, backtesting …). As an engineer and economist, I started to approach financial markets almost a couple of decades ago from an eminently fundamental, theory-based, and macro-economic perspective, expressing tradeable views through all kind of asset classes and financial instruments. From the very beginning I felt the tension between the widely accepted/taught financial models in academia –– The Black Scholes Model, The Efficient Market Hypothesis, the Capital Asset Pricing Model, Factor-based Investing … –– and professional practice, while not sure how best to introduce ‘my intuition’ around markets; trends and positioning. Over the years, I’ve tried adding tools to my arsenal (a combination of self-learning, ‘on the desk’, as well as a couple of MSc’s degrees), such as modern econometrics, Behavioural Economics/Finance, Chaos Theory, and, more recently, Machine Learning techniques. MLdP’s book (and work) is a further advancement and contribution to make Finance more rigorous, closer to reality, a field that should take as much as possible the best traits and practices from Science and the scientific method, becoming a Pseudoscience. As in MLdP’s paper “Finance as an Industrial Science” it’s stated: “[f]inance cannot become a rigorous science (in the Popperian or Lakatosian sense), however it can still operate as an “industrial science”. This article describes the scientific method by which industrial finance discovers through experimentation, and avoids false discoveries.” No fundamental laws are advocated. No models are proposed for trading/investing. No algorithmic trading strategies. No dogmas. Instead, a transparent and verifiable set of approaches and paradigms, that will allow the reader-practitioner to increase the robustness and reliability of whatever forms of interaction have with financial markets: portfolio/asset allocation, trading strategies, risk measurement … This book elicited on me similar feelings and responses as when reading other career-changing/awakening books devoted to less conventional approaches such as Multifractal Geometry and Chaos Theory, Genetic Algorithms, Quantum Finance or Machine Learning, to try to better understand and model markets. *** Behavioural Finance, Predictions and Storytelling *** Economics Nobel Laureate Daniel Kahneman (author of the popular “Thinking, Fast and Slow”) has cautioned against future prediction, overconfidence, explaining why investors and entrepreneurs should rely on data to predict outcomes, not on their intuition. Such claims and advice are grounded on his research that showed the lapses in human judgement when making economic decisions. When reading MLdP’s analogy between - the ‘over-the-ground’ gold extraction at the time when the New Continent was discovered more than five hundred years ago (“visible and over ground gold made their way from the Americas into Spain and Europe” (MLDP 2018, p.6)) and the ‘under-the-ground’ gold, the latter requiring mining technology in order to be located and extracted while accounting for the vast majority of gold on planet Earth, on the one hand, with - the ‘macro’ versus ‘microscopic‘ alpha (the former able to be identified by the savvy value-fundamental investor and/or through the old-school classical econometric and statistical models, while the latter mainly discoverable to the kind of the model-free ML approaches), on the other hand , it led me to establish a connection with Behavioural Finance’s (a subset of Behavioural Economics) main lessons and warnings, while endorsing ML techniques to uncover such dynamics. “Just like with gold, microscopic alpha doesn’t mean smaller overall profits. Microscopic alpha today is much more abundant than macroscopic alpha has ever been in history. There’s a lot of money to be made, but you will need to use heavy ML tools” (MLdP 2018, p.6) *** “Advances in Financial Machine Learning” - Hidden Gems *** MLdP’s book is full of treasures. But first, something to be thankful for are the Bibliography and Reference sections at the end of every chapter, which allows for a deeper and complementary learning of the issue/topic of interest. Same goes for the introductory Motivation sections, serving as the Abstract section in a typical Journal technical paper, as well as for the numerous Python codes throughout the book. Among the numerous treasures in the book that will prompt the reader to ponder about pre-existing beliefs and assumptions, as well as expanding and enrichening his perspectives, are: - ‘Meta-Labelling’ and the accompanying reflexions on the (hedge fund) ‘Quantamental Way’ (Chapter 3) - The ‘Stationary vs Memory Dilemma’ (Chapter 5), presenting a novel fractional-differentiated approach to make non-stationary time series stationary, balancing the trade-off between stationary (needed for ML performance) and memory retention (needed for good predictive power) - The ‘Ensemble Approach’ and the ‘Three Sources of Errors’ (Chapter 6) - The ‘Purging-Embargo’ approach (Chapter 7), aimed at reducing the likelihood that the popular k-fold Cross Validation method fails in model development applications - ‘Feature Importance’ (Chapter 8), which introduces us to the author’s ‘First Law of Backtesting’: “Backtesting is not a research tool. Feature importance is” - ‘Trading Rules’ (Chapter 13) and the characterization of the underlying price-generating stochastic process by using historical time series, minimizing the risk of ‘backtest overfitting’ - Proof this is a book written by (and for) a practitioner is illustrated in Chapter 14, without which, the previous three chapters containing a trio of ‘Back Testing Paradigms’ would have been rendered less valuable and trustworthy, as we want to assess and compare the backtests of investment strategies in a rigorous and robust manner. Chapter 14 introduces numerous backtesting metrics, aimed at measuring different features: noteworthy, the risk-adjusted performance metrics in section ‘Efficiency’, which captures the essence of MLdP’s papers on the Probabilistic and Deflated Sharpe Ratio. It’s astonishing that since the standard Sharpe Ratio was introduced by William Sharpe back in the mid-60s, some of its various shortcomings have not been addressed earlier. This chapter also introduces us to MLdP’s ‘Third Law of Backtesting’. - “Strategy risk is not the risk of the underlying portfolio […] Strategy risk is the risk that the investment strategy will fail to succeed over time” (MLdP 2018, p.217). MLdP’s ‘Strategy Probability of Failure’ algorithm in Chapter 15 adds to the investment professional arsenal of tools to monitor and assess ‘risk’, not only ‘portfolio risk’ - Much has been written, and alternative approaches been suggested, on the shortcomings of Markowitz’s (correlation-based) expected return-variance portfolio construction paradigm, but I wonder how many non-quadratic optimizers (like Markowitz’s one) the reader has encountered in his professional career. Graph Theory coupled with ML are behind the novel ‘Hierarchical Risk Parity’ (HRP) method presented in Chapter 16, which doesn’t require to invert any covariance matrix. Applications beyond portfolio construction are suggested, as well as an advancement to Ray Dalio’s Bridgewater Associates “The All Weather Story” Risk Parity paradigm. On random data, the HRP allocation is compared with the min-variance Markowitz’s allocation and a Risk Parity traditional allocation, resulting in an interesting balanced portfolio, seemingly well-position to weather systematic and idiosyncratic shocks (for which the Markowitz’s and Risk Parity portfolios are better/worst placed, respectively). - From acclaimed information theorist Claude Shannon, MLdP suggest potential applications of the ‘Entropy’ to be estimated from financial price series. Chapter 18 feels like the start to a fruitful empirical research stream, with various applications such as trading strategies development or portfolio construction. - Being one of the longest chapters (16-page long), Chapter 19 on ‘Market Microstructure’ offers a succinct insight into the extensive and productive author’s work on these matters *** Summary and Conclusion *** This is not a typical book on Machine Learning, nor the typical book on Investing, Trading Strategies, Portfolio Construction, Asset Allocation, or Risk Management. It’s a book on the application of ML on Financial matters and intricacies. It’s a book that will help to debunk some of the recent myths around Financial ML, and to identify true applications where a wise deployment of ML technology will result in a competitive advantage. It’s a book for the various stages of the financial research laboratory model (the ‘Meta-Strategy’ paradigm), like a BMW car factory) that successful quant firms seem to be implementing: “[A]mateurs develop individual strategies […] [P]rofessionals develop methods to produce strategies […] Your goal is to run a research lab like a factory, where true discoveries are not born out of inspiration, but out of methodic hard work” (MLdP 2018, p.11) “Advances in Financial Machine Learning” succeeds in fulfilling the author’s three motivation to write it, namely - To cross the divide between academia and the financial industry - To upgrade the role of Finance in society - To explain the nuances, complexities and challenges when deploying ML techniques in Investing: “[b]eating the wisdom of the crowds is harder than recognizing faces or driving cars”